Convert Chain to a Chain Factory (#4605)

## Change Chain argument in client to accept a chain factory

The `run_over_dataset` functionality seeks to treat each iteration of an
example as an independent trial.
Chains have memory, so it's easier to permit this type of behavior if we
accept a factory method rather than the chain object directly.

There's still corner cases / UX pains people will likely run into, like:
- Caching may cause issues
- if memory is persisted to a shared object (e.g., same redis queue) ,
this could impact what is retrieved
- If we're running the async methods with concurrency using local
models, if someone naively instantiates the chain and loads each time,
it could lead to tons of disk I/O or OOM
This commit is contained in:
Zander Chase
2023-05-13 02:13:21 +00:00
committed by GitHub
parent ed0d557ede
commit 0c6ed657ef
3 changed files with 197 additions and 61 deletions

View File

@@ -218,7 +218,7 @@ async def test_arun_on_dataset(monkeypatch: pytest.MonkeyPatch) -> None:
num_repetitions = 3
results = await client.arun_on_dataset(
dataset_name="test",
llm_or_chain=chain,
llm_or_chain_factory=lambda: chain,
concurrency_level=2,
session_name="test_session",
num_repetitions=num_repetitions,