Commit Graph

3 Commits

Author SHA1 Message Date
Hasegawa Yuya
e08961ab25
Fixed openai embeddings to be safe by batching them based on token size calculation. (#991)
I modified the logic of the batch calculation for embedding according to
this cookbook

https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb
2023-02-15 23:02:32 -08:00
Harrison Chase
91c6cea227
Harrison/batch embeds (#972)
Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com>
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net>
2023-02-10 06:59:50 -08:00
Harrison Chase
76aff023d7
FAISS and embedding support (#48)
also adds embeddings and an in memory docstore
2022-11-01 21:29:39 -07:00