Deep Lake retriever example analyzing Twitter the-algorithm source code (#2602)

Improvements to Deep Lake Vector Store
- much faster view loading of embeddings after filters with
`fetch_chunks=True`
- 2x faster ingestion
- use np.float32 for embeddings to save 2x storage, LZ4 compression for
text and metadata storage (saves up to 4x storage for text data)
- user defined functions as filters

Docs
- Added retriever full example for analyzing twitter the-algorithm
source code with GPT4
- Added a use case for code analysis (please let us know your thoughts
how we can improve it)

---------

Co-authored-by: Davit Buniatyan <d@activeloop.ai>
This commit is contained in:
Davit Buniatyan
2023-04-09 12:29:47 -07:00
committed by GitHub
parent 5c0c5fafb2
commit aaac7071a3
5 changed files with 527 additions and 26 deletions

View File

@@ -71,6 +71,8 @@ The above modules can be used in a variety of ways. LangChain also provides guid
- `Querying Tabular Data <./use_cases/tabular.html>`_: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.
- `Code Understanding <./use_cases/code.html>`_: If you want to understand how to use LLMs to query source code from github, you should read this page.
- `Interacting with APIs <./use_cases/apis.html>`_: Enabling LLMs to interact with APIs is extremely powerful in order to give them more up-to-date information and allow them to take actions.
- `Extraction <./use_cases/extraction.html>`_: Extract structured information from text.