bump version to 199 (#6102)

This adds implementation of MMR search in pinecone; and I have two
semi-related observations about this vector store class:
- Maybe we should also have a
`similarity_search_by_vector_returning_embeddings` like in supabase, but
it's not in the base `VectorStore` class so I didn't implement
- Talking about the base class, there's
`similarity_search_with_relevance_scores`, but in pinecone it is called
`similarity_search_with_score`; maybe we should consider renaming it to
align with other `VectorStore` base and sub classes (or add that as an
alias for backward compatibility)

#### Who can review?

Tag maintainers/contributors who might be interested:
 - VectorStores / Retrievers / Memory - @dev2049

.github/ISSUE_TEMPLATE/bug-report.yml

@@ -46,7 +46,7 @@ body:
         - @agola11
 
         Tools / Toolkits
-        - @vowelparrot
+        - ...
 
       placeholder: "@Username ..."
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,5 +1,3 @@
-# Your PR Title (What it does)
-
 <!--
 Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.
 
@@ -14,7 +12,7 @@ Finally, we'd love to show appreciation for your contribution - if you'd like us
 
 Fixes # (issue)
 
-## Before submitting
+#### Before submitting
 
 <!-- If you're adding a new integration, please include:
 
@@ -28,9 +26,9 @@ etc:
 https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
 
-## Who can review?
+#### Who can review?
 
-Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
+Tag maintainers/contributors who might be interested:
 
 <!-- For a quicker response, figure out the right person to tag with @
 
@@ -50,7 +48,7 @@ Community members can review the PR once tests pass. Tag maintainers/contributor
   - @agola11
 
   Agents / Tools / Toolkits
-  - @vowelparrot
+  - @hwchase17
 
   VectorStores / Retrievers / Memory
   - @dev2049

.gitignore

@@ -149,4 +149,7 @@ wandb/
 
 # integration test artifacts
 data_map*
-\[('_type', 'fake'), ('stop', None)]
+\[('_type', 'fake'), ('stop', None)]
+
+# Replit files
+*replit*

README.md

@@ -2,6 +2,7 @@
 
 ⚡ Building applications with LLMs through composability ⚡
 
+[![Release Notes](https://img.shields.io/github/release/hwchase17/langchain)](https://github.com/hwchase17/langchain/releases)
 [![lint](https://github.com/hwchase17/langchain/actions/workflows/lint.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/lint.yml)
 [![test](https://github.com/hwchase17/langchain/actions/workflows/test.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/test.yml)
 [![linkcheck](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml/badge.svg)](https://github.com/hwchase17/langchain/actions/workflows/linkcheck.yml)
@@ -12,6 +13,8 @@
 [![Open in Dev Containers](https://img.shields.io/static/v1?label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/hwchase17/langchain)
 [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/hwchase17/langchain)
 [![GitHub star chart](https://img.shields.io/github/stars/hwchase17/langchain?style=social)](https://star-history.com/#hwchase17/langchain)
+[![Dependency Status](https://img.shields.io/librariesio/github/hwchase17/langchain)](https://libraries.io/github/hwchase17/langchain)
+[![Open Issues](https://img.shields.io/github/issues-raw/hwchase17/langchain)](https://github.com/hwchase17/langchain/issues)
 
 
 Looking for the JS/TS version? Check out [LangChain.js](https://github.com/hwchase17/langchainjs).

docs/_static/js/mendablesearch.js

@@ -37,6 +37,7 @@ document.addEventListener('DOMContentLoaded', () => {
         style: { darkMode: false, accentColor: '#010810' },
         floatingButtonStyle: { color: '#ffffff', backgroundColor: '#010810' },
         anon_key: '82842b36-3ea6-49b2-9fb8-52cfc4bde6bf', // Mendable Search Public ANON key, ok to be public
+        cmdShortcutKey:'j',
         messageSettings: {
           openSourcesInNewTab: false,
           prettySources: true // Prettify the sources displayed now

docs/additional_resources/deploy_llms.rst

@@ -0,0 +1,137 @@
+
+===========================
+Deploying LLMs in Production
+===========================
+
+In today's fast-paced technological landscape, the use of Large Language Models (LLMs) is rapidly expanding. As a result, it's crucial for developers to understand how to effectively deploy these models in production environments. LLM interfaces typically fall into two categories:
+
+- **Case 1: Utilizing External LLM Providers (OpenAI, Anthropic, etc.)**
+    In this scenario, most of the computational burden is handled by the LLM providers, while LangChain simplifies the implementation of business logic around these services. This approach includes features such as prompt templating, chat message generation, caching, vector embedding database creation, preprocessing, etc.
+
+- **Case 2: Self-hosted Open-Source Models**
+    Alternatively, developers can opt to use smaller, yet comparably capable, self-hosted open-source LLM models. This approach can significantly decrease costs, latency, and privacy concerns associated with transferring data to external LLM providers.
+
+Regardless of the framework that forms the backbone of your product, deploying LLM applications comes with its own set of challenges. It's vital to understand the trade-offs and key considerations when evaluating serving frameworks.
+
+Outline
+=======
+
+This guide aims to provide a comprehensive overview of the requirements for deploying LLMs in a production setting, focusing on:
+
+- `Designing a Robust LLM Application Service <#robust>`_
+- `Maintaining Cost-Efficiency <#cost>`_
+- `Ensuring Rapid Iteration <#iteration>`_
+
+Understanding these components is crucial when assessing serving systems. LangChain integrates with several open-source projects designed to tackle these issues, providing a robust framework for productionizing your LLM applications. Some notable frameworks include:
+
+- `Ray Serve <../integrations/ray_serve.html>`_
+- `BentoML <https://github.com/ssheng/BentoChain>`_
+- `Modal <../integrations/modal.html>`_
+
+These links will provide further information on each ecosystem, assisting you in finding the best fit for your LLM deployment needs.
+
+Designing a Robust LLM Application Service
+===========================================
+.. _robust:
+
+When deploying an LLM service in production, it's imperative to provide a seamless user experience free from outages. Achieving 24/7 service availability involves creating and maintaining several sub-systems surrounding your application.
+
+Monitoring
+----------
+
+Monitoring forms an integral part of any system running in a production environment. In the context of LLMs, it is essential to monitor both performance and quality metrics.
+
+**Performance Metrics:** These metrics provide insights into the efficiency and capacity of your model. Here are some key examples:
+
+- Query per second (QPS): This measures the number of queries your model processes in a second, offering insights into its utilization.
+- Latency: This metric quantifies the delay from when your client sends a request to when they receive a response.
+- Tokens Per Second (TPS): This represents the number of tokens your model can generate in a second.
+
+**Quality Metrics:** These metrics are typically customized according to the business use-case. For instance, how does the output of your system compare to a baseline, such as a previous version? Although these metrics can be calculated offline, you need to log the necessary data to use them later.
+
+Fault tolerance
+---------------
+
+Your application may encounter errors such as exceptions in your model inference or business logic code, causing failures and disrupting traffic. Other potential issues could arise from the machine running your application, such as unexpected hardware breakdowns or loss of spot-instances during high-demand periods. One way to mitigate these risks is by increasing redundancy through replica scaling and implementing recovery mechanisms for failed replicas. However, model replicas aren't the only potential points of failure. It's essential to build resilience against various failures that could occur at any point in your stack.
+
+
+Zero down time upgrade
+----------------------
+
+System upgrades are often necessary but can result in service disruptions if not handled correctly. One way to prevent downtime during upgrades is by implementing a smooth transition process from the old version to the new one. Ideally, the new version of your LLM service is deployed, and traffic gradually shifts from the old to the new version, maintaining a constant QPS throughout the process.
+
+
+Load balancing
+--------------
+
+Load balancing, in simple terms, is a technique to distribute work evenly across multiple computers, servers, or other resources to optimize the utilization of the system, maximize throughput, minimize response time, and avoid overload of any single resource. Think of it as a traffic officer directing cars (requests) to different roads (servers) so that no single road becomes too congested.
+
+There are several strategies for load balancing. For example, one common method is the *Round Robin* strategy, where each request is sent to the next server in line, cycling back to the first when all servers have received a request. This works well when all servers are equally capable. However, if some servers are more powerful than others, you might use a *Weighted Round Robin* or *Least Connections* strategy, where more requests are sent to the more powerful servers, or to those currently handling the fewest active requests. Let's imagine you're running a LLM chain. If your application becomes popular, you could have hundreds or even thousands of users asking questions at the same time. If one server gets too busy (high load), the load balancer would direct new requests to another server that is less busy. This way, all your users get a timely response and the system remains stable.
+
+
+
+Maintaining Cost-Efficiency and Scalability
+============================================
+.. _cost:
+
+Deploying LLM services can be costly, especially when you're handling a large volume of user interactions. Charges by LLM providers are usually based on tokens used, making a chat system inference on these models potentially expensive. However, several strategies can help manage these costs without compromising the quality of the service.
+
+
+Self-hosting models
+-------------------
+
+Several smaller and open-source LLMs are emerging to tackle the issue of reliance on LLM providers. Self-hosting allows you to maintain similar quality to LLM provider models while managing costs. The challenge lies in building a reliable, high-performing LLM serving system on your own machines. 
+
+Resource Management and Auto-Scaling
+------------------------------------
+
+Computational logic within your application requires precise resource allocation. For instance, if part of your traffic is served by an OpenAI endpoint and another part by a self-hosted model, it's crucial to allocate suitable resources for each. Auto-scaling—adjusting resource allocation based on traffic—can significantly impact the cost of running your application. This strategy requires a balance between cost and responsiveness, ensuring neither resource over-provisioning nor compromised application responsiveness.
+
+Utilizing Spot Instances
+------------------------
+
+On platforms like AWS, spot instances offer substantial cost savings, typically priced at about a third of on-demand instances. The trade-off is a higher crash rate, necessitating a robust fault-tolerance mechanism for effective use.
+
+Independent Scaling
+-------------------
+
+When self-hosting your models, you should consider independent scaling. For example, if you have two translation models, one fine-tuned for French and another for Spanish, incoming requests might necessitate different scaling requirements for each.
+
+Batching requests
+-----------------
+
+In the context of Large Language Models, batching requests can enhance efficiency by better utilizing your GPU resources. GPUs are inherently parallel processors, designed to handle multiple tasks simultaneously. If you send individual requests to the model, the GPU might not be fully utilized as it's only working on a single task at a time. On the other hand, by batching requests together, you're allowing the GPU to work on multiple tasks at once, maximizing its utilization and improving inference speed. This not only leads to cost savings but can also improve the overall latency of your LLM service.
+
+
+In summary, managing costs while scaling your LLM services requires a strategic approach. Utilizing self-hosting models, managing resources effectively, employing auto-scaling, using spot instances, independently scaling models, and batching requests are key strategies to consider. Open-source libraries such as Ray Serve and BentoML are designed to deal with these complexities. 
+
+
+
+Ensuring Rapid Iteration
+========================
+
+.. _iteration:
+
+The LLM landscape is evolving at an unprecedented pace, with new libraries and model architectures being introduced constantly. Consequently, it's crucial to avoid tying yourself to a solution specific to one particular framework. This is especially relevant in serving, where changes to your infrastructure can be time-consuming, expensive, and risky. Strive for infrastructure that is not locked into any specific machine learning library or framework, but instead offers a general-purpose, scalable serving layer. Here are some aspects where flexibility plays a key role:
+
+Model composition
+-----------------
+
+Deploying systems like LangChain demands the ability to piece together different models and connect them via logic. Take the example of building a natural language input SQL query engine. Querying an LLM and obtaining the SQL command is only part of the system. You need to extract metadata from the connected database, construct a prompt for the LLM, run the SQL query on an engine, collect and feed back the response to the LLM as the query runs, and present the results to the user. This demonstrates the need to seamlessly integrate various complex components built in Python into a dynamic chain of logical blocks that can be served together.
+
+Cloud providers
+---------------
+
+Many hosted solutions are restricted to a single cloud provider, which can limit your options in today's multi-cloud world. Depending on where your other infrastructure components are built, you might prefer to stick with your chosen cloud provider.
+
+
+Infrastructure as Code (IaC)
+---------------------------
+
+Rapid iteration also involves the ability to recreate your infrastructure quickly and reliably. This is where Infrastructure as Code (IaC) tools like Terraform, CloudFormation, or Kubernetes YAML files come into play. They allow you to define your infrastructure in code files, which can be version controlled and quickly deployed, enabling faster and more reliable iterations.
+
+
+CI/CD
+-----
+
+In a fast-paced environment, implementing CI/CD pipelines can significantly speed up the iteration process. They help automate the testing and deployment of your LLM applications, reducing the risk of errors and enabling faster feedback and iteration.

docs/dependents.md

@@ -2,191 +2,230 @@
 
 Dependents stats for `hwchase17/langchain`
 
-[![](https://img.shields.io/static/v1?label=Used%20by&message=5152&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(public)&message=172&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(private)&message=4980&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
-[![](https://img.shields.io/static/v1?label=Used%20by%20(stars)&message=17239&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
+[![](https://img.shields.io/static/v1?label=Used%20by&message=7484&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
+[![](https://img.shields.io/static/v1?label=Used%20by%20(public)&message=212&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
+[![](https://img.shields.io/static/v1?label=Used%20by%20(private)&message=7272&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
+[![](https://img.shields.io/static/v1?label=Used%20by%20(stars)&message=19095&color=informational&logo=slickpic)](https://github.com/hwchase17/langchain/network/dependents)
 
-[update: 2023-05-17; only dependent repositories with Stars > 100]
+[update: 2023-06-05; only dependent repositories with Stars > 100]
 
 
 | Repository | Stars  |
 | :--------  | -----: |
-|[openai/openai-cookbook](https://github.com/openai/openai-cookbook) | 35401 |
-|[LAION-AI/Open-Assistant](https://github.com/LAION-AI/Open-Assistant) | 32861 |
-|[microsoft/TaskMatrix](https://github.com/microsoft/TaskMatrix) | 32766 |
-|[hpcaitech/ColossalAI](https://github.com/hpcaitech/ColossalAI) | 29560 |
-|[reworkd/AgentGPT](https://github.com/reworkd/AgentGPT) | 22315 |
-|[imartinez/privateGPT](https://github.com/imartinez/privateGPT) | 17474 |
-|[openai/chatgpt-retrieval-plugin](https://github.com/openai/chatgpt-retrieval-plugin) | 16923 |
-|[mindsdb/mindsdb](https://github.com/mindsdb/mindsdb) | 16112 |
-|[jerryjliu/llama_index](https://github.com/jerryjliu/llama_index) | 15407 |
-|[mlflow/mlflow](https://github.com/mlflow/mlflow) | 14345 |
-|[GaiZhenbiao/ChuanhuChatGPT](https://github.com/GaiZhenbiao/ChuanhuChatGPT) | 10372 |
-|[databrickslabs/dolly](https://github.com/databrickslabs/dolly) | 9919 |
-|[AIGC-Audio/AudioGPT](https://github.com/AIGC-Audio/AudioGPT) | 8177 |
-|[logspace-ai/langflow](https://github.com/logspace-ai/langflow) | 6807 |
-|[imClumsyPanda/langchain-ChatGLM](https://github.com/imClumsyPanda/langchain-ChatGLM) | 6087 |
-|[arc53/DocsGPT](https://github.com/arc53/DocsGPT) | 5292 |
-|[e2b-dev/e2b](https://github.com/e2b-dev/e2b) | 4622 |
-|[nsarrazin/serge](https://github.com/nsarrazin/serge) | 4076 |
-|[madawei2699/myGPTReader](https://github.com/madawei2699/myGPTReader) | 3952 |
-|[zauberzeug/nicegui](https://github.com/zauberzeug/nicegui) | 3952 |
-|[go-skynet/LocalAI](https://github.com/go-skynet/LocalAI) | 3762 |
-|[GreyDGL/PentestGPT](https://github.com/GreyDGL/PentestGPT) | 3388 |
-|[mmabrouk/chatgpt-wrapper](https://github.com/mmabrouk/chatgpt-wrapper) | 3243 |
-|[zilliztech/GPTCache](https://github.com/zilliztech/GPTCache) | 3189 |
-|[wenda-LLM/wenda](https://github.com/wenda-LLM/wenda) | 3050 |
-|[marqo-ai/marqo](https://github.com/marqo-ai/marqo) | 2930 |
-|[gkamradt/langchain-tutorials](https://github.com/gkamradt/langchain-tutorials) | 2710 |
-|[PrefectHQ/marvin](https://github.com/PrefectHQ/marvin) | 2545 |
-|[project-baize/baize-chatbot](https://github.com/project-baize/baize-chatbot) | 2479 |
-|[whitead/paper-qa](https://github.com/whitead/paper-qa) | 2399 |
-|[langgenius/dify](https://github.com/langgenius/dify) | 2344 |
-|[GerevAI/gerev](https://github.com/GerevAI/gerev) | 2283 |
-|[hwchase17/chat-langchain](https://github.com/hwchase17/chat-langchain) | 2266 |
-|[guangzhengli/ChatFiles](https://github.com/guangzhengli/ChatFiles) | 1903 |
-|[Azure-Samples/azure-search-openai-demo](https://github.com/Azure-Samples/azure-search-openai-demo) | 1884 |
-|[OpenBMB/BMTools](https://github.com/OpenBMB/BMTools) | 1860 |
-|[Farama-Foundation/PettingZoo](https://github.com/Farama-Foundation/PettingZoo) | 1813 |
-|[OpenGVLab/Ask-Anything](https://github.com/OpenGVLab/Ask-Anything) | 1571 |
-|[IntelligenzaArtificiale/Free-Auto-GPT](https://github.com/IntelligenzaArtificiale/Free-Auto-GPT) | 1480 |
-|[hwchase17/notion-qa](https://github.com/hwchase17/notion-qa) | 1464 |
-|[NVIDIA/NeMo-Guardrails](https://github.com/NVIDIA/NeMo-Guardrails) | 1419 |
-|[Unstructured-IO/unstructured](https://github.com/Unstructured-IO/unstructured) | 1410 |
-|[Kav-K/GPTDiscord](https://github.com/Kav-K/GPTDiscord) | 1363 |
-|[paulpierre/RasaGPT](https://github.com/paulpierre/RasaGPT) | 1344 |
-|[StanGirard/quivr](https://github.com/StanGirard/quivr) | 1330 |
-|[lunasec-io/lunasec](https://github.com/lunasec-io/lunasec) | 1318 |
-|[vocodedev/vocode-python](https://github.com/vocodedev/vocode-python) | 1286 |
-|[agiresearch/OpenAGI](https://github.com/agiresearch/OpenAGI) | 1156 |
-|[h2oai/h2ogpt](https://github.com/h2oai/h2ogpt) | 1141 |
-|[jina-ai/thinkgpt](https://github.com/jina-ai/thinkgpt) | 1106 |
-|[yanqiangmiffy/Chinese-LangChain](https://github.com/yanqiangmiffy/Chinese-LangChain) | 1072 |
-|[ttengwang/Caption-Anything](https://github.com/ttengwang/Caption-Anything) | 1064 |
-|[jina-ai/dev-gpt](https://github.com/jina-ai/dev-gpt) | 1057 |
-|[juncongmoo/chatllama](https://github.com/juncongmoo/chatllama) | 1003 |
-|[greshake/llm-security](https://github.com/greshake/llm-security) | 1002 |
-|[visual-openllm/visual-openllm](https://github.com/visual-openllm/visual-openllm) | 957 |
-|[richardyc/Chrome-GPT](https://github.com/richardyc/Chrome-GPT) | 918 |
-|[irgolic/AutoPR](https://github.com/irgolic/AutoPR) | 886 |
-|[mmz-001/knowledge_gpt](https://github.com/mmz-001/knowledge_gpt) | 867 |
-|[thomas-yanxin/LangChain-ChatGLM-Webui](https://github.com/thomas-yanxin/LangChain-ChatGLM-Webui) | 850 |
-|[microsoft/X-Decoder](https://github.com/microsoft/X-Decoder) | 837 |
-|[peterw/Chat-with-Github-Repo](https://github.com/peterw/Chat-with-Github-Repo) | 826 |
-|[cirediatpl/FigmaChain](https://github.com/cirediatpl/FigmaChain) | 782 |
-|[hashintel/hash](https://github.com/hashintel/hash) | 778 |
-|[seanpixel/Teenage-AGI](https://github.com/seanpixel/Teenage-AGI) | 773 |
-|[jina-ai/langchain-serve](https://github.com/jina-ai/langchain-serve) | 738 |
-|[corca-ai/EVAL](https://github.com/corca-ai/EVAL) | 737 |
-|[ai-sidekick/sidekick](https://github.com/ai-sidekick/sidekick) | 717 |
-|[rlancemartin/auto-evaluator](https://github.com/rlancemartin/auto-evaluator) | 703 |
-|[poe-platform/api-bot-tutorial](https://github.com/poe-platform/api-bot-tutorial) | 689 |
-|[SamurAIGPT/Camel-AutoGPT](https://github.com/SamurAIGPT/Camel-AutoGPT) | 666 |
-|[eyurtsev/kor](https://github.com/eyurtsev/kor) | 608 |
-|[run-llama/llama-lab](https://github.com/run-llama/llama-lab) | 559 |
-|[namuan/dr-doc-search](https://github.com/namuan/dr-doc-search) | 544 |
-|[pieroit/cheshire-cat](https://github.com/pieroit/cheshire-cat) | 520 |
-|[griptape-ai/griptape](https://github.com/griptape-ai/griptape) | 514 |
-|[getmetal/motorhead](https://github.com/getmetal/motorhead) | 481 |
-|[hwchase17/chat-your-data](https://github.com/hwchase17/chat-your-data) | 462 |
-|[langchain-ai/langchain-aiplugin](https://github.com/langchain-ai/langchain-aiplugin) | 452 |
-|[jina-ai/agentchain](https://github.com/jina-ai/agentchain) | 439 |
-|[SamurAIGPT/ChatGPT-Developer-Plugins](https://github.com/SamurAIGPT/ChatGPT-Developer-Plugins) | 437 |
-|[alexanderatallah/window.ai](https://github.com/alexanderatallah/window.ai) | 433 |
-|[michaelthwan/searchGPT](https://github.com/michaelthwan/searchGPT) | 427 |
-|[mpaepper/content-chatbot](https://github.com/mpaepper/content-chatbot) | 425 |
-|[mckaywrigley/repo-chat](https://github.com/mckaywrigley/repo-chat) | 422 |
-|[whyiyhw/chatgpt-wechat](https://github.com/whyiyhw/chatgpt-wechat) | 421 |
-|[freddyaboulton/gradio-tools](https://github.com/freddyaboulton/gradio-tools) | 407 |
-|[jonra1993/fastapi-alembic-sqlmodel-async](https://github.com/jonra1993/fastapi-alembic-sqlmodel-async) | 395 |
-|[yeagerai/yeagerai-agent](https://github.com/yeagerai/yeagerai-agent) | 383 |
-|[akshata29/chatpdf](https://github.com/akshata29/chatpdf) | 374 |
-|[OpenGVLab/InternGPT](https://github.com/OpenGVLab/InternGPT) | 368 |
-|[ruoccofabrizio/azure-open-ai-embeddings-qna](https://github.com/ruoccofabrizio/azure-open-ai-embeddings-qna) | 358 |
-|[101dotxyz/GPTeam](https://github.com/101dotxyz/GPTeam) | 357 |
-|[mtenenholtz/chat-twitter](https://github.com/mtenenholtz/chat-twitter) | 354 |
-|[amosjyng/langchain-visualizer](https://github.com/amosjyng/langchain-visualizer) | 343 |
-|[msoedov/langcorn](https://github.com/msoedov/langcorn) | 334 |
-|[showlab/VLog](https://github.com/showlab/VLog) | 330 |
-|[continuum-llms/chatgpt-memory](https://github.com/continuum-llms/chatgpt-memory) | 324 |
-|[steamship-core/steamship-langchain](https://github.com/steamship-core/steamship-langchain) | 323 |
-|[daodao97/chatdoc](https://github.com/daodao97/chatdoc) | 320 |
-|[xuwenhao/geektime-ai-course](https://github.com/xuwenhao/geektime-ai-course) | 308 |
-|[StevenGrove/GPT4Tools](https://github.com/StevenGrove/GPT4Tools) | 301 |
-|[logan-markewich/llama_index_starter_pack](https://github.com/logan-markewich/llama_index_starter_pack) | 300 |
-|[andylokandy/gpt-4-search](https://github.com/andylokandy/gpt-4-search) | 299 |
-|[Anil-matcha/ChatPDF](https://github.com/Anil-matcha/ChatPDF) | 287 |
-|[itamargol/openai](https://github.com/itamargol/openai) | 273 |
-|[BlackHC/llm-strategy](https://github.com/BlackHC/llm-strategy) | 267 |
-|[momegas/megabots](https://github.com/momegas/megabots) | 259 |
-|[bborn/howdoi.ai](https://github.com/bborn/howdoi.ai) | 238 |
-|[Cheems-Seminar/grounded-segment-any-parts](https://github.com/Cheems-Seminar/grounded-segment-any-parts) | 232 |
-|[ur-whitelab/exmol](https://github.com/ur-whitelab/exmol) | 227 |
-|[sullivan-sean/chat-langchainjs](https://github.com/sullivan-sean/chat-langchainjs) | 227 |
-|[explosion/spacy-llm](https://github.com/explosion/spacy-llm) | 226 |
-|[recalign/RecAlign](https://github.com/recalign/RecAlign) | 218 |
-|[jupyterlab/jupyter-ai](https://github.com/jupyterlab/jupyter-ai) | 218 |
-|[alvarosevilla95/autolang](https://github.com/alvarosevilla95/autolang) | 215 |
-|[conceptofmind/toolformer](https://github.com/conceptofmind/toolformer) | 213 |
-|[MagnivOrg/prompt-layer-library](https://github.com/MagnivOrg/prompt-layer-library) | 209 |
-|[JohnSnowLabs/nlptest](https://github.com/JohnSnowLabs/nlptest) | 208 |
-|[airobotlab/KoChatGPT](https://github.com/airobotlab/KoChatGPT) | 197 |
-|[langchain-ai/auto-evaluator](https://github.com/langchain-ai/auto-evaluator) | 195 |
-|[yvann-hub/Robby-chatbot](https://github.com/yvann-hub/Robby-chatbot) | 195 |
-|[alejandro-ao/langchain-ask-pdf](https://github.com/alejandro-ao/langchain-ask-pdf) | 192 |
-|[daveebbelaar/langchain-experiments](https://github.com/daveebbelaar/langchain-experiments) | 189 |
-|[NimbleBoxAI/ChainFury](https://github.com/NimbleBoxAI/ChainFury) | 187 |
-|[kaleido-lab/dolphin](https://github.com/kaleido-lab/dolphin) | 184 |
-|[Anil-matcha/Website-to-Chatbot](https://github.com/Anil-matcha/Website-to-Chatbot) | 183 |
-|[plchld/InsightFlow](https://github.com/plchld/InsightFlow) | 180 |
-|[OpenBMB/AgentVerse](https://github.com/OpenBMB/AgentVerse) | 166 |
-|[benthecoder/ClassGPT](https://github.com/benthecoder/ClassGPT) | 166 |
-|[jbrukh/gpt-jargon](https://github.com/jbrukh/gpt-jargon) | 161 |
-|[hardbyte/qabot](https://github.com/hardbyte/qabot) | 160 |
-|[shaman-ai/agent-actors](https://github.com/shaman-ai/agent-actors) | 153 |
-|[radi-cho/datasetGPT](https://github.com/radi-cho/datasetGPT) | 153 |
-|[poe-platform/poe-protocol](https://github.com/poe-platform/poe-protocol) | 152 |
-|[paolorechia/learn-langchain](https://github.com/paolorechia/learn-langchain) | 149 |
-|[ajndkr/lanarky](https://github.com/ajndkr/lanarky) | 149 |
-|[fengyuli-dev/multimedia-gpt](https://github.com/fengyuli-dev/multimedia-gpt) | 147 |
-|[yasyf/compress-gpt](https://github.com/yasyf/compress-gpt) | 144 |
-|[homanp/superagent](https://github.com/homanp/superagent) | 143 |
-|[realminchoi/babyagi-ui](https://github.com/realminchoi/babyagi-ui) | 141 |
-|[ethanyanjiali/minChatGPT](https://github.com/ethanyanjiali/minChatGPT) | 141 |
-|[ccurme/yolopandas](https://github.com/ccurme/yolopandas) | 139 |
-|[hwchase17/langchain-streamlit-template](https://github.com/hwchase17/langchain-streamlit-template) | 138 |
-|[Jaseci-Labs/jaseci](https://github.com/Jaseci-Labs/jaseci) | 136 |
+|[openai/openai-cookbook](https://github.com/openai/openai-cookbook) | 38024 |
+|[LAION-AI/Open-Assistant](https://github.com/LAION-AI/Open-Assistant) | 33609 |
+|[microsoft/TaskMatrix](https://github.com/microsoft/TaskMatrix) | 33136 |
+|[hpcaitech/ColossalAI](https://github.com/hpcaitech/ColossalAI) | 30032 |
+|[imartinez/privateGPT](https://github.com/imartinez/privateGPT) | 28094 |
+|[reworkd/AgentGPT](https://github.com/reworkd/AgentGPT) | 23430 |
+|[openai/chatgpt-retrieval-plugin](https://github.com/openai/chatgpt-retrieval-plugin) | 17942 |
+|[jerryjliu/llama_index](https://github.com/jerryjliu/llama_index) | 16697 |
+|[mindsdb/mindsdb](https://github.com/mindsdb/mindsdb) | 16410 |
+|[mlflow/mlflow](https://github.com/mlflow/mlflow) | 14517 |
+|[GaiZhenbiao/ChuanhuChatGPT](https://github.com/GaiZhenbiao/ChuanhuChatGPT) | 10793 |
+|[databrickslabs/dolly](https://github.com/databrickslabs/dolly) | 10155 |
+|[openai/evals](https://github.com/openai/evals) | 10076 |
+|[AIGC-Audio/AudioGPT](https://github.com/AIGC-Audio/AudioGPT) | 8619 |
+|[logspace-ai/langflow](https://github.com/logspace-ai/langflow) | 8211 |
+|[imClumsyPanda/langchain-ChatGLM](https://github.com/imClumsyPanda/langchain-ChatGLM) | 8154 |
+|[PromtEngineer/localGPT](https://github.com/PromtEngineer/localGPT) | 6853 |
+|[StanGirard/quivr](https://github.com/StanGirard/quivr) | 6830 |
+|[PipedreamHQ/pipedream](https://github.com/PipedreamHQ/pipedream) | 6520 |
+|[go-skynet/LocalAI](https://github.com/go-skynet/LocalAI) | 6018 |
+|[arc53/DocsGPT](https://github.com/arc53/DocsGPT) | 5643 |
+|[e2b-dev/e2b](https://github.com/e2b-dev/e2b) | 5075 |
+|[langgenius/dify](https://github.com/langgenius/dify) | 4281 |
+|[nsarrazin/serge](https://github.com/nsarrazin/serge) | 4228 |
+|[zauberzeug/nicegui](https://github.com/zauberzeug/nicegui) | 4084 |
+|[madawei2699/myGPTReader](https://github.com/madawei2699/myGPTReader) | 4039 |
+|[wenda-LLM/wenda](https://github.com/wenda-LLM/wenda) | 3871 |
+|[GreyDGL/PentestGPT](https://github.com/GreyDGL/PentestGPT) | 3837 |
+|[zilliztech/GPTCache](https://github.com/zilliztech/GPTCache) | 3625 |
+|[csunny/DB-GPT](https://github.com/csunny/DB-GPT) | 3545 |
+|[gkamradt/langchain-tutorials](https://github.com/gkamradt/langchain-tutorials) | 3404 |
+|[mmabrouk/chatgpt-wrapper](https://github.com/mmabrouk/chatgpt-wrapper) | 3303 |
+|[postgresml/postgresml](https://github.com/postgresml/postgresml) | 3052 |
+|[marqo-ai/marqo](https://github.com/marqo-ai/marqo) | 3014 |
+|[MineDojo/Voyager](https://github.com/MineDojo/Voyager) | 2945 |
+|[PrefectHQ/marvin](https://github.com/PrefectHQ/marvin) | 2761 |
+|[project-baize/baize-chatbot](https://github.com/project-baize/baize-chatbot) | 2673 |
+|[hwchase17/chat-langchain](https://github.com/hwchase17/chat-langchain) | 2589 |
+|[whitead/paper-qa](https://github.com/whitead/paper-qa) | 2572 |
+|[Azure-Samples/azure-search-openai-demo](https://github.com/Azure-Samples/azure-search-openai-demo) | 2366 |
+|[GerevAI/gerev](https://github.com/GerevAI/gerev) | 2330 |
+|[OpenGVLab/InternGPT](https://github.com/OpenGVLab/InternGPT) | 2289 |
+|[ParisNeo/gpt4all-ui](https://github.com/ParisNeo/gpt4all-ui) | 2159 |
+|[OpenBMB/BMTools](https://github.com/OpenBMB/BMTools) | 2158 |
+|[guangzhengli/ChatFiles](https://github.com/guangzhengli/ChatFiles) | 2005 |
+|[h2oai/h2ogpt](https://github.com/h2oai/h2ogpt) | 1939 |
+|[Farama-Foundation/PettingZoo](https://github.com/Farama-Foundation/PettingZoo) | 1845 |
+|[OpenGVLab/Ask-Anything](https://github.com/OpenGVLab/Ask-Anything) | 1749 |
+|[IntelligenzaArtificiale/Free-Auto-GPT](https://github.com/IntelligenzaArtificiale/Free-Auto-GPT) | 1740 |
+|[Unstructured-IO/unstructured](https://github.com/Unstructured-IO/unstructured) | 1628 |
+|[hwchase17/notion-qa](https://github.com/hwchase17/notion-qa) | 1607 |
+|[NVIDIA/NeMo-Guardrails](https://github.com/NVIDIA/NeMo-Guardrails) | 1544 |
+|[SamurAIGPT/privateGPT](https://github.com/SamurAIGPT/privateGPT) | 1543 |
+|[paulpierre/RasaGPT](https://github.com/paulpierre/RasaGPT) | 1526 |
+|[yanqiangmiffy/Chinese-LangChain](https://github.com/yanqiangmiffy/Chinese-LangChain) | 1485 |
+|[Kav-K/GPTDiscord](https://github.com/Kav-K/GPTDiscord) | 1402 |
+|[vocodedev/vocode-python](https://github.com/vocodedev/vocode-python) | 1387 |
+|[Chainlit/chainlit](https://github.com/Chainlit/chainlit) | 1336 |
+|[lunasec-io/lunasec](https://github.com/lunasec-io/lunasec) | 1323 |
+|[psychic-api/psychic](https://github.com/psychic-api/psychic) | 1248 |
+|[agiresearch/OpenAGI](https://github.com/agiresearch/OpenAGI) | 1208 |
+|[jina-ai/thinkgpt](https://github.com/jina-ai/thinkgpt) | 1193 |
+|[thomas-yanxin/LangChain-ChatGLM-Webui](https://github.com/thomas-yanxin/LangChain-ChatGLM-Webui) | 1182 |
+|[ttengwang/Caption-Anything](https://github.com/ttengwang/Caption-Anything) | 1137 |
+|[jina-ai/dev-gpt](https://github.com/jina-ai/dev-gpt) | 1135 |
+|[greshake/llm-security](https://github.com/greshake/llm-security) | 1086 |
+|[keephq/keep](https://github.com/keephq/keep) | 1063 |
+|[juncongmoo/chatllama](https://github.com/juncongmoo/chatllama) | 1037 |
+|[richardyc/Chrome-GPT](https://github.com/richardyc/Chrome-GPT) | 1035 |
+|[visual-openllm/visual-openllm](https://github.com/visual-openllm/visual-openllm) | 997 |
+|[mmz-001/knowledge_gpt](https://github.com/mmz-001/knowledge_gpt) | 995 |
+|[jina-ai/langchain-serve](https://github.com/jina-ai/langchain-serve) | 949 |
+|[irgolic/AutoPR](https://github.com/irgolic/AutoPR) | 936 |
+|[microsoft/X-Decoder](https://github.com/microsoft/X-Decoder) | 908 |
+|[poe-platform/api-bot-tutorial](https://github.com/poe-platform/api-bot-tutorial) | 902 |
+|[peterw/Chat-with-Github-Repo](https://github.com/peterw/Chat-with-Github-Repo) | 875 |
+|[cirediatpl/FigmaChain](https://github.com/cirediatpl/FigmaChain) | 822 |
+|[homanp/superagent](https://github.com/homanp/superagent) | 806 |
+|[seanpixel/Teenage-AGI](https://github.com/seanpixel/Teenage-AGI) | 800 |
+|[chatarena/chatarena](https://github.com/chatarena/chatarena) | 796 |
+|[hashintel/hash](https://github.com/hashintel/hash) | 795 |
+|[SamurAIGPT/Camel-AutoGPT](https://github.com/SamurAIGPT/Camel-AutoGPT) | 786 |
+|[rlancemartin/auto-evaluator](https://github.com/rlancemartin/auto-evaluator) | 770 |
+|[corca-ai/EVAL](https://github.com/corca-ai/EVAL) | 769 |
+|[101dotxyz/GPTeam](https://github.com/101dotxyz/GPTeam) | 755 |
+|[noahshinn024/reflexion](https://github.com/noahshinn024/reflexion) | 706 |
+|[eyurtsev/kor](https://github.com/eyurtsev/kor) | 695 |
+|[cheshire-cat-ai/core](https://github.com/cheshire-cat-ai/core) | 681 |
+|[e-johnstonn/BriefGPT](https://github.com/e-johnstonn/BriefGPT) | 656 |
+|[run-llama/llama-lab](https://github.com/run-llama/llama-lab) | 635 |
+|[griptape-ai/griptape](https://github.com/griptape-ai/griptape) | 583 |
+|[namuan/dr-doc-search](https://github.com/namuan/dr-doc-search) | 555 |
+|[getmetal/motorhead](https://github.com/getmetal/motorhead) | 550 |
+|[kreneskyp/ix](https://github.com/kreneskyp/ix) | 543 |
+|[hwchase17/chat-your-data](https://github.com/hwchase17/chat-your-data) | 510 |
+|[Anil-matcha/ChatPDF](https://github.com/Anil-matcha/ChatPDF) | 501 |
+|[whyiyhw/chatgpt-wechat](https://github.com/whyiyhw/chatgpt-wechat) | 497 |
+|[SamurAIGPT/ChatGPT-Developer-Plugins](https://github.com/SamurAIGPT/ChatGPT-Developer-Plugins) | 496 |
+|[microsoft/PodcastCopilot](https://github.com/microsoft/PodcastCopilot) | 492 |
+|[debanjum/khoj](https://github.com/debanjum/khoj) | 485 |
+|[akshata29/chatpdf](https://github.com/akshata29/chatpdf) | 485 |
+|[langchain-ai/langchain-aiplugin](https://github.com/langchain-ai/langchain-aiplugin) | 462 |
+|[jina-ai/agentchain](https://github.com/jina-ai/agentchain) | 460 |
+|[alexanderatallah/window.ai](https://github.com/alexanderatallah/window.ai) | 457 |
+|[yeagerai/yeagerai-agent](https://github.com/yeagerai/yeagerai-agent) | 451 |
+|[mckaywrigley/repo-chat](https://github.com/mckaywrigley/repo-chat) | 446 |
+|[michaelthwan/searchGPT](https://github.com/michaelthwan/searchGPT) | 446 |
+|[mpaepper/content-chatbot](https://github.com/mpaepper/content-chatbot) | 441 |
+|[freddyaboulton/gradio-tools](https://github.com/freddyaboulton/gradio-tools) | 439 |
+|[ruoccofabrizio/azure-open-ai-embeddings-qna](https://github.com/ruoccofabrizio/azure-open-ai-embeddings-qna) | 429 |
+|[StevenGrove/GPT4Tools](https://github.com/StevenGrove/GPT4Tools) | 422 |
+|[jonra1993/fastapi-alembic-sqlmodel-async](https://github.com/jonra1993/fastapi-alembic-sqlmodel-async) | 407 |
+|[msoedov/langcorn](https://github.com/msoedov/langcorn) | 405 |
+|[amosjyng/langchain-visualizer](https://github.com/amosjyng/langchain-visualizer) | 395 |
+|[ajndkr/lanarky](https://github.com/ajndkr/lanarky) | 384 |
+|[mtenenholtz/chat-twitter](https://github.com/mtenenholtz/chat-twitter) | 376 |
+|[steamship-core/steamship-langchain](https://github.com/steamship-core/steamship-langchain) | 371 |
+|[langchain-ai/auto-evaluator](https://github.com/langchain-ai/auto-evaluator) | 365 |
+|[xuwenhao/geektime-ai-course](https://github.com/xuwenhao/geektime-ai-course) | 358 |
+|[continuum-llms/chatgpt-memory](https://github.com/continuum-llms/chatgpt-memory) | 357 |
+|[opentensor/bittensor](https://github.com/opentensor/bittensor) | 347 |
+|[showlab/VLog](https://github.com/showlab/VLog) | 345 |
+|[daodao97/chatdoc](https://github.com/daodao97/chatdoc) | 345 |
+|[logan-markewich/llama_index_starter_pack](https://github.com/logan-markewich/llama_index_starter_pack) | 332 |
+|[poe-platform/poe-protocol](https://github.com/poe-platform/poe-protocol) | 320 |
+|[explosion/spacy-llm](https://github.com/explosion/spacy-llm) | 312 |
+|[andylokandy/gpt-4-search](https://github.com/andylokandy/gpt-4-search) | 311 |
+|[alejandro-ao/langchain-ask-pdf](https://github.com/alejandro-ao/langchain-ask-pdf) | 310 |
+|[jupyterlab/jupyter-ai](https://github.com/jupyterlab/jupyter-ai) | 294 |
+|[BlackHC/llm-strategy](https://github.com/BlackHC/llm-strategy) | 283 |
+|[itamargol/openai](https://github.com/itamargol/openai) | 281 |
+|[momegas/megabots](https://github.com/momegas/megabots) | 279 |
+|[personoids/personoids-lite](https://github.com/personoids/personoids-lite) | 277 |
+|[yvann-hub/Robby-chatbot](https://github.com/yvann-hub/Robby-chatbot) | 267 |
+|[Anil-matcha/Website-to-Chatbot](https://github.com/Anil-matcha/Website-to-Chatbot) | 266 |
+|[Cheems-Seminar/grounded-segment-any-parts](https://github.com/Cheems-Seminar/grounded-segment-any-parts) | 260 |
+|[sullivan-sean/chat-langchainjs](https://github.com/sullivan-sean/chat-langchainjs) | 248 |
+|[bborn/howdoi.ai](https://github.com/bborn/howdoi.ai) | 245 |
+|[daveebbelaar/langchain-experiments](https://github.com/daveebbelaar/langchain-experiments) | 240 |
+|[MagnivOrg/prompt-layer-library](https://github.com/MagnivOrg/prompt-layer-library) | 237 |
+|[ur-whitelab/exmol](https://github.com/ur-whitelab/exmol) | 234 |
+|[conceptofmind/toolformer](https://github.com/conceptofmind/toolformer) | 234 |
+|[recalign/RecAlign](https://github.com/recalign/RecAlign) | 226 |
+|[OpenBMB/AgentVerse](https://github.com/OpenBMB/AgentVerse) | 220 |
+|[alvarosevilla95/autolang](https://github.com/alvarosevilla95/autolang) | 219 |
+|[JohnSnowLabs/nlptest](https://github.com/JohnSnowLabs/nlptest) | 216 |
+|[kaleido-lab/dolphin](https://github.com/kaleido-lab/dolphin) | 215 |
+|[truera/trulens](https://github.com/truera/trulens) | 208 |
+|[NimbleBoxAI/ChainFury](https://github.com/NimbleBoxAI/ChainFury) | 208 |
+|[airobotlab/KoChatGPT](https://github.com/airobotlab/KoChatGPT) | 207 |
+|[monarch-initiative/ontogpt](https://github.com/monarch-initiative/ontogpt) | 200 |
+|[paolorechia/learn-langchain](https://github.com/paolorechia/learn-langchain) | 195 |
+|[shaman-ai/agent-actors](https://github.com/shaman-ai/agent-actors) | 185 |
+|[Haste171/langchain-chatbot](https://github.com/Haste171/langchain-chatbot) | 184 |
+|[plchld/InsightFlow](https://github.com/plchld/InsightFlow) | 182 |
+|[su77ungr/CASALIOY](https://github.com/su77ungr/CASALIOY) | 180 |
+|[jbrukh/gpt-jargon](https://github.com/jbrukh/gpt-jargon) | 177 |
+|[benthecoder/ClassGPT](https://github.com/benthecoder/ClassGPT) | 174 |
+|[billxbf/ReWOO](https://github.com/billxbf/ReWOO) | 170 |
+|[filip-michalsky/SalesGPT](https://github.com/filip-michalsky/SalesGPT) | 168 |
+|[hwchase17/langchain-streamlit-template](https://github.com/hwchase17/langchain-streamlit-template) | 168 |
+|[radi-cho/datasetGPT](https://github.com/radi-cho/datasetGPT) | 164 |
+|[hardbyte/qabot](https://github.com/hardbyte/qabot) | 164 |
+|[gia-guar/JARVIS-ChatGPT](https://github.com/gia-guar/JARVIS-ChatGPT) | 158 |
+|[plastic-labs/tutor-gpt](https://github.com/plastic-labs/tutor-gpt) | 154 |
+|[yasyf/compress-gpt](https://github.com/yasyf/compress-gpt) | 154 |
+|[fengyuli-dev/multimedia-gpt](https://github.com/fengyuli-dev/multimedia-gpt) | 154 |
+|[ethanyanjiali/minChatGPT](https://github.com/ethanyanjiali/minChatGPT) | 153 |
+|[hwchase17/chroma-langchain](https://github.com/hwchase17/chroma-langchain) | 153 |
+|[edreisMD/plugnplai](https://github.com/edreisMD/plugnplai) | 148 |
+|[chakkaradeep/pyCodeAGI](https://github.com/chakkaradeep/pyCodeAGI) | 145 |
+|[ccurme/yolopandas](https://github.com/ccurme/yolopandas) | 145 |
+|[shamspias/customizable-gpt-chatbot](https://github.com/shamspias/customizable-gpt-chatbot) | 144 |
+|[realminchoi/babyagi-ui](https://github.com/realminchoi/babyagi-ui) | 143 |
+|[PradipNichite/Youtube-Tutorials](https://github.com/PradipNichite/Youtube-Tutorials) | 140 |
+|[gustavz/DataChad](https://github.com/gustavz/DataChad) | 140 |
+|[Klingefjord/chatgpt-telegram](https://github.com/Klingefjord/chatgpt-telegram) | 140 |
+|[Jaseci-Labs/jaseci](https://github.com/Jaseci-Labs/jaseci) | 139 |
+|[handrew/browserpilot](https://github.com/handrew/browserpilot) | 137 |
+|[jmpaz/promptlib](https://github.com/jmpaz/promptlib) | 137 |
+|[SamPink/dev-gpt](https://github.com/SamPink/dev-gpt) | 135 |
+|[menloparklab/langchain-cohere-qdrant-doc-retrieval](https://github.com/menloparklab/langchain-cohere-qdrant-doc-retrieval) | 135 |
 |[hirokidaichi/wanna](https://github.com/hirokidaichi/wanna) | 135 |
-|[Haste171/langchain-chatbot](https://github.com/Haste171/langchain-chatbot) | 134 |
-|[jmpaz/promptlib](https://github.com/jmpaz/promptlib) | 130 |
-|[Klingefjord/chatgpt-telegram](https://github.com/Klingefjord/chatgpt-telegram) | 130 |
-|[filip-michalsky/SalesGPT](https://github.com/filip-michalsky/SalesGPT) | 128 |
-|[handrew/browserpilot](https://github.com/handrew/browserpilot) | 128 |
-|[shauryr/S2QA](https://github.com/shauryr/S2QA) | 127 |
-|[steamship-core/vercel-examples](https://github.com/steamship-core/vercel-examples) | 127 |
-|[yasyf/summ](https://github.com/yasyf/summ) | 127 |
-|[gia-guar/JARVIS-ChatGPT](https://github.com/gia-guar/JARVIS-ChatGPT) | 126 |
-|[jerlendds/osintbuddy](https://github.com/jerlendds/osintbuddy) | 125 |
-|[ibiscp/LLM-IMDB](https://github.com/ibiscp/LLM-IMDB) | 124 |
-|[Teahouse-Studios/akari-bot](https://github.com/Teahouse-Studios/akari-bot) | 124 |
-|[hwchase17/chroma-langchain](https://github.com/hwchase17/chroma-langchain) | 124 |
-|[menloparklab/langchain-cohere-qdrant-doc-retrieval](https://github.com/menloparklab/langchain-cohere-qdrant-doc-retrieval) | 123 |
-|[peterw/StoryStorm](https://github.com/peterw/StoryStorm) | 123 |
-|[chakkaradeep/pyCodeAGI](https://github.com/chakkaradeep/pyCodeAGI) | 123 |
-|[petehunt/langchain-github-bot](https://github.com/petehunt/langchain-github-bot) | 115 |
-|[su77ungr/CASALIOY](https://github.com/su77ungr/CASALIOY) | 113 |
-|[eunomia-bpf/GPTtrace](https://github.com/eunomia-bpf/GPTtrace) | 113 |
+|[steamship-core/vercel-examples](https://github.com/steamship-core/vercel-examples) | 134 |
+|[pablomarin/GPT-Azure-Search-Engine](https://github.com/pablomarin/GPT-Azure-Search-Engine) | 133 |
+|[ibiscp/LLM-IMDB](https://github.com/ibiscp/LLM-IMDB) | 133 |
+|[shauryr/S2QA](https://github.com/shauryr/S2QA) | 133 |
+|[jerlendds/osintbuddy](https://github.com/jerlendds/osintbuddy) | 132 |
+|[yuanjie-ai/ChatLLM](https://github.com/yuanjie-ai/ChatLLM) | 132 |
+|[yasyf/summ](https://github.com/yasyf/summ) | 132 |
+|[WongSaang/chatgpt-ui-server](https://github.com/WongSaang/chatgpt-ui-server) | 130 |
+|[peterw/StoryStorm](https://github.com/peterw/StoryStorm) | 127 |
+|[Teahouse-Studios/akari-bot](https://github.com/Teahouse-Studios/akari-bot) | 126 |
+|[vaibkumr/prompt-optimizer](https://github.com/vaibkumr/prompt-optimizer) | 125 |
+|[preset-io/promptimize](https://github.com/preset-io/promptimize) | 124 |
+|[homanp/vercel-langchain](https://github.com/homanp/vercel-langchain) | 124 |
+|[petehunt/langchain-github-bot](https://github.com/petehunt/langchain-github-bot) | 123 |
+|[eunomia-bpf/GPTtrace](https://github.com/eunomia-bpf/GPTtrace) | 118 |
+|[nicknochnack/LangchainDocuments](https://github.com/nicknochnack/LangchainDocuments) | 116 |
+|[jiran214/GPT-vup](https://github.com/jiran214/GPT-vup) | 112 |
+|[rsaryev/talk-codebase](https://github.com/rsaryev/talk-codebase) | 112 |
 |[zenml-io/zenml-projects](https://github.com/zenml-io/zenml-projects) | 112 |
-|[pablomarin/GPT-Azure-Search-Engine](https://github.com/pablomarin/GPT-Azure-Search-Engine) | 111 |
-|[shamspias/customizable-gpt-chatbot](https://github.com/shamspias/customizable-gpt-chatbot) | 109 |
-|[WongSaang/chatgpt-ui-server](https://github.com/WongSaang/chatgpt-ui-server) | 108 |
-|[davila7/file-gpt](https://github.com/davila7/file-gpt) | 104 |
-|[enhancedocs/enhancedocs](https://github.com/enhancedocs/enhancedocs) | 102 |
-|[aurelio-labs/arxiv-bot](https://github.com/aurelio-labs/arxiv-bot) | 101 |
-
+|[microsoft/azure-openai-in-a-day-workshop](https://github.com/microsoft/azure-openai-in-a-day-workshop) | 112 |
+|[davila7/file-gpt](https://github.com/davila7/file-gpt) | 112 |
+|[prof-frink-lab/slangchain](https://github.com/prof-frink-lab/slangchain) | 111 |
+|[aurelio-labs/arxiv-bot](https://github.com/aurelio-labs/arxiv-bot) | 110 |
+|[fixie-ai/fixie-examples](https://github.com/fixie-ai/fixie-examples) | 108 |
+|[miaoshouai/miaoshouai-assistant](https://github.com/miaoshouai/miaoshouai-assistant) | 105 |
+|[flurb18/AgentOoba](https://github.com/flurb18/AgentOoba) | 103 |
+|[solana-labs/chatgpt-plugin](https://github.com/solana-labs/chatgpt-plugin) | 102 |
+|[Significant-Gravitas/Auto-GPT-Benchmarks](https://github.com/Significant-Gravitas/Auto-GPT-Benchmarks) | 102 |
+|[kaarthik108/snowChat](https://github.com/kaarthik108/snowChat) | 100 |
 
 
 _Generated by [github-dependents-info](https://github.com/nvuillam/github-dependents-info)_
 
-[github-dependents-info --repo hwchase17/langchain --markdownfile dependents.md --minstars 100 --sort stars]
+`github-dependents-info --repo hwchase17/langchain --markdownfile dependents.md --minstars 100 --sort stars`

docs/ecosystem/baseten.md

@@ -0,0 +1,25 @@
+# Baseten
+
+Learn how to use LangChain with models deployed on Baseten.
+
+## Installation and setup
+
+- Create a [Baseten](https://baseten.co) account and [API key](https://docs.baseten.co/settings/api-keys).
+- Install the Baseten Python client with `pip install baseten`
+- Use your API key to authenticate with `baseten login`
+
+## Invoking a model
+
+Baseten integrates with LangChain through the LLM module, which provides a standardized and interoperable interface for models that are deployed on your Baseten workspace.
+
+You can deploy foundation models like WizardLM and Alpaca with one click from the [Baseten model library](https://app.baseten.co/explore/) or if you have your own model, [deploy it with this tutorial](https://docs.baseten.co/deploying-models/deploy).
+
+In this example, we'll work with WizardLM. [Deploy WizardLM here](https://app.baseten.co/explore/wizardlm) and follow along with the deployed [model's version ID](https://docs.baseten.co/managing-models/manage).
+
+```python
+from langchain.llms import Baseten
+
+wizardlm = Baseten(model="MODEL_VERSION_ID", verbose=True)
+
+wizardlm("What is the difference between a Wizard and a Sorcerer?")
+```

docs/ecosystem/deployments.md

@@ -6,6 +6,11 @@ This section covers several options for that. Note that these options are meant
 
 What follows is a list of template GitHub repositories designed to be easily forked and modified to use your chain. This list is far from exhaustive, and we are EXTREMELY open to contributions here.
 
+## [Anyscale](https://www.anyscale.com/model-serving)
+
+Anyscale is a unified compute platform that makes it easy to develop, deploy, and manage scalable LLM applications in production using Ray.
+With Anyscale you can scale the most challenging LLM-based workloads and both develop and deploy LLM-based apps on a single compute platform.
+
 ## [Streamlit](https://github.com/hwchase17/langchain-streamlit-template)
 
 This repo serves as a template for how to deploy a LangChain with Streamlit.

docs/getting_started/tutorials.md

@@ -1,15 +1,17 @@
 # Tutorials
 
-This is a collection of `LangChain` tutorials mostly on `YouTube`.
+⛓ icon marks a new addition [last update 2023-05-15]
 
-⛓ icon marks a new video [last update 2023-05-15]
+### DeepLearning.AI course
+⛓[LangChain for LLM Application Development](https://learn.deeplearning.ai/langchain) by Harrison Chase presented by [Andrew Ng](https://en.wikipedia.org/wiki/Andrew_Ng)
 
-###
+### Handbook
 [LangChain AI Handbook](https://www.pinecone.io/learn/langchain/) By **James Briggs** and **Francisco Ingham**
 
-###
+### Tutorials
 [LangChain Tutorials](https://www.youtube.com/watch?v=FuqdVNB_8c0&list=PL9V0lbeJ69brU-ojMpU1Y7Ic58Tap0Cw6) by [Edrick](https://www.youtube.com/@edrickdch):
-- ⛓ [LangChain, Chroma DB, OpenAI Beginner Guide | ChatGPT with your PDF](https://youtu.be/FuqdVNB_8c0) 
+- ⛓ [LangChain, Chroma DB, OpenAI Beginner Guide | ChatGPT with your PDF](https://youtu.be/FuqdVNB_8c0)
+- ⛓ [LangChain 101: The Complete Beginner's Guide](https://youtu.be/P3MAbZ2eMUI) 
 
 [LangChain Crash Course: Build an AutoGPT app in 25 minutes](https://youtu.be/MlK6SIjcjE8) by [Nicholas Renotte](https://www.youtube.com/@NicholasRenotte)
 
@@ -108,4 +110,4 @@ LangChain by [Chat with data](https://www.youtube.com/@chatwithdata)
 - ⛓ [Build ChatGPT Chatbots with LangChain Memory: Understanding and Implementing Memory in Conversations](https://youtu.be/CyuUlf54wTs)
 
 ---------------------
-⛓ icon marks a new video [last update 2023-05-15]
+⛓ icon marks a new addition [last update 2023-05-15]

docs/index.rst

@@ -176,6 +176,8 @@ Additional Resources
 
 - `Gallery <https://github.com/kyrolabs/awesome-langchain>`_: A collection of great projects that use Langchain, compiled by the folks at `Kyrolabs <https://kyrolabs.com>`_. Useful for finding inspiration and example implementations.
 
+- `Deploying LLMs in Production <./additional_resources/deploy_llms.html>`_: A collection of best practices and tutorials for deploying LLMs in production.
+
 - `Tracing <./additional_resources/tracing.html>`_: A guide on using tracing in LangChain to visualize the execution of chains and agents.
 
 - `Model Laboratory <./additional_resources/model_laboratory.html>`_: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.
@@ -194,6 +196,8 @@ Additional Resources
    :hidden:
 
    LangChainHub <https://github.com/hwchase17/langchain-hub>
+   ./additional_resources/deployments.md
+   ./additional_resources/deploy_llms.rst
    Gallery <https://github.com/kyrolabs/awesome-langchain>
    ./additional_resources/tracing.md
    ./additional_resources/model_laboratory.ipynb

docs/integrations/amazon_bedrock.md

@@ -18,7 +18,7 @@ from langchain import Bedrock
 
 ## Text Embedding Models
 
-See a [usage example](../modules/models/text_embedding/examples/bedrock.ipynb).
+See a [usage example](../modules/models/text_embedding/examples/amazon_bedrock.ipynb).
 ```python
 from langchain.embeddings import BedrockEmbeddings
 ```

docs/integrations/annoy.md

@@ -0,0 +1,18 @@
+# Annoy
+
+> [Annoy](https://github.com/spotify/annoy) (`Approximate Nearest Neighbors Oh Yeah`) is a C++ library with Python bindings to search for points in space that are close to a given query point. It also creates large read-only file-based data structures that are mmapped into memory so that many processes may share the same data. 
+## Installation and Setup
+
+
+```bash
+pip install annoy
+```
+
+
+## Vectorstore
+
+See a [usage example](../modules/indexes/vectorstores/examples/annoy.ipynb).
+
+```python
+from langchain.vectorstores import Annoy
+```

docs/integrations/anthropic.md

@@ -0,0 +1,26 @@
+# Anthropic
+
+>[Anthropic](https://en.wikipedia.org/wiki/Anthropic) is an American artificial intelligence (AI) startup and 
+> public-benefit corporation, founded by former members of OpenAI. `Anthropic` specializes in developing general AI 
+> systems and language models, with a company ethos of responsible AI usage.
+> `Anthropic` develops a chatbot, named `Claude`. Similar to `ChatGPT`, `Claude` uses a messaging 
+> interface where users can submit questions or requests and receive highly detailed and relevant responses.
+ 
+## Installation and Setup
+
+
+```bash
+pip install anthropic
+```
+
+See the [setup documentation](https://console.anthropic.com/docs/access).
+
+
+
+## Chat Models
+
+See a [usage example](../modules/models/chat/integrations/anthropic.ipynb)
+
+```python
+from langchain.chat_models import ChatAnthropic
+```

docs/integrations/argilla.md

@@ -0,0 +1,29 @@
+# Argilla
+
+![Argilla - Open-source data platform for LLMs](https://argilla.io/og.png)
+
+>[Argilla](https://argilla.io/) is an open-source data curation platform for LLMs.
+> Using Argilla, everyone can build robust language models through faster data curation 
+> using both human and machine feedback. We provide support for each step in the MLOps cycle, 
+> from data labeling to model monitoring.
+
+## Installation and Setup
+
+First, you'll need to install the  `argilla` Python package as follows:
+
+```bash
+pip install argilla --upgrade
+```
+
+If you already have an Argilla Server running, then you're good to go; but if
+you don't, follow the next steps to install it.
+
+If you don't you can refer to [Argilla - 🚀 Quickstart](https://docs.argilla.io/en/latest/getting_started/quickstart.html#Running-Argilla-Quickstart) to deploy Argilla either on HuggingFace Spaces, locally, or on a server.
+
+## Tracking
+
+See a [usage example of `ArgillaCallbackHandler`](../modules/callbacks/examples/examples/argilla.ipynb).
+
+```python
+from langchain.callbacks import ArgillaCallbackHandler
+```

docs/integrations/arxiv.md

@@ -26,3 +26,11 @@ See a [usage example](../modules/indexes/document_loaders/examples/arxiv.ipynb).
 ```python
 from langchain.document_loaders import ArxivLoader
 ```
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/arxiv.ipynb).
+
+```python
+from langchain.retrievers import ArxivRetriever
+```

docs/integrations/awadb.md

@@ -0,0 +1,21 @@
+# AwaDB
+
+>[AwaDB](https://github.com/awa-ai/awadb) is an AI Native database for the search and storage of embedding vectors used by LLM Applications.
+
+## Installation and Setup
+
+```bash
+pip install awadb
+```
+
+
+## VectorStore
+
+There exists a wrapper around AwaDB vector databases, allowing you to use it as a vectorstore,
+whether for semantic search or example selection.
+
+```python
+from langchain.vectorstores import AwaDB
+```
+
+For a more detailed walkthrough of the AwaDB wrapper, see [this notebook](../modules/indexes/vectorstores/examples/awadb.ipynb)

docs/integrations/azure_cognitive_search_.md

@@ -0,0 +1,24 @@
+# Azure Cognitive Search
+
+>[Azure Cognitive Search](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) (formerly known as `Azure Search`) is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.
+
+>Search is foundational to any app that surfaces text to users, where common scenarios include catalog or document search, online retail apps, or data exploration over proprietary content. When you create a search service, you'll work with the following capabilities:
+>- A search engine for full text search over a search index containing user-owned content
+>- Rich indexing, with lexical analysis and optional AI enrichment for content extraction and transformation
+>- Rich query syntax for text search, fuzzy search, autocomplete, geo-search and more
+>- Programmability through REST APIs and client libraries in Azure SDKs
+>- Azure integration at the data layer, machine learning layer, and AI (Cognitive Services)
+
+
+## Installation and Setup
+
+See [set up instructions](https://learn.microsoft.com/en-us/azure/search/search-create-service-portal).
+
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/azure_cognitive_search.ipynb).
+
+```python
+from langchain.retrievers import AzureCognitiveSearchRetriever
+```

docs/integrations/beam.md

@@ -1,7 +1,8 @@
 # Beam
 
-This page covers how to use Beam within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Beam wrappers.
+>[Beam](https://docs.beam.cloud/introduction) makes it easy to run code on GPUs, deploy scalable web APIs, 
+> schedule cron jobs, and run massively parallel workloads — without managing any infrastructure.
+ 
 
 ## Installation and Setup
 
@@ -9,19 +10,19 @@ It is broken into two parts: installation and setup, and then references to spec
 - Install the Beam CLI with `curl https://raw.githubusercontent.com/slai-labs/get-beam/main/get-beam.sh -sSfL | sh`
 - Register API keys with `beam configure`
 - Set environment variables (`BEAM_CLIENT_ID`) and (`BEAM_CLIENT_SECRET`)
-- Install the Beam SDK `pip install beam-sdk`
+- Install the Beam SDK:
+```bash
+pip install beam-sdk
+```
 
-## Wrappers
+## LLM
 
-### LLM
-
-There exists a Beam LLM wrapper, which you can access with
 
 ```python
 from langchain.llms.beam import Beam
 ```
 
-## Define your Beam app.
+### Example of the Beam app
 
 This is the environment you’ll be developing against once you start the app.
 It's also used to define the maximum response length from the model.
@@ -44,7 +45,7 @@ llm = Beam(model_name="gpt2",
            verbose=False)
 ```
 
-## Deploy your Beam app
+### Deploy the Beam app
 
 Once defined, you can deploy your Beam app by calling your model's `_deploy()` method.
 
@@ -52,9 +53,9 @@ Once defined, you can deploy your Beam app by calling your model's `_deploy()` m
 llm._deploy()
 ```
 
-## Call your Beam app
+### Call the Beam app
 
-Once a beam model is deployed, it can be called by callying your model's `_call()` method.
+Once a beam model is deployed, it can be called by calling your model's `_call()` method.
 This returns the GPT2 text response to your prompt.
 
 ```python

docs/integrations/cassandra.md

@@ -0,0 +1,23 @@
+# Cassandra
+
+>[Cassandra](https://en.wikipedia.org/wiki/Apache_Cassandra) is a free and open-source, distributed, wide-column 
+> store, NoSQL database management system designed to handle large amounts of data across many commodity servers, 
+> providing high availability with no single point of failure. `Cassandra` offers support for clusters spanning 
+> multiple datacenters, with asynchronous masterless replication allowing low latency operations for all clients. 
+> `Cassandra` was designed to implement a combination of `Amazon's Dynamo` distributed storage and replication 
+> techniques combined with `Google's Bigtable` data and storage engine model.
+ 
+## Installation and Setup
+
+```bash
+pip install cassandra-drive
+```
+
+
+## Memory
+
+See a [usage example](../modules/memory/examples/cassandra_chat_message_history.ipynb).
+
+```python
+from langchain.memory import CassandraChatMessageHistory
+```

docs/integrations/chroma.md

@@ -1,20 +1,29 @@
 # Chroma
 
-This page covers how to use the Chroma ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Chroma wrappers.
+>[Chroma](https://docs.trychroma.com/getting-started) is a database for building AI applications with embeddings.
 
 ## Installation and Setup
-- Install the Python package with `pip install chromadb`
-## Wrappers
 
-### VectorStore
+```bash
+pip install chromadb
+```
+
+
+## VectorStore
 
 There exists a wrapper around Chroma vector databases, allowing you to use it as a vectorstore,
 whether for semantic search or example selection.
 
-To import this vectorstore:
 ```python
 from langchain.vectorstores import Chroma
 ```
 
 For a more detailed walkthrough of the Chroma wrapper, see [this notebook](../modules/indexes/vectorstores/getting_started.ipynb)
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/chroma_self_query.ipynb).
+
+```python
+from langchain.retrievers import SelfQueryRetriever
+```

docs/integrations/clickhouse.md

@@ -0,0 +1,52 @@
+# ClickHouse
+
+This page covers how to use ClickHouse Vector Search within LangChain.
+
+[ClickHouse](https://clickhouse.com) is a open source real-time OLAP database with full SQL support and a wide range of functions to assist users in writing analytical queries. Some of these functions and data structures perform distance operations between vectors, enabling ClickHouse to be used as a vector database.
+
+Due to the fully parallelized query pipeline, ClickHouse can process vector search operations very quickly, especially when performing exact matching through a linear scan over all rows, delivering processing speed comparable to dedicated vector databases.
+
+High compression levels, tunable through custom compression codecs, enable very large datasets to be stored and queried. ClickHouse is not memory-bound, allowing multi-TB datasets containing embeddings to be queried.
+
+The capabilities for computing the distance between two vectors are just another SQL function and can be effectively combined with more traditional SQL filtering and aggregation capabilities. This allows vectors to be stored and queried alongside metadata, and even rich text, enabling a broad array of use cases and applications.
+
+Finally, experimental ClickHouse capabilities like [Approximate Nearest Neighbour (ANN) indices](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/annindexes) support faster approximate matching of vectors and provide a promising development aimed to further enhance the vector matching capabilities of ClickHouse.
+
+## Installation
+- Install clickhouse server by [binary](https://clickhouse.com/docs/en/install) or [docker image](https://hub.docker.com/r/clickhouse/clickhouse-server/)
+- Install the Python SDK with `pip install clickhouse-connect`
+
+### Configure clickhouse vector index
+
+Customize `ClickhouseSettings` object with parameters
+
+    ```python
+    from langchain.vectorstores import ClickHouse, ClickhouseSettings
+    config = ClickhouseSettings(host="<clickhouse-server-host>", port=8123, ...)
+    index = Clickhouse(embedding_function, config)
+    index.add_documents(...)
+    ```
+  
+## Wrappers
+supported functions:
+- `add_texts`
+- `add_documents`
+- `from_texts`
+- `from_documents`
+- `similarity_search`
+- `asimilarity_search`
+- `similarity_search_by_vector`
+- `asimilarity_search_by_vector`
+- `similarity_search_with_relevance_scores`
+
+### VectorStore
+
+There exists a wrapper around open source Clickhouse database, allowing you to use it as a vectorstore,
+whether for semantic search or similar example retrieval.
+
+To import this vectorstore:
+```python
+from langchain.vectorstores import Clickhouse
+```
+
+For a more detailed walkthrough of the MyScale wrapper, see [this notebook](../modules/indexes/vectorstores/examples/clickhouse.ipynb)

docs/integrations/cohere.md

@@ -1,25 +1,38 @@
 # Cohere
 
-This page covers how to use the Cohere ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Cohere wrappers.
+>[Cohere](https://cohere.ai/about) is a Canadian startup that provides natural language processing models
+> that help companies improve human-machine interactions.
 
 ## Installation and Setup
-- Install the Python SDK with `pip install cohere`
-- Get an Cohere api key and set it as an environment variable (`COHERE_API_KEY`)
+- Install the Python SDK :
+```bash
+pip install cohere
+```
 
-## Wrappers
+Get a [Cohere api key](https://dashboard.cohere.ai/) and set it as an environment variable (`COHERE_API_KEY`)
 
-### LLM
+
+## LLM
 
 There exists an Cohere LLM wrapper, which you can access with 
+See a [usage example](../modules/models/llms/integrations/cohere.ipynb).
+
 ```python
 from langchain.llms import Cohere
 ```
 
-### Embeddings
+## Text Embedding Model
 
-There exists an Cohere Embeddings wrapper, which you can access with 
+There exists an Cohere Embedding model, which you can access with 
 ```python
 from langchain.embeddings import CohereEmbeddings
 ```
 For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/cohere.ipynb)
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/cohere-reranker.ipynb).
+
+```python
+from langchain.retrievers.document_compressors import CohereRerank
+```

docs/integrations/databerry.md

@@ -1,25 +1,17 @@
 # Databerry
 
-This page covers how to use the [Databerry](https://databerry.ai) within LangChain.
+>[Databerry](https://databerry.ai) is an [open source](https://github.com/gmpetrov/databerry) document retrieval platform that helps to connect your personal data with Large Language Models.
 
-## What is Databerry?
 
-Databerry is an [open source](https://github.com/gmpetrov/databerry) document retrievial platform that helps to connect your personal data with Large Language Models.
+## Installation and Setup
 
-![Databerry](../_static/DataberryDashboard.png)
+We need to sign up for Databerry, create a datastore, add some data and get your datastore api endpoint url. 
+We need the [API Key](https://docs.databerry.ai/api-reference/authentication).
 
-## Quick start
+## Retriever
 
-Retrieving documents stored in Databerry from LangChain is very easy!
+See a [usage example](../modules/indexes/retrievers/examples/databerry.ipynb).
 
 ```python
 from langchain.retrievers import DataberryRetriever
-
-retriever = DataberryRetriever(
-    datastore_url="https://api.databerry.ai/query/clg1xg2h80000l708dymr0fxc",
-    # api_key="DATABERRY_API_KEY", # optional if datastore is public
-    # top_k=10 # optional
-)
-
-docs = retriever.get_relevant_documents("What's Databerry?")
 ```

docs/integrations/databricks.md

@@ -0,0 +1,36 @@
+Databricks
+==========
+
+The [Databricks](https://www.databricks.com/) Lakehouse Platform unifies data, analytics, and AI on one platform.
+
+Databricks embraces the LangChain ecosystem in various ways:
+
+1. Databricks connector for the SQLDatabase Chain: SQLDatabase.from_databricks() provides an easy way to query your data on Databricks through LangChain
+2. Databricks-managed MLflow integrates with LangChain: Tracking and serving LangChain applications with fewer steps
+3. Databricks as an LLM provider: Deploy your fine-tuned LLMs on Databricks via serving endpoints or cluster driver proxy apps, and query it as langchain.llms.Databricks
+4. Databricks Dolly: Databricks open-sourced Dolly which allows for commercial use, and can be accessed through the Hugging Face Hub
+
+Databricks connector for the SQLDatabase Chain
+----------------------------------------------
+You can connect to [Databricks runtimes](https://docs.databricks.com/runtime/index.html) and [Databricks SQL](https://www.databricks.com/product/databricks-sql) using the SQLDatabase wrapper of LangChain. See the notebook [Connect to Databricks](./databricks/databricks.html) for details.
+
+Databricks-managed MLflow integrates with LangChain
+---------------------------------------------------
+
+MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. See the notebook [MLflow Callback Handler](./mlflow_tracking.ipynb) for details about MLflow's integration with LangChain.
+
+Databricks provides a fully managed and hosted version of MLflow integrated with enterprise security features, high availability, and other Databricks workspace features such as experiment and run management and notebook revision capture. MLflow on Databricks offers an integrated experience for tracking and securing machine learning model training runs and running machine learning projects. See [MLflow guide](https://docs.databricks.com/mlflow/index.html) for more details.
+
+Databricks-managed MLflow makes it more convenient to develop LangChain applications on Databricks. For MLflow tracking, you don't need to set the tracking uri. For MLflow Model Serving, you can save LangChain Chains in the MLflow langchain flavor, and then register and serve the Chain with a few clicks on Databricks, with credentials securely managed by MLflow Model Serving.
+
+Databricks as an LLM provider
+-----------------------------
+
+The notebook [Wrap Databricks endpoints as LLMs](../modules/models/llms/integrations/databricks.html) illustrates the method to wrap Databricks endpoints as LLMs in LangChain. It supports two types of endpoints: the serving endpoint, which is recommended for both production and development, and the cluster driver proxy app, which is recommended for interactive development. 
+
+Databricks endpoints support Dolly, but are also great for hosting models like MPT-7B or any other models from the Hugging Face ecosystem. Databricks endpoints can also be used with proprietary models like OpenAI to provide a governance layer for enterprises.
+
+Databricks Dolly
+----------------
+
+Databricks’ Dolly is an instruction-following large language model trained on the Databricks machine learning platform that is licensed for commercial use. The model is available on Hugging Face Hub as databricks/dolly-v2-12b. See the notebook [Hugging Face Hub](../modules/models/llms/integrations/huggingface_hub.html) for instructions to access it through the Hugging Face Hub integration with LangChain. 

docs/integrations/databricks/databricks.ipynb

@@ -58,7 +58,7 @@
     "### Optional Parameters\n",
     "There following parameters are optional. When executing the method in a Databricks notebook, you don't need to provide them in most of the cases.\n",
     "* `host`: The Databricks workspace hostname, excluding 'https://' part. Defaults to 'DATABRICKS_HOST' environment variable or current workspace if in a Databricks notebook.\n",
-    "* `api_token`: The Databricks personal access token for accessing the Databricks SQL warehouse or the cluster. Defaults to 'DATABRICKS_API_TOKEN' environment variable or a temporary one is generated if in a Databricks notebook.\n",
+    "* `api_token`: The Databricks personal access token for accessing the Databricks SQL warehouse or the cluster. Defaults to 'DATABRICKS_TOKEN' environment variable or a temporary one is generated if in a Databricks notebook.\n",
     "* `warehouse_id`: The warehouse ID in the Databricks SQL.\n",
     "* `cluster_id`: The cluster ID in the Databricks Runtime. If running in a Databricks notebook and both 'warehouse_id' and 'cluster_id' are None, it uses the ID of the cluster the notebook is attached to.\n",
     "* `engine_args`: The arguments to be used when connecting Databricks.\n",

docs/integrations/discord.md

@@ -0,0 +1,30 @@
+# Discord
+
+>[Discord](https://discord.com/) is a VoIP and instant messaging social platform. Users have the ability to communicate 
+> with voice calls, video calls, text messaging, media and files in private chats or as part of communities called 
+> "servers". A server is a collection of persistent chat rooms and voice channels which can be accessed via invite links.
+
+## Installation and Setup
+
+
+```bash
+pip install pandas
+```
+
+Follow these steps to download your `Discord` data:
+
+1. Go to your **User Settings**
+2. Then go to **Privacy and Safety**
+3. Head over to the **Request all of my Data** and click on **Request Data** button
+
+It might take 30 days for you to receive your data. You'll receive an email at the address which is registered 
+with Discord. That email will have a download button using which you would be able to download your personal Discord data.
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/discord.ipynb).
+
+```python
+from langchain.document_loaders import DiscordChatLoader
+```

docs/integrations/docugami.md

@@ -1,25 +1,20 @@
 # Docugami
 
->[Docugami](https://docugami.com) converts business documents into a Document XML Knowledge Graph, generating forests of 
-> XML semantic trees representing entire documents. 
-> This is a rich representation that includes the semantic and 
+>[Docugami](https://docugami.com) converts business documents into a Document XML Knowledge Graph, generating forests 
+> of XML semantic trees representing entire documents. This is a rich representation that includes the semantic and 
 > structural characteristics of various chunks in the document as an XML tree.
 
+## Installation and Setup
 
-## Quick start
 
-1. Create a Docugami workspace: <a href="http://www.docugami.com">http://www.docugami.com</a> (free trials available)
-2. Add your documents (PDF, DOCX or DOC) and allow Docugami to ingest and cluster them into sets of similar documents, e.g. NDAs, Lease Agreements, and Service Agreements. There is no fixed set of document types supported by the system, the clusters created depend on your particular documents, and you can [change the docset assignments](https://help.docugami.com/home/working-with-the-doc-sets-view) later.
-3. Create an access token via the Developer Playground for your workspace. Detailed instructions: https://help.docugami.com/home/docugami-api
-4. Explore the Docugami API at <a href="https://api-docs.docugami.com">https://api-docs.docugami.com</a> to get a list of your processed docset IDs, or just the document IDs for a particular docset. 
-6. Use the DocugamiLoader as detailed in [this notebook](../modules/indexes/document_loaders/examples/docugami.ipynb), to get rich semantic chunks for your documents.
-7. Optionally, build and publish one or more [reports or abstracts](https://help.docugami.com/home/reports). This helps Docugami improve the semantic XML with better tags based on your preferences, which are then added to the DocugamiLoader output as metadata. Use techniques like [self-querying retriever](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/self_query_retriever.html) to do high accuracy Document QA.
+```bash
+pip install lxml
+```
 
-## Advantages vs Other Chunking Techniques
+## Document Loader
 
-Appropriate chunking of your documents is critical for retrieval from documents. Many chunking techniques exist, including simple ones that rely on whitespace and recursive chunk splitting based on character length. Docugami offers a different approach:
+See a [usage example](../modules/indexes/document_loaders/examples/docugami.ipynb).
 
-1.	**Intelligent Chunking:** Docugami breaks down every document into a hierarchical semantic XML tree of chunks of varying sizes, from single words or numerical values to entire sections. These chunks follow the semantic contours of the document, providing a more meaningful representation than arbitrary length or simple whitespace-based chunking.
-2.	**Structured Representation:** In addition, the XML tree indicates the structural contours of every document, using attributes denoting headings, paragraphs, lists, tables, and other common elements, and does that consistently across all supported document formats, such as scanned PDFs or DOCX files. It appropriately handles long-form document characteristics like page headers/footers or multi-column flows for clean text extraction.
-3.	**Semantic Annotations:** Chunks are annotated with semantic tags that are coherent across the document set, facilitating consistent hierarchical queries across multiple documents, even if they are written and formatted differently. For example, in set of lease agreements, you can easily identify key provisions like the Landlord, Tenant, or Renewal Date, as well as more complex information such as the wording of any sub-lease provision or whether a specific jurisdiction has an exception section within a Termination Clause.
-4.	**Additional Metadata:** Chunks are also annotated with additional metadata, if a user has been using Docugami. This additional metadata can be used for high-accuracy Document QA without context window restrictions. See detailed code walk-through in [this notebook](../modules/indexes/document_loaders/examples/docugami.ipynb).
+```python
+from langchain.document_loaders import DocugamiLoader
+```

docs/integrations/duckdb.md

@@ -0,0 +1,19 @@
+# DuckDB
+
+>[DuckDB](https://duckdb.org/) is an in-process SQL OLAP database management system.
+
+## Installation and Setup
+
+First, you need to install `duckdb` python package.
+
+```bash
+pip install duckdb
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/duckdb.ipynb).
+
+```python
+from langchain.document_loaders import DuckDBLoader
+```

docs/integrations/elasticsearch.md

@@ -0,0 +1,24 @@
+# Elasticsearch
+
+>[Elasticsearch](https://www.elastic.co/elasticsearch/) is a distributed, RESTful search and analytics engine. 
+> It provides a distributed, multi-tenant-capable full-text search engine with an HTTP web interface and schema-free 
+> JSON documents.
+
+
+## Installation and Setup
+
+```bash
+pip install elasticsearch
+```
+
+## Retriever
+
+>In information retrieval, [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25) (BM is an abbreviation of best matching) is a ranking function used by search engines to estimate the relevance of documents to a given search query. It is based on the probabilistic retrieval framework developed in the 1970s and 1980s by Stephen E. Robertson, Karen Spärck Jones, and others.
+
+>The name of the actual ranking function is BM25. The fuller name, Okapi BM25, includes the name of the first system to use it, which was the Okapi information retrieval system, implemented at London's City University in the 1980s and 1990s. BM25 and its newer variants, e.g. BM25F (a version of BM25 that can take document structure and anchor text into account), represent TF-IDF-like retrieval functions used in document retrieval.
+
+See a [usage example](../modules/indexes/retrievers/examples/elastic_search_bm25.ipynb).
+
+```python
+from langchain.retrievers import ElasticSearchBM25Retriever
+```

docs/integrations/evernote.md

@@ -0,0 +1,20 @@
+# EverNote
+
+>[EverNote](https://evernote.com/) is intended for archiving and creating notes in which photos, audio and saved web content can be embedded. Notes are stored in virtual "notebooks" and can be tagged, annotated, edited, searched, and exported.
+
+## Installation and Setup
+
+First, you need to install `lxml` and `html2text` python packages.
+
+```bash
+pip install lxml
+pip install html2text
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/evernote.ipynb).
+
+```python
+from langchain.document_loaders import EverNoteLoader
+```

docs/integrations/facebook_chat.md

@@ -0,0 +1,21 @@
+# Facebook Chat
+
+>[Messenger](https://en.wikipedia.org/wiki/Messenger_(software)) is an American proprietary instant messaging app and 
+> platform developed by `Meta Platforms`. Originally developed as `Facebook Chat` in 2008, the company revamped its
+> messaging service in 2010.
+
+## Installation and Setup
+
+First, you need to install `pandas` python package.
+
+```bash
+pip install pandas
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/facebook_chat.ipynb).
+
+```python
+from langchain.document_loaders import FacebookChatLoader
+```

docs/integrations/figma.md

@@ -0,0 +1,21 @@
+# Figma
+
+>[Figma](https://www.figma.com/) is a collaborative web application for interface design.
+
+## Installation and Setup
+
+The Figma API requires an `access token`, `node_ids`, and a `file key`.
+
+The `file key` can be pulled from the URL.  https://www.figma.com/file/{filekey}/sampleFilename
+
+`Node IDs` are also available in the URL. Click on anything and look for the '?node-id={node_id}' param.
+
+`Access token` [instructions](https://help.figma.com/hc/en-us/articles/8085703771159-Manage-personal-access-tokens).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/figma.ipynb).
+
+```python
+from langchain.document_loaders import FigmaFileLoader
+```

docs/integrations/git.md

@@ -0,0 +1,19 @@
+# Git
+
+>[Git](https://en.wikipedia.org/wiki/Git) is a distributed version control system that tracks changes in any set of computer files, usually used for coordinating work among programmers collaboratively developing source code during software development.
+
+## Installation and Setup
+
+First, you need to install `GitPython` python package.
+
+```bash
+pip install GitPython
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/git.ipynb).
+
+```python
+from langchain.document_loaders import GitLoader
+```

docs/integrations/gitbook.md

@@ -0,0 +1,15 @@
+# GitBook
+
+>[GitBook](https://docs.gitbook.com/) is a modern documentation platform where teams can document everything from products to internal knowledge bases and APIs.
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/gitbook.ipynb).
+
+```python
+from langchain.document_loaders import GitbookLoader
+```

docs/integrations/google_bigquery.md

@@ -0,0 +1,20 @@
+# Google BigQuery
+
+>[Google BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
+`BigQuery` is a part of the `Google Cloud Platform`.
+
+## Installation and Setup
+
+First, you need to install `google-cloud-bigquery` python package.
+
+```bash
+pip install google-cloud-bigquery
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/google_bigquery.ipynb).
+
+```python
+from langchain.document_loaders import BigQueryLoader
+```

docs/integrations/google_cloud_storage.md

@@ -0,0 +1,26 @@
+# Google Cloud Storage
+
+>[Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
+
+## Installation and Setup
+
+First, you need to install `google-cloud-bigquery` python package.
+
+```bash
+pip install google-cloud-storage
+```
+
+## Document Loader
+
+There are two loaders for the `Google Cloud Storage`: the `Directory` and the `File` loaders.
+
+See a [usage example](../modules/indexes/document_loaders/examples/google_cloud_storage_directory.ipynb).
+
+```python
+from langchain.document_loaders import GCSDirectoryLoader
+```
+See a [usage example](../modules/indexes/document_loaders/examples/google_cloud_storage_file.ipynb).
+
+```python
+from langchain.document_loaders import GCSFileLoader
+```

docs/integrations/google_drive.md

@@ -0,0 +1,22 @@
+# Google Drive
+
+>[Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.
+
+Currently, only `Google Docs` are supported.
+
+## Installation and Setup
+
+First, you need to install several python package.
+
+```bash
+pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
+```
+
+## Document Loader
+
+See a [usage example and authorizing instructions](../modules/indexes/document_loaders/examples/google_drive.ipynb).
+
+
+```python
+from langchain.document_loaders import GoogleDriveLoader
+```

docs/integrations/google_vertex_ai.md

@@ -0,0 +1,24 @@
+# Google Vertex AI
+
+>[Vertex AI](https://cloud.google.com/vertex-ai/docs/start/introduction-unified-platform) is a machine learning (ML) 
+> platform that lets you train and deploy ML models and AI applications. 
+> `Vertex AI` combines data engineering, data science, and ML engineering workflows, enabling your teams to 
+> collaborate using a common toolset.
+ 
+## Installation and Setup
+
+
+```bash
+pip install google-cloud-aiplatform
+```
+
+See the [setup instructions](../modules/models/chat/integrations/google_vertex_ai_palm.ipynb)
+
+
+## Chat Models
+
+See a [usage example](../modules/models/chat/integrations/google_vertex_ai_palm.ipynb)
+
+```python
+from langchain.chat_models import ChatVertexAI
+```

docs/integrations/gutenberg.md

@@ -0,0 +1,15 @@
+# Gutenberg
+
+>[Project Gutenberg](https://www.gutenberg.org/about/) is an online library of free eBooks.
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/gutenberg.ipynb).
+
+```python
+from langchain.document_loaders import GutenbergLoader
+```

docs/integrations/hacker_news.md

@@ -0,0 +1,18 @@
+# Hacker News
+
+>[Hacker News](https://en.wikipedia.org/wiki/Hacker_News) (sometimes abbreviated as `HN`) is a social news 
+> website focusing on computer science and entrepreneurship. It is run by the investment fund and startup 
+> incubator `Y Combinator`. In general, content that can be submitted is defined as "anything that gratifies 
+> one's intellectual curiosity."
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/hacker_news.ipynb).
+
+```python
+from langchain.document_loaders import HNLoader
+```

docs/integrations/huggingface.md

@@ -47,7 +47,7 @@ To use a the wrapper for a model hosted on Hugging Face Hub:
 ```python
 from langchain.embeddings import HuggingFaceHubEmbeddings
 ```
-For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/huggingfacehub.ipynb)
+For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/huggingface_hub.ipynb)
 
 ### Tokenizer
 

docs/integrations/ifixit.md

@@ -0,0 +1,16 @@
+# iFixit
+
+>[iFixit](https://www.ifixit.com) is the largest, open repair community on the web. The site contains nearly 100k 
+> repair manuals, 200k Questions & Answers on 42k devices, and all the data is licensed under `CC-BY-NC-SA 3.0`.
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/ifixit.ipynb).
+
+```python
+from langchain.document_loaders import IFixitLoader
+```

docs/integrations/imsdb.md

@@ -0,0 +1,16 @@
+# IMSDb
+
+>[IMSDb](https://imsdb.com/) is the `Internet Movie Script Database`.
+> 
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/imsdb.ipynb).
+
+
+```python
+from langchain.document_loaders import IMSDbLoader
+```

docs/integrations/langchain_decorators.md

@@ -0,0 +1,368 @@
+# LangChain Decorators ✨
+
+lanchchain decorators is a layer on the top of LangChain that provides syntactic sugar 🍭 for writing custom langchain prompts and chains
+
+For Feedback, Issues, Contributions - please raise an issue here: 
+[ju-bezdek/langchain-decorators](https://github.com/ju-bezdek/langchain-decorators)
+
+
+
+Main principles and benefits:
+
+- more `pythonic` way of writing code
+- write multiline prompts that wont break your code flow with indentation
+- making use of IDE in-built support for **hinting**, **type checking** and **popup with docs** to quickly peek in the function to see the prompt, parameters it consumes etc.
+- leverage all the power of 🦜🔗 LangChain ecosystem
+- adding support for **optional parameters**
+- easily share parameters between the prompts by binding them to one class
+
+
+
+Here is a simple example of a code written with **LangChain Decorators ✨**
+
+``` python
+
+@llm_prompt
+def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers")->str:
+    """
+    Write me a short header for my post about {topic} for {platform} platform. 
+    It should be for {audience} audience.
+    (Max 15 words)
+    """
+    return
+
+# run it naturaly
+write_me_short_post(topic="starwars")
+# or
+write_me_short_post(topic="starwars", platform="redit")
+```
+
+# Quick start
+## Installation
+```bash
+pip install langchain_decorators
+```
+
+## Examples
+
+Good idea on how to start is to review the examples here:
+ - [jupyter notebook](https://github.com/ju-bezdek/langchain-decorators/blob/main/example_notebook.ipynb)
+ - [colab notebook](https://colab.research.google.com/drive/1no-8WfeP6JaLD9yUtkPgym6x0G9ZYZOG#scrollTo=N4cf__D0E2Yk)
+
+# Defining other parameters
+Here we are just marking a function as a prompt with `llm_prompt` decorator, turning it effectively into a LLMChain. Instead of running it 
+
+
+Standard LLMchain takes much more init parameter than just inputs_variables and prompt... here is this implementation detail hidden in the decorator.
+Here is how it works:
+
+1. Using **Global settings**:
+
+``` python
+# define global settings for all prompty (if not set - chatGPT is the current default)
+from langchain_decorators import GlobalSettings
+
+GlobalSettings.define_settings(
+    default_llm=ChatOpenAI(temperature=0.0), this is default... can change it here globally
+    default_streaming_llm=ChatOpenAI(temperature=0.0,streaming=True), this is default... can change it here for all ... will be used for streaming
+)
+```
+
+2. Using predefined **prompt types**
+
+``` python
+#You can change the default prompt types
+from langchain_decorators import PromptTypes, PromptTypeSettings
+
+PromptTypes.AGENT_REASONING.llm = ChatOpenAI()
+
+# Or you can just define your own ones:
+class MyCustomPromptTypes(PromptTypes):
+    GPT4=PromptTypeSettings(llm=ChatOpenAI(model="gpt-4"))
+
+@llm_prompt(prompt_type=MyCustomPromptTypes.GPT4) 
+def write_a_complicated_code(app_idea:str)->str:
+    ...
+
+```
+
+3.  Define the settings **directly in the decorator**
+
+``` python
+from langchain.llms import OpenAI
+
+@llm_prompt(
+    llm=OpenAI(temperature=0.7),
+    stop_tokens=["\nObservation"],
+    ...
+    )
+def creative_writer(book_title:str)->str:
+    ...
+```
+
+## Passing a memory and/or callbacks:
+
+To pass any of these, just declare them in the function (or use kwargs to pass anything)
+
+```python
+
+@llm_prompt()
+async def write_me_short_post(topic:str, platform:str="twitter", memory:SimpleMemory = None):
+    """
+    {history_key}
+    Write me a short header for my post about {topic} for {platform} platform. 
+    It should be for {audience} audience.
+    (Max 15 words)
+    """
+    pass
+
+await write_me_short_post(topic="old movies")
+
+```
+
+# Simplified streaming
+
+If we wan't to leverage streaming:
+ - we need to define prompt as async function 
+ - turn on the streaming on the decorator, or we can define PromptType with streaming on
+ - capture the stream using StreamingContext
+
+This way we just mark which prompt should be streamed, not needing to tinker with what LLM should we use, passing around the creating and distribute streaming handler into particular part of our chain... just turn the streaming on/off on prompt/prompt type...
+
+The streaming will happen only if we call it in streaming context ... there we can define a simple function to handle the stream
+
+``` python
+# this code example is complete and should run as it is
+
+from langchain_decorators import StreamingContext, llm_prompt
+
+# this will mark the prompt for streaming (useful if we want stream just some prompts in our app... but don't want to pass distribute the callback handlers)
+# note that only async functions can be streamed (will get an error if it's not)
+@llm_prompt(capture_stream=True) 
+async def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers"):
+    """
+    Write me a short header for my post about {topic} for {platform} platform. 
+    It should be for {audience} audience.
+    (Max 15 words)
+    """
+    pass
+
+
+
+# just an arbitrary  function to demonstrate the streaming... wil be some websockets code in the real world
+tokens=[]
+def capture_stream_func(new_token:str):
+    tokens.append(new_token)
+
+# if we want to capture the stream, we need to wrap the execution into StreamingContext... 
+# this will allow us to capture the stream even if the prompt call is hidden inside higher level method
+# only the prompts marked with capture_stream will be captured here
+with StreamingContext(stream_to_stdout=True, callback=capture_stream_func):
+    result = await run_prompt()
+    print("Stream finished ... we can distinguish tokens thanks to alternating colors")
+
+
+print("\nWe've captured",len(tokens),"tokens🎉\n")
+print("Here is the result:")
+print(result)
+```
+
+
+# Prompt declarations
+By default the prompt is is the whole function docs, unless you mark your prompt 
+
+## Documenting your prompt
+
+We can specify what part of our docs is the prompt definition, by specifying a code block with **<prompt>** language tag
+
+``` python
+@llm_prompt
+def write_me_short_post(topic:str, platform:str="twitter", audience:str = "developers"):
+    """
+    Here is a good way to write a prompt as part of a function docstring, with additional documentation for devs.
+
+    It needs to be a code block, marked as a `<prompt>` language
+    ```<prompt>
+    Write me a short header for my post about {topic} for {platform} platform. 
+    It should be for {audience} audience.
+    (Max 15 words)
+    ```
+
+    Now only to code block above will be used as a prompt, and the rest of the docstring will be used as a description for developers.
+    (It has also a nice benefit that IDE (like VS code) will display the prompt properly (not trying to parse it as markdown, and thus not showing new lines properly))
+    """
+    return 
+```
+
+## Chat messages prompt
+
+For chat models is very useful to define prompt as a set of message templates... here is how to do it:
+
+``` python
+@llm_prompt
+def simulate_conversation(human_input:str, agent_role:str="a pirate"):
+    """
+    ## System message
+     - note the `:system` sufix inside the <prompt:_role_> tag
+     
+
+    ```<prompt:system>
+    You are a {agent_role} hacker. You mus act like one.
+    You reply always in code, using python or javascript code block...
+    for example:
+    
+    ... do not reply with anything else.. just with code - respecting your role.
+    ```
+
+    # human message 
+    (we are using the real role that are enforced by the LLM - GPT supports system, assistant, user)
+    ``` <prompt:user>
+    Helo, who are you
+    ```
+    a reply:
+    
+
+    ``` <prompt:assistant>
+    \``` python <<- escaping inner code block with \ that should be part of the prompt
+    def hello():
+        print("Argh... hello you pesky pirate")
+    \```
+    ```
+    
+    we can also add some history using placeholder
+    ```<prompt:placeholder>
+    {history}
+    ```
+    ```<prompt:user>
+    {human_input}
+    ```
+
+    Now only to code block above will be used as a prompt, and the rest of the docstring will be used as a description for developers.
+    (It has also a nice benefit that IDE (like VS code) will display the prompt properly (not trying to parse it as markdown, and thus not showing new lines properly))
+    """
+    pass
+
+```
+
+the roles here are model native roles (assistant, user, system for chatGPT)
+
+
+
+# Optional sections
+- you can define a whole sections of your prompt that should be optional
+- if any input in the section is missing, the whole section wont be rendered
+
+the syntax for this is as follows:
+
+``` python
+@llm_prompt
+def prompt_with_optional_partials():
+    """
+    this text will be rendered always, but
+
+    {? anything inside this block will be rendered only if all the {value}s parameters are not empty (None | "")   ?}
+
+    you can also place it in between the words
+    this too will be rendered{? , but
+        this  block will be rendered only if {this_value} and {this_value}
+        is not empty?} !
+    """
+```
+
+
+# Output parsers
+
+- llm_prompt decorator natively tries to detect the best output parser based on the output type. (if not set, it returns the raw string)
+- list, dict and pydantic outputs are also supported natively (automaticaly)
+
+``` python
+# this code example is complete and should run as it is
+
+from langchain_decorators import llm_prompt
+
+@llm_prompt
+def write_name_suggestions(company_business:str, count:int)->list:
+    """ Write me {count} good name suggestions for company that {company_business}
+    """
+    pass
+
+write_name_suggestions(company_business="sells cookies", count=5)
+```
+
+## More complex structures
+
+for dict / pydantic you need to specify the formatting instructions... 
+this can be tedious, that's why you can let the output parser gegnerate you the instructions based on the model (pydantic)
+
+``` python
+from langchain_decorators import llm_prompt
+from pydantic import BaseModel, Field
+
+
+class TheOutputStructureWeExpect(BaseModel):
+    name:str = Field (description="The name of the company")
+    headline:str = Field( description="The description of the company (for landing page)")
+    employees:list[str] = Field(description="5-8 fake employee names with their positions")
+
+@llm_prompt()
+def fake_company_generator(company_business:str)->TheOutputStructureWeExpect:
+    """ Generate a fake company that {company_business}
+    {FORMAT_INSTRUCTIONS}
+    """
+    return
+
+company = fake_company_generator(company_business="sells cookies")
+
+# print the result nicely formatted
+print("Company name: ",company.name)
+print("company headline: ",company.headline)
+print("company employees: ",company.employees)
+
+```
+
+
+# Binding the prompt to an object
+
+``` python
+from pydantic import BaseModel
+from langchain_decorators import llm_prompt
+
+class AssistantPersonality(BaseModel):
+    assistant_name:str
+    assistant_role:str
+    field:str
+
+    @property
+    def a_property(self):
+        return "whatever"
+
+    def hello_world(self, function_kwarg:str=None):
+        """
+        We can reference any {field} or {a_property} inside our prompt... and combine it with {function_kwarg} in the method
+        """
+
+    
+    @llm_prompt
+    def introduce_your_self(self)->str:
+        """
+        ``` <prompt:system>
+        You are an assistant named {assistant_name}. 
+        Your role is to act as {assistant_role}
+        ```
+        ```<prompt:user>
+        Introduce your self (in less than 20 words)
+        ```
+        """
+
+    
+
+personality = AssistantPersonality(assistant_name="John", assistant_role="a pirate")
+
+print(personality.introduce_your_self(personality))
+```
+
+
+# More examples:
+
+- these and few more examples are also available in the [colab notebook here](https://colab.research.google.com/drive/1no-8WfeP6JaLD9yUtkPgym6x0G9ZYZOG#scrollTo=N4cf__D0E2Yk)
+- including the [ReAct Agent re-implementation](https://colab.research.google.com/drive/1no-8WfeP6JaLD9yUtkPgym6x0G9ZYZOG#scrollTo=3bID5fryE2Yp) using purely langchain decorators

docs/integrations/mediawikidump.md

@@ -0,0 +1,31 @@
+# MediaWikiDump
+
+>[MediaWiki XML Dumps](https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps) contain the content of a wiki 
+> (wiki pages with all their revisions), without the site-related data. A XML dump does not create a full backup 
+> of the wiki database, the dump does not contain user accounts, images, edit logs, etc.
+
+
+## Installation and Setup
+
+We need to install several python packages.
+
+The `mediawiki-utilities` supports XML schema 0.11 in unmerged branches.
+```bash
+pip install -qU git+https://github.com/mediawiki-utilities/python-mwtypes@updates_schema_0.11
+```
+
+The `mediawiki-utilities mwxml` has a bug, fix PR pending.
+
+```bash
+pip install -qU git+https://github.com/gdedrouas/python-mwxml@xml_format_0.11
+pip install -qU mwparserfromhell
+```
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/mediawikidump.ipynb).
+
+
+```python
+from langchain.document_loaders import MWDumpLoader
+```

docs/integrations/microsoft_onedrive.md

@@ -0,0 +1,22 @@
+# Microsoft OneDrive
+
+>[Microsoft OneDrive](https://en.wikipedia.org/wiki/OneDrive) (formerly `SkyDrive`) is a file-hosting service operated by Microsoft.
+
+## Installation and Setup
+
+First, you need to install a python package.
+
+```bash
+pip install o365
+```
+
+Then follow instructions [here](../modules/indexes/document_loaders/examples/microsoft_onedrive.ipynb).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/microsoft_onedrive.ipynb).
+
+
+```python
+from langchain.document_loaders import OneDriveLoader
+```

docs/integrations/microsoft_powerpoint.md

@@ -0,0 +1,16 @@
+# Microsoft PowerPoint
+
+>[Microsoft PowerPoint](https://en.wikipedia.org/wiki/Microsoft_PowerPoint) is a presentation program by Microsoft.
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/microsoft_powerpoint.ipynb).
+
+
+```python
+from langchain.document_loaders import UnstructuredPowerPointLoader
+```

docs/integrations/microsoft_word.md

@@ -0,0 +1,16 @@
+# Microsoft Word
+
+>[Microsoft Word](https://www.microsoft.com/en-us/microsoft-365/word) is a word processor developed by Microsoft.
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/microsoft_word.ipynb).
+
+
+```python
+from langchain.document_loaders import UnstructuredWordDocumentLoader
+```

docs/integrations/modern_treasury.md

@@ -0,0 +1,19 @@
+# Modern Treasury
+
+>[Modern Treasury](https://www.moderntreasury.com/) simplifies complex payment operations. It is a unified platform to power products and processes that move money.
+>- Connect to banks and payment systems
+>- Track transactions and balances in real-time
+>- Automate payment operations for scale
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/modern_treasury.ipynb).
+
+
+```python
+from langchain.document_loaders import ModernTreasuryLoader
+```

docs/integrations/momento.md

@@ -1,20 +1,21 @@
 # Momento
 
+>[Momento Cache](https://docs.momentohq.com/) is the world's first truly serverless caching service. It provides instant elasticity, scale-to-zero 
+> capability, and blazing-fast performance.  
+> With Momento Cache, you grab the SDK, you get an end point, input a few lines into your code, and you're off and running.
+
 This page covers how to use the [Momento](https://gomomento.com) ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Momento wrappers.
 
 ## Installation and Setup
 
 - Sign up for a free account [here](https://docs.momentohq.com/getting-started) and get an auth token
 - Install the Momento Python SDK with `pip install momento`
 
-## Wrappers
 
-### Cache
+## Cache
 
 The Cache wrapper allows for [Momento](https://gomomento.com) to be used as a serverless, distributed, low-latency cache for LLM prompts and responses.
 
-#### Standard Cache
 
 The standard cache is the go-to use case for [Momento](https://gomomento.com) users in any environment.
 
@@ -44,10 +45,10 @@ cache_name = "langchain"
 langchain.llm_cache = MomentoCache(cache_client, cache_name)
 ```
 
-### Memory
+## Memory
 
 Momento can be used as a distributed memory store for LLMs.
 
-#### Chat Message History Memory
+### Chat Message History Memory
 
 See [this notebook](../modules/memory/examples/momento_chat_message_history.ipynb) for a walkthrough of how to use Momento as a memory store for chat message history.

docs/integrations/notion.md

@@ -0,0 +1,27 @@
+# Notion DB
+
+>[Notion](https://www.notion.so/) is a collaboration platform with modified Markdown support that integrates kanban 
+> boards, tasks, wikis and databases. It is an all-in-one workspace for notetaking, knowledge and data management, 
+> and project and task management.
+
+## Installation and Setup
+
+All instructions are in examples below.
+
+## Document Loader
+
+We have two different loaders: `NotionDirectoryLoader` and `NotionDBLoader`.
+
+See a [usage example for the NotionDirectoryLoader](../modules/indexes/document_loaders/examples/notion.ipynb).
+
+
+```python
+from langchain.document_loaders import NotionDirectoryLoader
+```
+
+See a [usage example for the NotionDBLoader](../modules/indexes/document_loaders/examples/notiondb.ipynb).
+
+
+```python
+from langchain.document_loaders import NotionDBLoader
+```

docs/integrations/obsidian.md

@@ -0,0 +1,19 @@
+# Obsidian
+
+>[Obsidian](https://obsidian.md/) is a powerful and extensible knowledge base
+that works on top of your local folder of plain text files.
+
+## Installation and Setup
+
+All instructions are in examples below.
+
+## Document Loader
+
+
+See a [usage example](../modules/indexes/document_loaders/examples/obsidian.ipynb).
+
+
+```python
+from langchain.document_loaders import ObsidianLoader
+```
+

docs/integrations/openai.md

@@ -35,7 +35,6 @@ from langchain.llms import AzureOpenAI
 For a more detailed walkthrough of the `Azure` wrapper, see [this notebook](../modules/models/llms/integrations/azure_openai_example.ipynb)
 
 
-
 ## Text Embedding Model
 
 ```python
@@ -44,6 +43,14 @@ from langchain.embeddings import OpenAIEmbeddings
 For a more detailed walkthrough of this, see [this notebook](../modules/models/text_embedding/examples/openai.ipynb)
 
 
+## Chat Model
+
+```python
+from langchain.chat_models import ChatOpenAI
+```
+For a more detailed walkthrough of this, see [this notebook](../modules/models/chat/integrations/openai.ipynb)
+
+
 ## Tokenizer
 
 There are several places you can use the `tiktoken` tokenizer. By default, it is used to count tokens
@@ -71,3 +78,11 @@ See a [usage example](../modules/indexes/document_loaders/examples/chatgpt_loade
 ```python
 from langchain.document_loaders.chatgpt import ChatGPTLoader
 ```
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/chatgpt-plugin.ipynb).
+
+```python
+from langchain.retrievers import ChatGPTPluginRetriever
+```

docs/integrations/pinecone.md

@@ -4,17 +4,19 @@ This page covers how to use the Pinecone ecosystem within LangChain.
 It is broken into two parts: installation and setup, and then references to specific Pinecone wrappers.
 
 ## Installation and Setup
-- Install the Python SDK with `pip install pinecone-client`
-## Wrappers
+Install the Python SDK:
+```bash
+pip install pinecone-client
+```
 
-### VectorStore
+
+## Vectorstore
 
 There exists a wrapper around Pinecone indexes, allowing you to use it as a vectorstore,
 whether for semantic search or example selection.
 
-To import this vectorstore:
 ```python
 from langchain.vectorstores import Pinecone
 ```
 
-For a more detailed walkthrough of the Pinecone wrapper, see [this notebook](../modules/indexes/vectorstores/examples/pinecone.ipynb)
+For a more detailed walkthrough of the Pinecone vectorstore, see [this notebook](../modules/indexes/vectorstores/examples/pinecone.ipynb)

docs/integrations/predictionguard.md

@@ -1,19 +1,23 @@
 # Prediction Guard
 
-This page covers how to use the Prediction Guard ecosystem within LangChain.
-It is broken into two parts: installation and setup, and then references to specific Prediction Guard wrappers.
+>[Prediction Guard](https://docs.predictionguard.com/) gives a quick and easy access to state-of-the-art open and closed access LLMs, without  needing to spend days and weeks figuring out all of the implementation details, managing a bunch of different API specs, and setting up the infrastructure for model deployments.
+
 
 ## Installation and Setup
-- Install the Python SDK with `pip install predictionguard`
+- Install the Python SDK:
+```bash
+pip install predictionguard
+```
+
 - Get an Prediction Guard access token (as described [here](https://docs.predictionguard.com/)) and set it as an environment variable (`PREDICTIONGUARD_TOKEN`)
 
-## LLM Wrapper
+## LLM 
 
-There exists a Prediction Guard LLM wrapper, which you can access with 
 ```python
 from langchain.llms import PredictionGuard
 ```
 
+### Example
 You can provide the name of the Prediction Guard model as an argument when initializing the LLM:
 ```python
 pgllm = PredictionGuard(model="MPT-7B-Instruct")
@@ -24,14 +28,12 @@ You can also provide your access token directly as an argument:
 pgllm = PredictionGuard(model="MPT-7B-Instruct", token="<your access token>")
 ```
 
-Finally, you can provide an "output" argument that is used to structure/ control the output of the LLM:
+Also, you can provide an "output" argument that is used to structure/ control the output of the LLM:
 ```python
 pgllm = PredictionGuard(model="MPT-7B-Instruct", output={"type": "boolean"})
 ```
 
-## Example usage
-
-Basic usage of the controlled or guarded LLM wrapper:
+#### Basic usage of the controlled or guarded LLM:
 ```python
 import os
 
@@ -72,7 +74,7 @@ pgllm = PredictionGuard(model="MPT-7B-Instruct",
 pgllm(prompt.format(query="What kind of post is this?"))
 ```
 
-Basic LLM Chaining with the Prediction Guard wrapper:
+#### Basic LLM Chaining with the Prediction Guard:
 ```python
 import os
 

docs/integrations/promptlayer.md

@@ -1,31 +1,35 @@
 # PromptLayer
 
-This page covers how to use [PromptLayer](https://www.promptlayer.com) within LangChain.
-It is broken into two parts: installation and setup, and then references to specific PromptLayer wrappers.
+>[PromptLayer](https://docs.promptlayer.com/what-is-promptlayer/wxpF9EZkUwvdkwvVE9XEvC/how-promptlayer-works/dvgGSxNe6nB1jj8mUVbG8r) 
+> is a devtool that allows you to track, manage, and share your GPT prompt engineering. 
+> It acts as a middleware between your code and OpenAI's python library, recording all your API requests 
+> and saving relevant metadata for easy exploration and search in the [PromptLayer](https://www.promptlayer.com) dashboard.
 
 ## Installation and Setup
 
-If you want to work with PromptLayer:
-- Install the promptlayer python library `pip install promptlayer`
+- Install the `promptlayer` python library 
+```bash
+pip install promptlayer
+```
 - Create a PromptLayer account
 - Create an api token and set it as an environment variable (`PROMPTLAYER_API_KEY`)
 
-## Wrappers
 
-### LLM
+## LLM
 
-There exists an PromptLayer OpenAI LLM wrapper, which you can access with
 ```python
 from langchain.llms import PromptLayerOpenAI
 ```
 
-To tag your requests, use the argument `pl_tags` when instanializing the LLM
+### Example
+
+To tag your requests, use the argument `pl_tags` when instantiating the LLM
 ```python
 from langchain.llms import PromptLayerOpenAI
 llm = PromptLayerOpenAI(pl_tags=["langchain-requests", "chatbot"])
 ```
 
-To get the PromptLayer request id, use the argument `return_pl_id` when instanializing the LLM
+To get the PromptLayer request id, use the argument `return_pl_id` when instantiating the LLM
 ```python
 from langchain.llms import PromptLayerOpenAI
 llm = PromptLayerOpenAI(return_pl_id=True)
@@ -42,8 +46,14 @@ You can use the PromptLayer request ID to add a prompt, score, or other metadata
 
 This LLM is identical to the [OpenAI LLM](./openai.md), except that
 - all your requests will be logged to your PromptLayer account
-- you can add `pl_tags` when instantializing to tag your requests on PromptLayer
-- you can add `return_pl_id` when instantializing to return a PromptLayer request id to use [while tracking requests](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9).
+- you can add `pl_tags` when instantiating to tag your requests on PromptLayer
+- you can add `return_pl_id` when instantiating to return a PromptLayer request id to use [while tracking requests](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9).
 
+## Chat Model
+
+```python
+from langchain.chat_models import PromptLayerChatOpenAI
+```
+
+See a [usage example](../modules/models/chat/integrations/promptlayer_chatopenai.ipynb).
 
-PromptLayer also provides native wrappers for [`PromptLayerChatOpenAI`](../modules/models/chat/integrations/promptlayer_chatopenai.ipynb) and `PromptLayerOpenAIChat`

docs/integrations/psychic.md

@@ -1,19 +1,25 @@
 # Psychic
 
-This page covers how to use [Psychic](https://www.psychic.dev/) within LangChain.
+>[Psychic](https://www.psychic.dev/) is a platform for integrating with SaaS tools like `Notion`, `Zendesk`, 
+> `Confluence`, and `Google Drive` via OAuth and syncing documents from these applications to your SQL or vector
+> database. You can think of it like Plaid for unstructured data. 
 
-## What is Psychic?
+## Installation and Setup
 
-Psychic is a platform for integrating with your customer’s SaaS tools like Notion, Zendesk, Confluence, and Google Drive via OAuth and syncing documents from these applications to your SQL or vector database. You can think of it like Plaid for unstructured data. Psychic is easy to set up - you use it by importing the react library and configuring it with your Sidekick API key, which you can get from the [Psychic dashboard](https://dashboard.psychic.dev/). When your users connect their applications, you can view these connections from the dashboard and retrieve data using the server-side libraries.
-
-## Quick start
+```bash
+pip install psychicapi
+```
 
+Psychic is easy to set up - you import the `react` library and configure it with your `Sidekick API` key, which you get 
+from the [Psychic dashboard](https://dashboard.psychic.dev/). When you connect the applications, you  
+view these connections from the dashboard and retrieve data using the server-side libraries.
+ 
 1. Create an account in the [dashboard](https://dashboard.psychic.dev/).
-2. Use the [react library](https://docs.psychic.dev/sidekick-link) to add the Psychic link modal to your frontend react app. Users will use this to connect their SaaS apps.
-3. Once your user has created a connection, you can use the langchain PsychicLoader by following the [example notebook](../modules/indexes/document_loaders/examples/psychic.ipynb)
+2. Use the [react library](https://docs.psychic.dev/sidekick-link) to add the Psychic link modal to your frontend react app. You will use this to connect the SaaS apps.
+3. Once you have created a connection, you can use the `PsychicLoader` by following the [example notebook](../modules/indexes/document_loaders/examples/psychic.ipynb)
 
 
-# Advantages vs Other Document Loaders
+## Advantages vs Other Document Loaders
 
 1.	**Universal API:** Instead of building OAuth flows and learning the APIs for every SaaS app, you integrate Psychic once and leverage our universal API to retrieve data.
 2.	**Data Syncs:** Data in your customers' SaaS apps can get stale fast. With Psychic you can configure webhooks to keep your documents up to date on a daily or realtime basis.

docs/integrations/ray_serve.ipynb

@@ -0,0 +1,233 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Ray Serve\n",
+    "\n",
+    "[Ray Serve](https://docs.ray.io/en/latest/serve/index.html) is a scalable model serving library for building online inference APIs. Serve is particularly well suited for system composition, enabling you to build a complex inference service consisting of multiple chains and business logic all in Python code. "
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Goal of this notebook\n",
+    "This notebook shows a simple example of how to deploy an OpenAI chain into production. You can extend it to deploy your own self-hosted models where you can easily define amount of hardware resources (GPUs and CPUs) needed to run your model in production efficiently. Read more about available options including autoscaling in the Ray Serve [documentation](https://docs.ray.io/en/latest/serve/getting_started.html).\n"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup Ray Serve\n",
+    "Install ray with `pip install ray[serve]`. "
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## General Skeleton"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The general skeleton for deploying a service is the following:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# 0: Import ray serve and request from starlette\n",
+    "from ray import serve\n",
+    "from starlette.requests import Request\n",
+    "\n",
+    "# 1: Define a Ray Serve deployment.\n",
+    "@serve.deployment\n",
+    "class LLMServe:\n",
+    "\n",
+    "    def __init__(self) -> None:\n",
+    "        # All the initialization code goes here\n",
+    "        pass\n",
+    "\n",
+    "    async def __call__(self, request: Request) -> str:\n",
+    "        # You can parse the request here\n",
+    "        # and return a response\n",
+    "        return \"Hello World\"\n",
+    "\n",
+    "# 2: Bind the model to deployment\n",
+    "deployment = LLMServe.bind()\n",
+    "\n",
+    "# 3: Run the deployment\n",
+    "serve.api.run(deployment)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Shutdown the deployment\n",
+    "serve.api.shutdown()"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Example of deploying and OpenAI chain with custom prompts"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Get an OpenAI API key from [here](https://platform.openai.com/account/api-keys). By running the following code, you will be asked to provide your API key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain import PromptTemplate, LLMChain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from getpass import getpass\n",
+    "OPENAI_API_KEY = getpass()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "@serve.deployment\n",
+    "class DeployLLM:\n",
+    "\n",
+    "    def __init__(self):\n",
+    "        # We initialize the LLM, template and the chain here\n",
+    "        llm = OpenAI(openai_api_key=OPENAI_API_KEY)\n",
+    "        template = \"Question: {question}\\n\\nAnswer: Let's think step by step.\"\n",
+    "        prompt = PromptTemplate(template=template, input_variables=[\"question\"])\n",
+    "        self.chain = LLMChain(llm=llm, prompt=prompt)\n",
+    "\n",
+    "    def _run_chain(self, text: str):\n",
+    "        return self.chain(text)\n",
+    "\n",
+    "    async def __call__(self, request: Request):\n",
+    "        # 1. Parse the request\n",
+    "        text = request.query_params[\"text\"]\n",
+    "        # 2. Run the chain\n",
+    "        resp = self._run_chain(text)\n",
+    "        # 3. Return the response\n",
+    "        return resp[\"text\"]"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we can bind the deployment."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Bind the model to deployment\n",
+    "deployment = DeployLLM.bind()"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can assign the port number and host when we want to run the deployment. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Example port number\n",
+    "PORT_NUMBER = 8282\n",
+    "# Run the deployment\n",
+    "serve.api.run(deployment, port=PORT_NUMBER)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now that service is deployed on port `localhost:8282` we can send a post request to get the results back."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import requests\n",
+    "\n",
+    "text = \"What NFL team won the Super Bowl in the year Justin Beiber was born?\"\n",
+    "response = requests.post(f'http://localhost:{PORT_NUMBER}/?text={text}')\n",
+    "print(response.content.decode())"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "ray",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/integrations/reddit.md

@@ -0,0 +1,22 @@
+# Reddit
+
+>[Reddit](www.reddit.com) is an American social news aggregation, content rating, and discussion website.
+
+## Installation and Setup
+
+First, you need to install a python package.
+
+```bash
+pip install praw
+```
+
+Make a [Reddit Application](https://www.reddit.com/prefs/apps/) and initialize the loader with with your Reddit API credentials.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/reddit.ipynb).
+
+
+```python
+from langchain.document_loaders import RedditPostsLoader
+```

docs/integrations/roam.md

@@ -0,0 +1,17 @@
+# Roam
+
+>[ROAM](https://roamresearch.com/) is a note-taking tool for networked thought, designed to create a personal knowledge base.
+ 
+## Installation and Setup
+
+There isn't any special setup for it.
+
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/roam.ipynb).
+
+```python
+from langchain.document_loaders import RoamLoader
+```

docs/integrations/shaleprotocol.md

@@ -0,0 +1,43 @@
+# Shale Protocol
+
+[Shale Protocol](https://shaleprotocol.com) provides production-ready inference APIs for open LLMs. It's a Plug & Play API as it's hosted on a highly scalable GPU cloud infrastructure. 
+
+Our free tier supports up to 1K daily requests per key as we want to eliminate the barrier for anyone to start building genAI apps with LLMs. 
+
+With Shale Protocol, developers/researchers can create apps and explore the capabilities of open LLMs at no cost.
+
+This page covers how Shale-Serve API can be incorporated with LangChain.
+
+As of June 2023, the API supports Vicuna-13B by default. We are going to support more LLMs such as Falcon-40B in future releases. 
+
+
+## How to
+
+### 1. Find the link to our Discord on https://shaleprotocol.com. Generate an API key through the "Shale Bot" on our Discord. No credit card is required and no free trials. It's a forever free tier with 1K limit per day per API key.
+
+### 2. Use https://shale.live/v1 as OpenAI API drop-in replacement 
+
+For example
+```python
+from langchain.llms import OpenAI
+from langchain import PromptTemplate, LLMChain
+
+import os
+os.environ['OPENAI_API_BASE'] = "https://shale.live/v1"
+os.environ['OPENAI_API_KEY'] = "ENTER YOUR API KEY"
+
+llm = OpenAI()
+
+template = """Question: {question}
+
+# Answer: Let's think step by step."""
+
+prompt = PromptTemplate(template=template, input_variables=["question"])
+
+llm_chain = LLMChain(prompt=prompt, llm=llm)
+
+question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
+
+llm_chain.run(question)
+
+```

docs/integrations/slack.md

@@ -0,0 +1,17 @@
+# Slack
+
+>[Slack](https://slack.com/) is an instant messaging program.
+ 
+## Installation and Setup
+
+There isn't any special setup for it.
+
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/slack.ipynb).
+
+```python
+from langchain.document_loaders import SlackDirectoryLoader
+```

docs/integrations/spacy.md

@@ -0,0 +1,20 @@
+# spaCy
+
+>[spaCy](https://spacy.io/) is an open-source software library for advanced natural language processing, written in the programming languages Python and Cython.
+ 
+## Installation and Setup
+
+
+```bash
+pip install spacy
+```
+
+
+
+## Text Splitter
+
+See a [usage example](../modules/indexes/text_splitters/examples/spacy.ipynb).
+
+```python
+from langchain.llms import SpacyTextSplitter
+```

docs/integrations/spreedly.md

@@ -0,0 +1,15 @@
+# Spreedly
+
+>[Spreedly](https://docs.spreedly.com/) is a service that allows you to securely store credit cards and use them to transact against any number of payment gateways and third party APIs. It does this by simultaneously providing a card tokenization/vault service as well as a gateway and receiver integration service. Payment methods tokenized by Spreedly are stored at `Spreedly`, allowing you to independently store a card and then pass that card to different end points based on your business requirements.
+ 
+## Installation and Setup
+
+See [setup instructions](../modules/indexes/document_loaders/examples/spreedly.ipynb).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/spreedly.ipynb).
+
+```python
+from langchain.document_loaders import SpreedlyLoader
+```

docs/integrations/stripe.md

@@ -0,0 +1,16 @@
+# Stripe
+
+>[Stripe](https://stripe.com/en-ca) is an Irish-American financial services and software as a service (SaaS) company. It offers payment-processing software and application programming interfaces for e-commerce websites and mobile applications.
+
+
+## Installation and Setup
+
+See [setup instructions](../modules/indexes/document_loaders/examples/stripe.ipynb).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/stripe.ipynb).
+
+```python
+from langchain.document_loaders import StripeLoader
+```

docs/integrations/telegram.md

@@ -0,0 +1,17 @@
+# Telegram
+
+>[Telegram Messenger](https://web.telegram.org/a/) is a globally accessible freemium, cross-platform, encrypted, cloud-based and centralized instant messaging service. The application also provides optional end-to-end encrypted chats and video calling, VoIP, file sharing and several other features.
+
+
+## Installation and Setup
+
+See [setup instructions](../modules/indexes/document_loaders/examples/telegram.ipynb).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/telegram.ipynb).
+
+```python
+from langchain.document_loaders import TelegramChatFileLoader
+from langchain.document_loaders import TelegramChatApiLoader
+```

docs/integrations/tensorflow_hub.md

@@ -0,0 +1,22 @@
+# Tensorflow Hub
+
+>[TensorFlow Hub](https://www.tensorflow.org/hub) is a repository of trained machine learning models ready for fine-tuning and deployable anywhere.
+
+>[TensorFlow Hub](https://tfhub.dev/) lets you search and discover hundreds of trained, ready-to-deploy machine learning models in one place.
+ 
+## Installation and Setup
+
+
+```bash
+pip install tensorflow-hub
+pip install tensorflow_text
+```
+
+
+## Text Embedding Models
+
+See a [usage example](../modules/models/text_embedding/examples/tensorflowhub.ipynb)
+
+```python
+from langchain.embeddings import TensorflowHubEmbeddings
+```

docs/integrations/tomarkdown.md

@@ -0,0 +1,16 @@
+# 2Markdown
+
+>[2markdown](https://2markdown.com/) service transforms website content into structured markdown files.
+
+
+## Installation and Setup
+
+We need the `API key`. See [instructions how to get it](https://2markdown.com/login).
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/tomarkdown.ipynb).
+
+```python
+from langchain.document_loaders import ToMarkdownLoader
+```

docs/integrations/trello.md

@@ -0,0 +1,22 @@
+# Trello
+
+>[Trello](https://www.atlassian.com/software/trello) is a web-based project management and collaboration tool that allows individuals and teams to organize and track their tasks and projects. It provides a visual interface known as a "board" where users can create lists and cards to represent their tasks and activities.
+>The TrelloLoader allows us to load cards from a `Trello` board.
+
+
+## Installation and Setup
+
+```bash
+pip install py-trello beautifulsoup4
+```
+
+See [setup instructions](../modules/indexes/document_loaders/examples/trello.ipynb).
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/trello.ipynb).
+
+```python
+from langchain.document_loaders import TrelloLoader
+```

docs/integrations/twitter.md

@@ -0,0 +1,21 @@
+# Twitter
+
+>[Twitter](https://twitter.com/) is an online social media and social networking service.
+
+
+## Installation and Setup
+
+```bash
+pip install tweepy
+```
+
+We must initialize the loader with the `Twitter API` token, and we need to set up the Twitter `username`.
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/twitter.ipynb).
+
+```python
+from langchain.document_loaders import TwitterTweetLoader
+```

docs/integrations/unstructured.md

@@ -4,8 +4,7 @@
 [Unstructured.IO](https://www.unstructured.io/) extracts clean text from raw source documents like
 PDFs and Word documents.
 This page covers how to use the [`unstructured`](https://github.com/Unstructured-IO/unstructured)
-ecosystem within LangChain. 
-
+ecosystem within LangChain.
 
 ## Installation and Setup
 
@@ -20,12 +19,6 @@ its dependencies running locally.
     - `tesseract-ocr`(images and PDFs)
     - `libreoffice` (MS Office docs)
     - `pandoc` (EPUBs)
-- If you are parsing PDFs using the `"hi_res"` strategy, run the following to install the `detectron2` model, which
-  `unstructured` uses for layout detection:
-    - `pip install "detectron2@git+https://github.com/facebookresearch/detectron2.git@e2ce8dc#egg=detectron2"`
-    - If `detectron2` is not installed, `unstructured` will fallback to processing PDFs
-      using the `"fast"` strategy, which uses `pdfminer` directly and doesn't require
-      `detectron2`.
 
 If you want to get up and running with less set up, you can
 simply run `pip install unstructured` and use `UnstructuredAPIFileLoader` or

docs/integrations/vectara.md

@@ -4,7 +4,7 @@
 What is Vectara?
 
 **Vectara Overview:**
-- Vectara is developer-first API platform for building conversational search applications
+- Vectara is developer-first API platform for building GenAI applications
 - To use Vectara - first [sign up](https://console.vectara.com/signup) and create an account. Then create a corpus and an API key for indexing and searching.
 - You can use Vectara's [indexing API](https://docs.vectara.com/docs/indexing-apis/indexing) to add documents into Vectara's index
 - You can use Vectara's [Search API](https://docs.vectara.com/docs/search-apis/search) to query Vectara's index (which also supports Hybrid search implicitly).
@@ -13,6 +13,13 @@ What is Vectara?
 ## Installation and Setup
 To use Vectara with LangChain no special installation steps are required. You just have to provide your customer_id, corpus ID, and an API key created within the Vectara console to enable indexing and searching.
 
+Alternatively these can be provided as environment variables
+- export `VECTARA_CUSTOMER_ID`="your_customer_id"
+- export `VECTARA_CORPUS_ID`="your_corpus_id"
+- export `VECTARA_API_KEY`="your-vectara-api-key"
+
+## Usage
+
 ### VectorStore
 
 There exists a wrapper around the Vectara platform, allowing you to use it as a vectorstore, whether for semantic search or example selection.
@@ -32,8 +39,21 @@ vectara = Vectara(
 ```
 The customer_id, corpus_id and api_key are optional, and if they are not supplied will be read from the environment variables `VECTARA_CUSTOMER_ID`, `VECTARA_CORPUS_ID` and `VECTARA_API_KEY`, respectively.
 
+To query the vectorstore, you can use the `similarity_search` method (or `similarity_search_with_score`), which takes a query string and returns a list of results:
+```python
+results = vectara.similarity_score("what is LangChain?")
+```
 
-For a more detailed walkthrough of the Vectara wrapper, see one of the two example notebooks:
+`similarity_search_with_score` also supports the following additional arguments:
+- `k`: number of results to return (defaults to 5)
+- `lambda_val`: the [lexical matching](https://docs.vectara.com/docs/api-reference/search-apis/lexical-matching) factor for hybrid search (defaults to 0.025)
+- `filter`: a [filter](https://docs.vectara.com/docs/common-use-cases/filtering-by-metadata/filter-overview) to apply to the results (default None)
+- `n_sentence_context`: number of sentences to include before/after the actual matching segment when returning results. This defaults to 0 so as to return the exact text segment that matches, but can be used with other values e.g. 2 or 3 to return adjacent text segments.
+
+The results are returned as a list of relevant documents, and a relevance score of each document.
+
+
+For a more detailed examples of using the Vectara wrapper, see one of these two sample notebooks:
 * [Chat Over Documents with Vectara](./vectara/vectara_chat.html)
 * [Vectara Text Generation](./vectara/vectara_text_generation.html)
 

docs/integrations/vectara/vectara_chat.ipynb

@@ -102,21 +102,11 @@
    "metadata": {
     "tags": []
    },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "<class 'langchain.vectorstores.vectara.Vectara'>\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "openai_api_key = os.environ['OPENAI_API_KEY']\n",
     "llm = OpenAI(openai_api_key=openai_api_key, temperature=0)\n",
-    "retriever = VectaraRetriever(vectorstore, alpha=0.025, k=5, filter=None)\n",
-    "\n",
-    "print(type(vectorstore))\n",
+    "retriever = vectorstore.as_retriever(lambda_val=0.025, k=5, filter=None)\n",
     "d = retriever.get_relevant_documents('What did the president say about Ketanji Brown Jackson')\n",
     "\n",
     "qa = ConversationalRetrievalChain.from_llm(llm, retriever, memory=memory)"
@@ -142,7 +132,7 @@
     {
      "data": {
       "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
+       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
       ]
      },
      "execution_count": 7,
@@ -174,7 +164,7 @@
     {
      "data": {
       "text/plain": [
-       "' Justice Stephen Breyer.'"
+       "' Justice Stephen Breyer'"
       ]
      },
      "execution_count": 9,
@@ -241,7 +231,7 @@
     {
      "data": {
       "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
+       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
       ]
      },
      "execution_count": 12,
@@ -286,7 +276,7 @@
     {
      "data": {
       "text/plain": [
-       "' Justice Stephen Breyer.'"
+       "' Justice Stephen Breyer'"
       ]
      },
      "execution_count": 14,
@@ -344,7 +334,7 @@
     {
      "data": {
       "text/plain": [
-       "Document(page_content='Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice. A former federal public defender.', metadata={'source': '../../modules/state_of_the_union.txt'})"
+       "Document(page_content='Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \\n\\nTonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \\n\\nOne of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \\n\\nAnd I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.', metadata={'source': '../../../state_of_the_union.txt'})"
       ]
      },
      "execution_count": 17,
@@ -392,6 +382,24 @@
     "result = qa({\"question\": query, \"chat_history\": chat_history, \"vectordbkwargs\": vectordbkwargs})"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 35,
+   "id": "24ebdaec",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(result['answer'])"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "99b96dae",
@@ -459,7 +467,7 @@
     {
      "data": {
       "text/plain": [
-       "' The president did not mention Ketanji Brown Jackson.'"
+       "\" The president said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson, who he described as one of the nation's top legal minds, to continue Justice Breyer's legacy of excellence.\""
       ]
      },
      "execution_count": 23,
@@ -538,7 +546,7 @@
     {
      "data": {
       "text/plain": [
-       "' The president did not mention Ketanji Brown Jackson.\\nSOURCES: ../../modules/state_of_the_union.txt'"
+       "\" The president said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson, who he described as one of the nation's top legal minds, and that she will continue Justice Breyer's legacy of excellence.\\nSOURCES: ../../../state_of_the_union.txt\""
       ]
      },
      "execution_count": 27,
@@ -598,7 +606,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      " The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender."
+      " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence."
      ]
     }
    ],
@@ -620,7 +628,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      " Justice Stephen Breyer."
+      " Justice Stephen Breyer"
      ]
     }
    ],
@@ -681,7 +689,7 @@
     {
      "data": {
       "text/plain": [
-       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, and a former federal public defender.\""
+       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
       ]
      },
      "execution_count": 33,

docs/integrations/vectara/vectara_text_generation.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "# Vectara Text Generation\n",
     "\n",
-    "This notebook is based on [chat_vector_db](https://github.com/hwchase17/langchain/blob/master/docs/modules/chains/index_examples/question_answering.ipynb) and adapted to Vectara."
+    "This notebook is based on [text generation](https://github.com/hwchase17/langchain/blob/master/docs/modules/chains/index_examples/vector_db_text_generation.ipynb) notebook and adapted to Vectara."
    ]
   },
   {
@@ -24,6 +24,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "import os\n",
     "from langchain.llms import OpenAI\n",
     "from langchain.docstore.document import Document\n",
     "import requests\n",
@@ -159,7 +160,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "[{'text': '\\n\\nEnvironment variables are an essential part of any development workflow. They provide a way to store and access information that is specific to the environment in which the code is running. This can be especially useful when working with different versions of a language or framework, or when running code on different machines.\\n\\nThe Deno CLI tasks extension provides a way to easily manage environment variables when running Deno commands. This extension provides a task definition for allowing you to create tasks that execute the `deno` CLI from within the editor. The template for the Deno CLI tasks has the following interface, which can be configured in a `tasks.json` within your workspace:\\n\\nThe task definition includes the `type` field, which should be set to `deno`, and the `command` field, which is the `deno` command to run (e.g. `run`, `test`, `cache`, etc.). Additionally, you can specify additional arguments to pass on the command line, the current working directory to execute the command, and any environment variables.\\n\\nUsing environment variables with the Deno CLI tasks extension is a great way to ensure that your code is running in the correct environment. For example, if you are running a test suite,'}, {'text': '\\n\\nEnvironment variables are an important part of any programming language, and they can be used to store and access data in a variety of ways. In this blog post, we\\'ll be taking a look at environment variables specifically for the shell.\\n\\nShell variables are similar to environment variables, but they won\\'t be exported to spawned commands. They are defined with the following syntax:\\n\\n```sh\\nVAR_NAME=value\\n```\\n\\nShell variables can be used to store and access data in a variety of ways. For example, you can use them to store values that you want to re-use, but don\\'t want to be available in any spawned processes.\\n\\nFor example, if you wanted to store a value and then use it in a command, you could do something like this:\\n\\n```sh\\nVAR=hello && echo $VAR && deno eval \"console.log(\\'Deno: \\' + Deno.env.get(\\'VAR\\'))\"\\n```\\n\\nThis would output the following:\\n\\n```\\nhello\\nDeno: undefined\\n```\\n\\nAs you can see, the value stored in the shell variable is not available in the spawned process.\\n\\n'}, {'text': '\\n\\nWhen it comes to developing applications, environment variables are an essential part of the process. Environment variables are used to store information that can be used by applications and scripts to customize their behavior. This is especially important when it comes to developing applications with Deno, as there are several environment variables that can impact the behavior of Deno.\\n\\nThe most important environment variable for Deno is `DENO_AUTH_TOKENS`. This environment variable is used to store authentication tokens that are used to access remote resources. This is especially important when it comes to accessing remote APIs or databases. Without the proper authentication tokens, Deno will not be able to access the remote resources.\\n\\nAnother important environment variable for Deno is `DENO_DIR`. This environment variable is used to store the directory where Deno will store its files. This includes the Deno executable, the Deno cache, and the Deno configuration files. By setting this environment variable, you can ensure that Deno will always be able to find the files it needs.\\n\\nFinally, there is the `DENO_PLUGINS` environment variable. This environment variable is used to store the list of plugins that Deno will use. This is important for customizing the'}, {'text': '\\n\\nEnvironment variables are a great way to store and access sensitive information in your Deno applications. Deno offers built-in support for environment variables with `Deno.env`, and you can also use a `.env` file to store and access environment variables. In this blog post, we\\'ll explore both of these options and how to use them in your Deno applications.\\n\\n## Built-in `Deno.env`\\n\\nThe Deno runtime offers built-in support for environment variables with [`Deno.env`](https://deno.land/api@v1.25.3?s=Deno.env). `Deno.env` has getter and setter methods. Here is example usage:\\n\\n```ts\\nDeno.env.set(\"FIREBASE_API_KEY\", \"examplekey123\");\\nDeno.env.set(\"FIREBASE_AUTH_DOMAIN\", \"firebasedomain.com\");\\n\\nconsole.log(Deno.env.get(\"FIREBASE_API_KEY\")); // examplekey123\\nconsole.log(Deno.env.get(\"FIREBASE_AUTH_'}]\n"
+      "[{'text': '\\n\\nEnvironment variables are a powerful tool for managing configuration settings in your applications. They allow you to store and access values from anywhere in your code, making it easier to keep your codebase organized and maintainable.\\n\\nHowever, there are times when you may want to use environment variables specifically for a single command. This is where shell variables come in. Shell variables are similar to environment variables, but they won\\'t be exported to spawned commands. They are defined with the following syntax:\\n\\n```sh\\nVAR_NAME=value\\n```\\n\\nFor example, if you wanted to use a shell variable instead of an environment variable in a command, you could do something like this:\\n\\n```sh\\nVAR=hello && echo $VAR && deno eval \"console.log(\\'Deno: \\' + Deno.env.get(\\'VAR\\'))\"\\n```\\n\\nThis would output the following:\\n\\n```\\nhello\\nDeno: undefined\\n```\\n\\nShell variables can be useful when you want to re-use a value, but don\\'t want it available in any spawned processes.\\n\\nAnother way to use environment variables is through pipelines. Pipelines provide a way to pipe the'}, {'text': '\\n\\nEnvironment variables are a great way to store and access sensitive information in your applications. They are also useful for configuring applications and managing different environments. In Deno, there are two ways to use environment variables: the built-in `Deno.env` and the `.env` file.\\n\\nThe `Deno.env` is a built-in feature of the Deno runtime that allows you to set and get environment variables. It has getter and setter methods that you can use to access and set environment variables. For example, you can set the `FIREBASE_API_KEY` and `FIREBASE_AUTH_DOMAIN` environment variables like this:\\n\\n```ts\\nDeno.env.set(\"FIREBASE_API_KEY\", \"examplekey123\");\\nDeno.env.set(\"FIREBASE_AUTH_DOMAIN\", \"firebasedomain.com\");\\n\\nconsole.log(Deno.env.get(\"FIREBASE_API_KEY\")); // examplekey123\\nconsole.log(Deno.env.get(\"FIREBASE_AUTH_DOMAIN\")); // firebasedomain'}, {'text': \"\\n\\nEnvironment variables are a powerful tool for managing configuration and settings in your applications. They allow you to store and access values that can be used in your code, and they can be set and changed without having to modify your code.\\n\\nIn Deno, environment variables are defined using the `export` command. For example, to set a variable called `VAR_NAME` to the value `value`, you would use the following command:\\n\\n```sh\\nexport VAR_NAME=value\\n```\\n\\nYou can then access the value of the environment variable in your code using the `Deno.env.get()` method. For example, if you wanted to log the value of the `VAR_NAME` variable, you could use the following code:\\n\\n```js\\nconsole.log(Deno.env.get('VAR_NAME'));\\n```\\n\\nYou can also set environment variables for a single command. To do this, you can list the environment variables before the command, like so:\\n\\n```\\nVAR=hello VAR2=bye deno run main.ts\\n```\\n\\nThis will set the environment variables `VAR` and `V\"}, {'text': \"\\n\\nEnvironment variables are a powerful tool for managing settings and configuration in your applications. They can be used to store information such as user preferences, application settings, and even passwords. In this blog post, we'll discuss how to make Deno scripts executable with a hashbang (shebang).\\n\\nA hashbang is a line of code that is placed at the beginning of a script. It tells the system which interpreter to use when running the script. In the case of Deno, the hashbang should be `#!/usr/bin/env -S deno run --allow-env`. This tells the system to use the Deno interpreter and to allow the script to access environment variables.\\n\\nOnce the hashbang is in place, you may need to give the script execution permissions. On Linux, this can be done with the command `sudo chmod +x hashbang.ts`. After that, you can execute the script by calling it like any other command: `./hashbang.ts`.\\n\\nIn the example program, we give the context permission to access the environment variables and print the Deno installation path. This is done by using the `Deno.env.get()` function, which returns the value of the specified environment\"}]\n"
      ]
     }
    ],

docs/integrations/vespa.md

@@ -0,0 +1,21 @@
+# Vespa
+
+>[Vespa](https://vespa.ai/) is a fully featured search engine and vector database. 
+> It supports vector search (ANN), lexical search, and search in structured data, all in the same query.
+ 
+## Installation and Setup
+
+
+```bash
+pip install pyvespa
+```
+
+
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/vespa.ipynb).
+
+```python
+from langchain.retrievers import VespaRetriever
+```

docs/integrations/wandb_tracking.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -8,9 +9,15 @@
     "\n",
     "This notebook goes over how to track your LangChain experiments into one centralized Weights and Biases dashboard. To learn more about prompt engineering and the callback please refer to this Report which explains both alongside the resultant dashboards you can expect to see.\n",
     "\n",
-    "Run in Colab: https://colab.research.google.com/drive/1DXH4beT4HFaRKy_Vm4PoxhXVDRf7Ym8L?usp=sharing\n",
     "\n",
-    "View Report: https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/Prompt-Engineering-LLMs-with-LangChain-and-W-B--VmlldzozNjk1NTUw#👋-how-to-build-a-callback-in-langchain-for-better-prompt-engineering"
+    "<a href=\"https://colab.research.google.com/drive/1DXH4beT4HFaRKy_Vm4PoxhXVDRf7Ym8L?usp=sharing\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
+    "\n",
+    "\n",
+    "[View Report](https://wandb.ai/a-sh0ts/langchain_callback_demo/reports/Prompt-Engineering-LLMs-with-LangChain-and-W-B--VmlldzozNjk1NTUw#👋-how-to-build-a-callback-in-langchain-for-better-prompt-engineering\n",
+    ") \n",
+    "\n",
+    "\n",
+    "**Note**: _the `WandbCallbackHandler` is being deprecated in favour of the `WandbTracer`_ . In future please use the `WandbTracer` as it is more flexible and allows for more granular logging. To know more about the `WandbTracer` refer to the [agent_with_wandb_tracing.ipynb](https://python.langchain.com/en/latest/integrations/agent_with_wandb_tracing.html) notebook or use the following [colab notebook](http://wandb.me/prompts-quickstart). To know more about Weights & Biases Prompts refer to the following [prompts documentation](https://docs.wandb.ai/guides/prompts)."
    ]
   },
   {
@@ -54,6 +61,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -75,6 +83,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {
     "id": "cxBFfZR8d9FC"
@@ -90,6 +99,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
@@ -200,6 +210,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {
     "id": "Q-65jwrDeK6w"
@@ -217,6 +228,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [

docs/integrations/weather.md

@@ -0,0 +1,21 @@
+# Weather
+
+>[OpenWeatherMap](https://openweathermap.org/) is an open source weather service provider.
+
+
+
+## Installation and Setup
+
+```bash
+pip install pyowm
+```
+
+We must set up the `OpenWeatherMap API token`.
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/weather.ipynb).
+
+```python
+from langchain.document_loaders import WeatherDataLoader
+```

docs/integrations/whatsapp.md

@@ -0,0 +1,18 @@
+# WhatsApp
+
+>[WhatsApp](https://www.whatsapp.com/) (also called `WhatsApp Messenger`) is a freeware, cross-platform, centralized instant messaging (IM) and voice-over-IP (VoIP) service. It allows users to send text and voice messages, make voice and video calls, and share images, documents, user locations, and other content.
+
+
+## Installation and Setup
+
+There isn't any special setup for it.
+
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/whatsapp_chat.ipynb).
+
+```python
+from langchain.document_loaders import WhatsAppChatLoader
+```

docs/integrations/wikipedia.md

@@ -0,0 +1,28 @@
+# Wikipedia
+
+>[Wikipedia](https://wikipedia.org/) is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system called MediaWiki. `Wikipedia` is the largest and most-read reference work in history.
+
+
+## Installation and Setup
+
+```bash
+pip install wikipedia
+```
+
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/wikipedia.ipynb).
+
+```python
+from langchain.document_loaders import WikipediaLoader
+```
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/wikipedia.ipynb).
+
+```python
+from langchain.retrievers import WikipediaRetriever
+```

docs/integrations/youtube.md

@@ -0,0 +1,22 @@
+# YouTube
+
+>[YouTube](https://www.youtube.com/) is an online video sharing and social media platform created by Google.
+> We download the `YouTube` transcripts and video information.
+
+## Installation and Setup
+
+```bash
+pip install youtube-transcript-api
+pip install pytube
+```
+See a [usage example](../modules/indexes/document_loaders/examples/youtube_transcript.ipynb).
+
+
+## Document Loader
+
+See a [usage example](../modules/indexes/document_loaders/examples/youtube_transcript.ipynb).
+
+```python
+from langchain.document_loaders import YoutubeLoader
+from langchain.document_loaders import GoogleApiYoutubeLoader
+```

docs/integrations/zep.md

@@ -0,0 +1,28 @@
+# Zep
+
+>[Zep](https://docs.getzep.com/) - A long-term memory store for LLM applications.
+
+>`Zep` stores, summarizes, embeds, indexes, and enriches conversational AI chat histories, and exposes them via simple, low-latency APIs.
+>- Long-term memory persistence, with access to historical messages irrespective of your summarization strategy.
+>- Auto-summarization of memory messages based on a configurable message window. A series of summaries are stored, providing flexibility for future summarization strategies.
+>- Vector search over memories, with messages automatically embedded on creation.
+>- Auto-token counting of memories and summaries, allowing finer-grained control over prompt assembly.
+>- Python and JavaScript SDKs.
+
+
+`Zep` [project](https://github.com/getzep/zep) 
+
+## Installation and Setup
+
+```bash
+pip install zep_python
+```
+
+
+## Retriever
+
+See a [usage example](../modules/indexes/retrievers/examples/zep_memorystore.ipynb).
+
+```python
+from langchain.retrievers import ZepRetriever
+```

docs/integrations/zilliz.md

@@ -1,19 +1,20 @@
 # Zilliz
 
-This page covers how to use the Zilliz Cloud ecosystem within LangChain.
-Zilliz uses the Milvus integration. 
-It is broken into two parts: installation and setup, and then references to specific Milvus wrappers.
+>[Zilliz Cloud](https://zilliz.com/doc/quick_start) is a fully managed service on cloud for `LF AI Milvus®`,
+
 
 ## Installation and Setup
-- Install the Python SDK with `pip install pymilvus`
-## Wrappers
 
-### VectorStore
+Install the Python SDK:
+```bash
+pip install pymilvus
+```
 
-There exists a wrapper around Zilliz indexes, allowing you to use it as a vectorstore,
+## Vectorstore
+
+A wrapper around Zilliz indexes allows you to use it as a vectorstore,
 whether for semantic search or example selection.
 
-To import this vectorstore:
 ```python
 from langchain.vectorstores import Milvus
 ```

docs/modules/agents.rst

@@ -5,108 +5,101 @@ Agents
    `Conceptual Guide <https://docs.langchain.com/docs/components/agents>`_
 
 
-Some applications will require not just a predetermined chain of calls to LLMs/other tools,
+Some applications require not just a predetermined chain of calls to LLMs/other tools,
 but potentially an unknown chain that depends on the user's input.
-In these types of chains, there is a “agent” which has access to a suite of tools.
+In these types of chains, there is an **agent** which has access to a suite of **tools**.
 Depending on the user input, the agent can then decide which, if any, of these tools to call.
 
 At the moment, there are two main types of agents:
 
-1. "Action Agents": these agents decide an action to take and take that action one step at a time
-2. "Plan-and-Execute Agents": these agents first decide a plan of actions to take, and then execute those actions one at a time.
+1. **Action Agents**: these agents decide the actions to take and execute that actions one action at a time.
+2. **Plan-and-Execute Agents**: these agents first decide a plan of actions to take, and then execute those actions one at a time.
 
 When should you use each one? Action Agents are more conventional, and good for small tasks.
-For more complex or long running tasks, the initial planning step helps to maintain long term objectives and focus. However, that comes at the expense of generally more calls and higher latency.
-These two agents are also not mutually exclusive - in fact, it is often best to have an Action Agent be in charge of the execution for the Plan and Execute agent.
+For more complex or long running tasks, the initial planning step helps to maintain long term objectives and focus.
+However, that comes at the expense of generally more calls and higher latency.
+These two agents are also not mutually exclusive - in fact, it is often best to have an Action Agent be in charge
+of the execution for the Plan and Execute agent.
 
 Action Agents
 -------------
 
-High level pseudocode of agents looks something like:
+High level pseudocode of the Action Agents:
 
-- Some user input is received
-- The `agent` decides which `tool` - if any - to use, and what the input to that tool should be
-- That `tool` is then called with that `tool input`, and an `observation` is recorded (this is just the output of calling that tool with that tool input)
-- That history of `tool`, `tool input`, and `observation` is passed back into the `agent`, and it decides what step to take next
-- This is repeated until the `agent` decides it no longer needs to use a `tool`, and then it responds directly to the user.
+- The **user input** is received
+- The **agent** decides which **tool** - if any - to use, and what the **tool input** should be
+- That **tool** is then called with the **tool input**, and an **observation** is recorded (the output of this calling)
+- That history of **tool**, **tool input**, and **observation** is passed back into the **agent**, and it decides the next step
+- This is repeated until the **agent** decides it no longer needs to use a **tool**, and then it responds directly to the user.
 
-The different abstractions involved in agents are as follows:
 
-- Agent: this is where the logic of the application lives. Agents expose an interface that takes in user input along with a list of previous steps the agent has taken, and returns either an `AgentAction` or `AgentFinish`
-    - `AgentAction` corresponds to the tool to use and the input to that tool
-    - `AgentFinish` means the agent is done, and has information around what to return to the user
-- Tools: these are the actions an agent can take. What tools you give an agent highly depend on what you want the agent to do
-- Toolkits: these are groups of tools designed for a specific use case. For example, in order for an agent to interact with a SQL database in the best way it may need access to one tool to execute queries and another tool to inspect tables.
-- Agent Executor: this wraps an agent and a list of tools. This is responsible for the loop of running the agent iteratively until the stopping criteria is met.
+The different abstractions involved in agents are:
 
-The most important abstraction of the four above to understand is that of the agent.
-Although an agent can be defined in whatever way one chooses, the typical way to construct an agent is with:
+- **Agent**: this is where the logic of the application lives. Agents expose an interface that takes in user input
+  along with a list of previous steps the agent has taken, and returns either an **AgentAction** or **AgentFinish**
 
-- PromptTemplate: this is responsible for taking the user input and previous steps and constructing a prompt to send to the language model
-- Language Model: this takes the prompt constructed by the PromptTemplate and returns some output
-- Output Parser: this takes the output of the Language Model and parses it into an `AgentAction` or `AgentFinish` object.
+  - **AgentAction** corresponds to the tool to use and the input to that tool
+  - **AgentFinish** means the agent is done, and has information around what to return to the user
+- **Tools**: these are the actions an agent can take. What tools you give an agent highly depend on what you want the agent to do
+- **Toolkits**: these are groups of tools designed for a specific use case. For example, in order for an agent to
+  interact with a SQL database in the best way it may need access to one tool to execute queries and another tool to inspect tables.
+- **Agent Executor**: this wraps an agent and a list of tools. This is responsible for the loop of running the agent
+  iteratively until the stopping criteria is met.
+
+
+|
+- `Getting Started <./agents/getting_started.html>`_: An overview of agents. It covers how to use all things related to agents in an end-to-end manner.
+
+
+|
+**Agent Construction:**
+
+Although an agent can be constructed in many way, the typical way to construct an agent is with:
+
+- **PromptTemplate**: this is responsible for taking the user input and previous steps and constructing a prompt
+  to send to the language model
+- **Language Model**: this takes the prompt constructed by the PromptTemplate and returns some output
+- **Output Parser**: this takes the output of the Language Model and parses it into an **AgentAction** or **AgentFinish** object.
+
+
+|
+**Additional Documentation:**
+
+
+- `Tools <./agents/tools.html>`_: Different types of **tools** LangChain supports natively. We also cover how to add your own tools.
+
+- `Agents <./agents/agents.html>`_: Different types of **agents** LangChain supports natively. We also cover how to
+  modify and create your own agents.
+
+- `Toolkits <./agents/toolkits.html>`_: Various **toolkits** that LangChain supports out of the box, and how to
+  create an agent from them.
+
+- `Agent Executor <./agents/agent_executors.html>`_: The **Agent Executor** class, which is responsible for calling
+  the agent and tools in a loop. We go over different ways to customize this, and options you can use for more control.
+
+
+Plan-and-Execute Agents
+-----------------------
+High level pseudocode of the **Plan-and-Execute Agents**:
+
+- The **user input** is received
+- The **planner** lists out the steps to take
+- The **executor** goes through the list of steps, executing them
+
+The most typical implementation is to have the planner be a language model, and the executor be an action agent.
+
+|
+- `Plan-and-Execute Agents <./agents/plan_and_execute.html>`_
 
-In this section of documentation, we first start with a Getting Started notebook to cover how to use all things related to agents in an end-to-end manner.
 
 .. toctree::
    :maxdepth: 1
    :hidden:
 
    ./agents/getting_started.ipynb
-
-
-We then split the documentation into the following sections:
-
-**Tools**
-
-In this section we cover the different types of tools LangChain supports natively.
-We then cover how to add your own tools.
-
-
-**Agents**
-
-In this section we cover the different types of agents LangChain supports natively.
-We then cover how to modify and create your own agents.
-
-
-**Toolkits**
-
-In this section we go over the various toolkits that LangChain supports out of the box,
-and how to create an agent from them.
-
-
-**Agent Executor**
-
-In this section we go over the Agent Executor class, which is responsible for calling
-the agent and tools in a loop. We go over different ways to customize this, and options you
-can use for more control.
-
-**Go Deeper**
-
-.. toctree::
-   :maxdepth: 1
-
    ./agents/tools.rst
    ./agents/agents.rst
    ./agents/toolkits.rst
    ./agents/agent_executors.rst
-
-Plan-and-Execute Agents
------------------------
-
-High level pseudocode of agents looks something like:
-
-- Some user input is received
-- The planner lists out the steps to take
-- The executor goes through the list of steps, executing them
-
-The most typical implementation is to have the planner be a language model,
-and the executor be an action agent.
-
-**Go Deeper**
-
-.. toctree::
-   :maxdepth: 1
-
    ./agents/plan_and_execute.ipynb
 

docs/modules/agents/agent_executors/examples/agent_vectorstore.ipynb

@@ -14,6 +14,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "9b22020a",
    "metadata": {},
@@ -139,6 +140,7 @@
    "source": []
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "c0a6c031",
    "metadata": {},
@@ -229,7 +231,7 @@
     }
    ],
    "source": [
-    "agent.run(\"What did biden say about ketanji brown jackson is the state of the union address?\")"
+    "agent.run(\"What did biden say about ketanji brown jackson in the state of the union address?\")"
    ]
   },
   {
@@ -271,6 +273,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "787a9b5e",
    "metadata": {},
@@ -279,6 +282,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "9161ba91",
    "metadata": {},
@@ -396,6 +400,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "49a0cbbe",
    "metadata": {},

docs/modules/agents/agent_executors/examples/sharedmemory_for_tools.ipynb

@@ -9,8 +9,8 @@
     "\n",
     "This notebook goes over adding memory to **both** of an Agent and its tools. Before going through this notebook, please walk through the following notebooks, as this will build on top of both of them:\n",
     "\n",
-    "- [Adding memory to an LLM Chain](../../memory/examples/adding_memory.ipynb)\n",
-    "- [Custom Agents](custom_agent.ipynb)\n",
+    "- [Adding memory to an LLM Chain](../../../memory/examples/adding_memory.ipynb)\n",
+    "- [Custom Agents](../../agents/custom_agent.ipynb)\n",
     "\n",
     "We are going to create a custom Agent. The agent has access to a conversation memory, search tool, and a summarization tool. And, the summarization tool also needs access to the conversation memory."
    ]

docs/modules/agents/how_to_guides.rst

@@ -36,7 +36,7 @@ The first category of how-to guides here cover specific parts of working with ag
    :glob:
    :hidden:
 
-   ./examples/*
+   ./agents/examples/*
 
 
 Agent Toolkits
@@ -46,26 +46,26 @@ The next set of examples covers agents with toolkits.
 As opposed to the examples above, these examples are not intended to show off an agent `type`,
 but rather to show off an agent applied to particular use case.
 
-`SQLDatabase Agent <./agent_toolkits/sql_database.html>`_: This notebook covers how to interact with an arbitrary SQL database using an agent.
+`SQLDatabase Agent <./toolkits/sql_database.html>`_: This notebook covers how to interact with an arbitrary SQL database using an agent.
 
-`JSON Agent <./agent_toolkits/json.html>`_: This notebook covers how to interact with a JSON dictionary using an agent.
+`JSON Agent <./toolkits/json.html>`_: This notebook covers how to interact with a JSON dictionary using an agent.
 
-`OpenAPI Agent <./agent_toolkits/openapi.html>`_: This notebook covers how to interact with an arbitrary OpenAPI endpoint using an agent.
+`OpenAPI Agent <./toolkits/openapi.html>`_: This notebook covers how to interact with an arbitrary OpenAPI endpoint using an agent.
 
-`VectorStore Agent <./agent_toolkits/vectorstore.html>`_: This notebook covers how to interact with VectorStores using an agent.
+`VectorStore Agent <./toolkits/vectorstore.html>`_: This notebook covers how to interact with VectorStores using an agent.
 
-`Python Agent <./agent_toolkits/python.html>`_: This notebook covers how to produce and execute python code using an agent.
+`Python Agent <./toolkits/python.html>`_: This notebook covers how to produce and execute python code using an agent.
 
-`Pandas DataFrame Agent <./agent_toolkits/pandas.html>`_: This notebook covers how to do question answering over a pandas dataframe using an agent. Under the hood this calls the Python agent..
+`Pandas DataFrame Agent <./toolkits/pandas.html>`_: This notebook covers how to do question answering over a pandas dataframe using an agent. Under the hood this calls the Python agent..
 
-`CSV Agent <./agent_toolkits/csv.html>`_: This notebook covers how to do question answering over a csv file. Under the hood this calls the Pandas DataFrame agent.
+`CSV Agent <./toolkits/csv.html>`_: This notebook covers how to do question answering over a csv file. Under the hood this calls the Pandas DataFrame agent.
 
 .. toctree::
    :maxdepth: 1
    :glob:
    :hidden:
 
-   ./agent_toolkits/*
+   ./toolkits/*
 
 
 Agent Types

docs/modules/agents/streaming_stdout_final_only.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "23234b50-e6c6-4c87-9f97-259c15f36894",
    "metadata": {
@@ -11,6 +12,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "29dd6333-307c-43df-b848-65001c01733b",
    "metadata": {},
@@ -36,6 +38,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "19a813f7",
    "metadata": {},
@@ -84,6 +87,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "53a743b8",
    "metadata": {},
@@ -92,11 +96,12 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "23602c62",
    "metadata": {},
    "source": [
-    "By default, we assume that the token sequence ``\"\\nFinal\", \" Answer\", \":\"`` indicates that the agent has reached an answers. We can, however, also pass a custom sequence to use as answer prefix."
+    "By default, we assume that the token sequence ``\"Final\", \"Answer\", \":\"`` indicates that the agent has reached an answers. We can, however, also pass a custom sequence to use as answer prefix."
    ]
   },
   {
@@ -108,26 +113,75 @@
    "source": [
     "llm = OpenAI(\n",
     "    streaming=True,\n",
-    "    callbacks=[FinalStreamingStdOutCallbackHandler(answer_prefix_tokens=[\"\\nThe\", \" answer\", \":\"])],\n",
+    "    callbacks=[FinalStreamingStdOutCallbackHandler(answer_prefix_tokens=[\"The\", \"answer\", \":\"])],\n",
     "    temperature=0\n",
     ")"
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "b1a96cc0",
    "metadata": {},
    "source": [
-    "Be aware you likely need to include whitespaces and new line characters in your token. "
+    "For convenience, the callback automatically strips whitespaces and new line characters when comparing to `answer_prefix_tokens`. I.e., if `answer_prefix_tokens = [\"The\", \" answer\", \":\"]` then both `[\"\\nThe\", \" answer\", \":\"]` and `[\"The\", \" answer\", \":\"]` would be recognized a the answer prefix."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "9278b522",
+   "metadata": {},
+   "source": [
+    "If you don't know the tokenized version of your answer prefix, you can determine it with the following code:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9278b522",
+   "id": "2f8f0640",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "from langchain.callbacks.base import BaseCallbackHandler\n",
+    "\n",
+    "class MyCallbackHandler(BaseCallbackHandler):\n",
+    "    def on_llm_new_token(self, token, **kwargs) -> None:\n",
+    "        # print every token on a new line\n",
+    "        print(f\"#{token}#\")\n",
+    "\n",
+    "llm = OpenAI(streaming=True, callbacks=[MyCallbackHandler()])\n",
+    "tools = load_tools([\"wikipedia\", \"llm-math\"], llm=llm)\n",
+    "agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False)\n",
+    "agent.run(\"It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany.\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "61190e58",
+   "metadata": {},
+   "source": [
+    "### Also streaming the answer prefixes"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "1255776f",
+   "metadata": {},
+   "source": [
+    "When the parameter `stream_prefix = True` is set, the answer prefix itself will also be streamed. This can be useful when the answer prefix itself is part of the answer. For example, when your answer is a JSON like\n",
+    "\n",
+    "`\n",
+    "{\n",
+    "    \"action\": \"Final answer\",\n",
+    "    \"action_input\": \"Konrad Adenauer became Chancellor 74 years ago.\"\n",
+    "}\n",
+    "`\n",
+    "\n",
+    "and you don't only want the action_input to be streamed, but the entire JSON."
+   ]
   }
  ],
  "metadata": {

docs/modules/agents/toolkits/examples/vectorstore.ipynb

@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "18ada398-dce6-4049-9b56-fc0ede63da9c",
    "metadata": {},
@@ -11,6 +12,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "eecb683b-3a46-4b9d-81a3-7caefbfec1a1",
    "metadata": {},
@@ -88,6 +90,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "f4814175-964d-42f1-aa9d-22801ce1e912",
    "metadata": {},
@@ -123,6 +126,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "8a38ad10",
    "metadata": {},
@@ -165,7 +169,7 @@
     }
    ],
    "source": [
-    "agent_executor.run(\"What did biden say about ketanji brown jackson is the state of the union address?\")"
+    "agent_executor.run(\"What did biden say about ketanji brown jackson in the state of the union address?\")"
    ]
   },
   {
@@ -203,10 +207,11 @@
     }
    ],
    "source": [
-    "agent_executor.run(\"What did biden say about ketanji brown jackson is the state of the union address? List the source.\")"
+    "agent_executor.run(\"What did biden say about ketanji brown jackson in the state of the union address? List the source.\")"
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "7ca07707",
    "metadata": {},
@@ -255,6 +260,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "71680984-edaf-4a63-90f5-94edbd263550",
    "metadata": {},
@@ -299,7 +305,7 @@
     }
    ],
    "source": [
-    "agent_executor.run(\"What did biden say about ketanji brown jackson is the state of the union address?\")"
+    "agent_executor.run(\"What did biden say about ketanji brown jackson in the state of the union address?\")"
    ]
   },
   {

docs/modules/agents/tools/examples/brave_search.ipynb

@@ -0,0 +1,94 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "eda326e4",
+   "metadata": {},
+   "source": [
+    "# Brave Search\n",
+    "\n",
+    "This notebook goes over how to use the Brave Search tool."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "a4c896e5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.tools import BraveSearch"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "6784d37c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "api_key = \"...\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "5b14008a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool = BraveSearch.from_api_key(api_key=api_key, search_kwargs={\"count\": 3})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "f11937b2",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'[{\"title\": \"Barack Obama - Wikipedia\", \"link\": \"https://en.wikipedia.org/wiki/Barack_Obama\", \"snippet\": \"Outside of politics, <strong>Obama</strong> has published three bestselling books: Dreams from My Father (1995), The Audacity of Hope (2006) and A Promised Land (2020). Rankings by scholars and historians, in which he has been featured since 2010, place him in the <strong>middle</strong> to upper tier of American presidents.\"}, {\"title\": \"Obama\\'s Middle Name -- My Last Name -- is \\'Hussein.\\' So?\", \"link\": \"https://www.cair.com/cair_in_the_news/obamas-middle-name-my-last-name-is-hussein-so/\", \"snippet\": \"Many Americans understand that common names don\\\\u2019t only come in the form of a \\\\u201cSmith\\\\u201d or a \\\\u201cJohnson.\\\\u201d Perhaps, they have a neighbor, mechanic or teacher named Hussein. Or maybe they\\\\u2019ve seen fashion designer Hussein Chalayan in the pages of Vogue or recall <strong>King Hussein</strong>, our ally in the Middle East.\"}, {\"title\": \"What\\'s up with Obama\\'s middle name? - Quora\", \"link\": \"https://www.quora.com/Whats-up-with-Obamas-middle-name\", \"snippet\": \"Answer (1 of 15): A better question would be, \\\\u201cWhat\\\\u2019s up with Obama\\\\u2019s first name?\\\\u201d President <strong>Barack Hussein Obama</strong>\\\\u2019s father\\\\u2019s name was <strong>Barack Hussein Obama</strong>. He was named after his father. Hussein, Obama\\\\u2019s middle name, is a very common Arabic name, meaning &quot;good,&quot; &quot;handsome,&quot; or &quot;beautiful.&quot;\"}]'"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "tool.run(\"obama middle name\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "da9c63d5",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/agents/tools/examples/google_search.ipynb

@@ -184,7 +184,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.2"
+   "version": "3.9.1"
   },
   "vscode": {
    "interpreter": {

docs/modules/agents/tools/examples/pubmed.ipynb

@@ -0,0 +1,86 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "64f20f38",
+   "metadata": {},
+   "source": [
+    "# PubMed Tool\n",
+    "\n",
+    "This notebook goes over how to use PubMed as a tool\n",
+    "\n",
+    "PubMed® comprises more than 35 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "c80b9273",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.tools import PubmedQueryRun"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f203c965",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool = PubmedQueryRun()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "baee7a2a",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Published: <Year>2023</Year><Month>May</Month><Day>31</Day>\\nTitle: Dermatology in the wake of an AI revolution: who gets a say?\\nSummary: \\n\\nPublished: <Year>2023</Year><Month>May</Month><Day>30</Day>\\nTitle: What is ChatGPT and what do we do with it? Implications of the age of AI for nursing and midwifery practice and education: An editorial.\\nSummary: \\n\\nPublished: <Year>2023</Year><Month>Jun</Month><Day>02</Day>\\nTitle: The Impact of ChatGPT on the Nursing Profession: Revolutionizing Patient Care and Education.\\nSummary: The nursing field has undergone notable changes over time and is projected to undergo further modifications in the future, owing to the advent of sophisticated technologies and growing healthcare needs. The advent of ChatGPT, an AI-powered language model, is expected to exert a significant influence on the nursing profession, specifically in the domains of patient care and instruction. The present article delves into the ramifications of ChatGPT within the nursing domain and accentuates its capacity and constraints to transform the discipline.'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "tool.run(\"chatgpt\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "965903ba",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/agents/tools/getting_started.md

@@ -160,3 +160,9 @@ Below is a list of all supported tools and relevant information:
 - Notes: A connection to the OpenWeatherMap API (https://api.openweathermap.org), specifically the `/data/2.5/weather` endpoint.
 - Requires LLM: No
 - Extra Parameters: `openweathermap_api_key` (your API key to access this endpoint)
+
+**sleep**
+
+- Tool Name: Sleep
+- Tool Description: Make agent sleep for some time.
+- Requires LLM: No

docs/modules/agents/tools/human_approval.ipynb

@@ -0,0 +1,322 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "144e77fe",
+   "metadata": {},
+   "source": [
+    "# Human-in-the-loop Tool Validation\n",
+    "\n",
+    "This walkthrough demonstrates how to add Human validation to any Tool. We'll do this using the `HumanApprovalCallbackhandler`.\n",
+    "\n",
+    "Let's suppose we need to make use of the ShellTool. Adding this tool to an automated flow poses obvious risks. Let's see how we could enforce manual human approval of inputs going into this tool.\n",
+    "\n",
+    "**Note**: We generally recommend against using the ShellTool. There's a lot of ways to misuse it, and it's not required for most use cases. We employ it here only for demonstration purposes."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "ad84c682",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.callbacks import HumanApprovalCallbackHandler\n",
+    "from langchain.tools import ShellTool"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "70090dd6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool = ShellTool()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "20d5175f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello World!\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(tool.run('echo Hello World!'))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0475dd6",
+   "metadata": {},
+   "source": [
+    "## Adding Human Approval\n",
+    "Adding the default HumanApprovalCallbackHandler to the tool will make it so that a user has to manually approve every input to the tool before the command is actually executed."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "f1c88793",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tool = ShellTool(callbacks=[HumanApprovalCallbackHandler()])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "f749815d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Do you approve of the following input? Anything except 'Y'/'Yes' (case-insensitive) will be treated as a no.\n",
+      "\n",
+      "ls /usr\n",
+      "yes\n",
+      "\u001b[35mX11\u001b[m\u001b[m\n",
+      "\u001b[35mX11R6\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mbin\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mlib\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mlibexec\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mlocal\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36msbin\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mshare\u001b[m\u001b[m\n",
+      "\u001b[1m\u001b[36mstandalone\u001b[m\u001b[m\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(tool.run(\"ls /usr\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "b6e455d1",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Do you approve of the following input? Anything except 'Y'/'Yes' (case-insensitive) will be treated as a no.\n",
+      "\n",
+      "ls /private\n",
+      "no\n"
+     ]
+    },
+    {
+     "ename": "HumanRejectedException",
+     "evalue": "Inputs ls /private to tool {'name': 'terminal', 'description': 'Run shell commands on this MacOS machine.'} were rejected.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mHumanRejectedException\u001b[0m                    Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[17], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[43mtool\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mls /private\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m)\n",
+      "File \u001b[0;32m~/langchain/langchain/tools/base.py:257\u001b[0m, in \u001b[0;36mBaseTool.run\u001b[0;34m(self, tool_input, verbose, start_color, color, callbacks, **kwargs)\u001b[0m\n\u001b[1;32m    255\u001b[0m \u001b[38;5;66;03m# TODO: maybe also pass through run_manager is _run supports kwargs\u001b[39;00m\n\u001b[1;32m    256\u001b[0m new_arg_supported \u001b[38;5;241m=\u001b[39m signature(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_run)\u001b[38;5;241m.\u001b[39mparameters\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_manager\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m--> 257\u001b[0m run_manager \u001b[38;5;241m=\u001b[39m \u001b[43mcallback_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mon_tool_start\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    258\u001b[0m \u001b[43m    \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mname\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mdescription\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdescription\u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    259\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43misinstance\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    260\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcolor\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstart_color\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    261\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    262\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    263\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    264\u001b[0m     tool_args, tool_kwargs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_to_args_and_kwargs(parsed_input)\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:672\u001b[0m, in \u001b[0;36mCallbackManager.on_tool_start\u001b[0;34m(self, serialized, input_str, run_id, parent_run_id, **kwargs)\u001b[0m\n\u001b[1;32m    669\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m run_id \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    670\u001b[0m     run_id \u001b[38;5;241m=\u001b[39m uuid4()\n\u001b[0;32m--> 672\u001b[0m \u001b[43m_handle_event\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    673\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandlers\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    674\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mon_tool_start\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    675\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mignore_agent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    676\u001b[0m \u001b[43m    \u001b[49m\u001b[43mserialized\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    677\u001b[0m \u001b[43m    \u001b[49m\u001b[43minput_str\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    678\u001b[0m \u001b[43m    \u001b[49m\u001b[43mrun_id\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    679\u001b[0m \u001b[43m    \u001b[49m\u001b[43mparent_run_id\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparent_run_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    680\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    681\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    683\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m CallbackManagerForToolRun(\n\u001b[1;32m    684\u001b[0m     run_id, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mhandlers, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39minheritable_handlers, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mparent_run_id\n\u001b[1;32m    685\u001b[0m )\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:157\u001b[0m, in \u001b[0;36m_handle_event\u001b[0;34m(handlers, event_name, ignore_condition_name, *args, **kwargs)\u001b[0m\n\u001b[1;32m    155\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    156\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m handler\u001b[38;5;241m.\u001b[39mraise_error:\n\u001b[0;32m--> 157\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[1;32m    158\u001b[0m     logging\u001b[38;5;241m.\u001b[39mwarning(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError in \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mevent_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m callback: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:139\u001b[0m, in \u001b[0;36m_handle_event\u001b[0;34m(handlers, event_name, ignore_condition_name, *args, **kwargs)\u001b[0m\n\u001b[1;32m    135\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    136\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m ignore_condition_name \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(\n\u001b[1;32m    137\u001b[0m         handler, ignore_condition_name\n\u001b[1;32m    138\u001b[0m     ):\n\u001b[0;32m--> 139\u001b[0m         \u001b[38;5;28;43mgetattr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mhandler\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mevent_name\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    140\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mNotImplementedError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    141\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m event_name \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mon_chat_model_start\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/human.py:48\u001b[0m, in \u001b[0;36mHumanApprovalCallbackHandler.on_tool_start\u001b[0;34m(self, serialized, input_str, run_id, parent_run_id, **kwargs)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mon_tool_start\u001b[39m(\n\u001b[1;32m     39\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m     40\u001b[0m     serialized: Dict[\u001b[38;5;28mstr\u001b[39m, Any],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     45\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs: Any,\n\u001b[1;32m     46\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Any:\n\u001b[1;32m     47\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_check(serialized) \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_approve(input_str):\n\u001b[0;32m---> 48\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m HumanRejectedException(\n\u001b[1;32m     49\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInputs \u001b[39m\u001b[38;5;132;01m{\u001b[39;00minput_str\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m to tool \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mserialized\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m were rejected.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m     50\u001b[0m         )\n",
+      "\u001b[0;31mHumanRejectedException\u001b[0m: Inputs ls /private to tool {'name': 'terminal', 'description': 'Run shell commands on this MacOS machine.'} were rejected."
+     ]
+    }
+   ],
+   "source": [
+    "print(tool.run(\"ls /private\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3b092ec",
+   "metadata": {},
+   "source": [
+    "## Configuring Human Approval\n",
+    "\n",
+    "Let's suppose we have an agent that takes in multiple tools, and we want it to only trigger human approval requests on certain tools and certain inputs. We can configure out callback handler to do just this."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4521c581",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.agents import load_tools\n",
+    "from langchain.agents import initialize_agent\n",
+    "from langchain.agents import AgentType\n",
+    "from langchain.llms import OpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "9e8d5428",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def _should_check(serialized_obj: dict) -> bool:\n",
+    "    # Only require approval on ShellTool.\n",
+    "    return serialized_obj.get(\"name\") == \"terminal\"\n",
+    "\n",
+    "def _approve(_input: str) -> bool:\n",
+    "    if _input == \"echo 'Hello World'\":\n",
+    "        return True\n",
+    "    msg = (\n",
+    "        \"Do you approve of the following input? \"\n",
+    "        \"Anything except 'Y'/'Yes' (case-insensitive) will be treated as a no.\"\n",
+    "    )\n",
+    "    msg += \"\\n\\n\" + _input + \"\\n\"\n",
+    "    resp = input(msg)\n",
+    "    return resp.lower() in (\"yes\", \"y\")\n",
+    "\n",
+    "callbacks = [HumanApprovalCallbackHandler(should_check=_should_check, approve=_approve)]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 34,
+   "id": "9922898e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(temperature=0)\n",
+    "tools = load_tools([\"wikipedia\", \"llm-math\", \"terminal\"], llm=llm)\n",
+    "agent = initialize_agent(\n",
+    "    tools, \n",
+    "    llm, \n",
+    "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, \n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "e69ea402",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Konrad Adenauer became Chancellor of Germany in 1949, 74 years ago.'"
+      ]
+     },
+     "execution_count": 38,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany.\", callbacks=callbacks)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "id": "25182a7e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'Hello World'"
+      ]
+     },
+     "execution_count": 36,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "agent.run(\"print 'Hello World' in the terminal\", callbacks=callbacks)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "2f5a93d0",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Do you approve of the following input? Anything except 'Y'/'Yes' (case-insensitive) will be treated as a no.\n",
+      "\n",
+      "ls /private\n",
+      "no\n"
+     ]
+    },
+    {
+     "ename": "HumanRejectedException",
+     "evalue": "Inputs ls /private to tool {'name': 'terminal', 'description': 'Run shell commands on this MacOS machine.'} were rejected.",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mHumanRejectedException\u001b[0m                    Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[39], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43magent\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlist all directories in /private\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcallbacks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcallbacks\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/langchain/langchain/chains/base.py:236\u001b[0m, in \u001b[0;36mChain.run\u001b[0;34m(self, callbacks, *args, **kwargs)\u001b[0m\n\u001b[1;32m    234\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(args) \u001b[38;5;241m!=\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[1;32m    235\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m`run` supports only one positional argument.\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m--> 236\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcallbacks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcallbacks\u001b[49m\u001b[43m)\u001b[49m[\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput_keys[\u001b[38;5;241m0\u001b[39m]]\n\u001b[1;32m    238\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m kwargs \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m args:\n\u001b[1;32m    239\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m(kwargs, callbacks\u001b[38;5;241m=\u001b[39mcallbacks)[\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput_keys[\u001b[38;5;241m0\u001b[39m]]\n",
+      "File \u001b[0;32m~/langchain/langchain/chains/base.py:140\u001b[0m, in \u001b[0;36mChain.__call__\u001b[0;34m(self, inputs, return_only_outputs, callbacks)\u001b[0m\n\u001b[1;32m    138\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m (\u001b[38;5;167;01mKeyboardInterrupt\u001b[39;00m, \u001b[38;5;167;01mException\u001b[39;00m) \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    139\u001b[0m     run_manager\u001b[38;5;241m.\u001b[39mon_chain_error(e)\n\u001b[0;32m--> 140\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[1;32m    141\u001b[0m run_manager\u001b[38;5;241m.\u001b[39mon_chain_end(outputs)\n\u001b[1;32m    142\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mprep_outputs(inputs, outputs, return_only_outputs)\n",
+      "File \u001b[0;32m~/langchain/langchain/chains/base.py:134\u001b[0m, in \u001b[0;36mChain.__call__\u001b[0;34m(self, inputs, return_only_outputs, callbacks)\u001b[0m\n\u001b[1;32m    128\u001b[0m run_manager \u001b[38;5;241m=\u001b[39m callback_manager\u001b[38;5;241m.\u001b[39mon_chain_start(\n\u001b[1;32m    129\u001b[0m     {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mname\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__class__\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m},\n\u001b[1;32m    130\u001b[0m     inputs,\n\u001b[1;32m    131\u001b[0m )\n\u001b[1;32m    132\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    133\u001b[0m     outputs \u001b[38;5;241m=\u001b[39m (\n\u001b[0;32m--> 134\u001b[0m         \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mrun_manager\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_manager\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    135\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m new_arg_supported\n\u001b[1;32m    136\u001b[0m         \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call(inputs)\n\u001b[1;32m    137\u001b[0m     )\n\u001b[1;32m    138\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m (\u001b[38;5;167;01mKeyboardInterrupt\u001b[39;00m, \u001b[38;5;167;01mException\u001b[39;00m) \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    139\u001b[0m     run_manager\u001b[38;5;241m.\u001b[39mon_chain_error(e)\n",
+      "File \u001b[0;32m~/langchain/langchain/agents/agent.py:953\u001b[0m, in \u001b[0;36mAgentExecutor._call\u001b[0;34m(self, inputs, run_manager)\u001b[0m\n\u001b[1;32m    951\u001b[0m \u001b[38;5;66;03m# We now enter the agent loop (until it returns something).\u001b[39;00m\n\u001b[1;32m    952\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_continue(iterations, time_elapsed):\n\u001b[0;32m--> 953\u001b[0m     next_step_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_take_next_step\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    954\u001b[0m \u001b[43m        \u001b[49m\u001b[43mname_to_tool_map\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    955\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcolor_mapping\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    956\u001b[0m \u001b[43m        \u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    957\u001b[0m \u001b[43m        \u001b[49m\u001b[43mintermediate_steps\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    958\u001b[0m \u001b[43m        \u001b[49m\u001b[43mrun_manager\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_manager\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    959\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    960\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(next_step_output, AgentFinish):\n\u001b[1;32m    961\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_return(\n\u001b[1;32m    962\u001b[0m             next_step_output, intermediate_steps, run_manager\u001b[38;5;241m=\u001b[39mrun_manager\n\u001b[1;32m    963\u001b[0m         )\n",
+      "File \u001b[0;32m~/langchain/langchain/agents/agent.py:820\u001b[0m, in \u001b[0;36mAgentExecutor._take_next_step\u001b[0;34m(self, name_to_tool_map, color_mapping, inputs, intermediate_steps, run_manager)\u001b[0m\n\u001b[1;32m    818\u001b[0m         tool_run_kwargs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mllm_prefix\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    819\u001b[0m     \u001b[38;5;66;03m# We then call the tool on the tool input to get an observation\u001b[39;00m\n\u001b[0;32m--> 820\u001b[0m     observation \u001b[38;5;241m=\u001b[39m \u001b[43mtool\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    821\u001b[0m \u001b[43m        \u001b[49m\u001b[43magent_action\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtool_input\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    822\u001b[0m \u001b[43m        \u001b[49m\u001b[43mverbose\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mverbose\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    823\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcolor\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcolor\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    824\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcallbacks\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mrun_manager\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m    825\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mtool_run_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    826\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    827\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    828\u001b[0m     tool_run_kwargs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39magent\u001b[38;5;241m.\u001b[39mtool_run_logging_kwargs()\n",
+      "File \u001b[0;32m~/langchain/langchain/tools/base.py:257\u001b[0m, in \u001b[0;36mBaseTool.run\u001b[0;34m(self, tool_input, verbose, start_color, color, callbacks, **kwargs)\u001b[0m\n\u001b[1;32m    255\u001b[0m \u001b[38;5;66;03m# TODO: maybe also pass through run_manager is _run supports kwargs\u001b[39;00m\n\u001b[1;32m    256\u001b[0m new_arg_supported \u001b[38;5;241m=\u001b[39m signature(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_run)\u001b[38;5;241m.\u001b[39mparameters\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_manager\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m--> 257\u001b[0m run_manager \u001b[38;5;241m=\u001b[39m \u001b[43mcallback_manager\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mon_tool_start\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    258\u001b[0m \u001b[43m    \u001b[49m\u001b[43m{\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mname\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mdescription\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdescription\u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    259\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43misinstance\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mtool_input\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    260\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcolor\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstart_color\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    261\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    262\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    263\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    264\u001b[0m     tool_args, tool_kwargs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_to_args_and_kwargs(parsed_input)\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:672\u001b[0m, in \u001b[0;36mCallbackManager.on_tool_start\u001b[0;34m(self, serialized, input_str, run_id, parent_run_id, **kwargs)\u001b[0m\n\u001b[1;32m    669\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m run_id \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    670\u001b[0m     run_id \u001b[38;5;241m=\u001b[39m uuid4()\n\u001b[0;32m--> 672\u001b[0m \u001b[43m_handle_event\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    673\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mhandlers\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    674\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mon_tool_start\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    675\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mignore_agent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    676\u001b[0m \u001b[43m    \u001b[49m\u001b[43mserialized\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    677\u001b[0m \u001b[43m    \u001b[49m\u001b[43minput_str\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    678\u001b[0m \u001b[43m    \u001b[49m\u001b[43mrun_id\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mrun_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    679\u001b[0m \u001b[43m    \u001b[49m\u001b[43mparent_run_id\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparent_run_id\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    680\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    681\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    683\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m CallbackManagerForToolRun(\n\u001b[1;32m    684\u001b[0m     run_id, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mhandlers, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39minheritable_handlers, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mparent_run_id\n\u001b[1;32m    685\u001b[0m )\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:157\u001b[0m, in \u001b[0;36m_handle_event\u001b[0;34m(handlers, event_name, ignore_condition_name, *args, **kwargs)\u001b[0m\n\u001b[1;32m    155\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    156\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m handler\u001b[38;5;241m.\u001b[39mraise_error:\n\u001b[0;32m--> 157\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m e\n\u001b[1;32m    158\u001b[0m     logging\u001b[38;5;241m.\u001b[39mwarning(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError in \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mevent_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m callback: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/manager.py:139\u001b[0m, in \u001b[0;36m_handle_event\u001b[0;34m(handlers, event_name, ignore_condition_name, *args, **kwargs)\u001b[0m\n\u001b[1;32m    135\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    136\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m ignore_condition_name \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(\n\u001b[1;32m    137\u001b[0m         handler, ignore_condition_name\n\u001b[1;32m    138\u001b[0m     ):\n\u001b[0;32m--> 139\u001b[0m         \u001b[38;5;28;43mgetattr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mhandler\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mevent_name\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    140\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mNotImplementedError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    141\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m event_name \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mon_chat_model_start\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n",
+      "File \u001b[0;32m~/langchain/langchain/callbacks/human.py:48\u001b[0m, in \u001b[0;36mHumanApprovalCallbackHandler.on_tool_start\u001b[0;34m(self, serialized, input_str, run_id, parent_run_id, **kwargs)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mon_tool_start\u001b[39m(\n\u001b[1;32m     39\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m     40\u001b[0m     serialized: Dict[\u001b[38;5;28mstr\u001b[39m, Any],\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m     45\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs: Any,\n\u001b[1;32m     46\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Any:\n\u001b[1;32m     47\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_check(serialized) \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_approve(input_str):\n\u001b[0;32m---> 48\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m HumanRejectedException(\n\u001b[1;32m     49\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInputs \u001b[39m\u001b[38;5;132;01m{\u001b[39;00minput_str\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m to tool \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mserialized\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m were rejected.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m     50\u001b[0m         )\n",
+      "\u001b[0;31mHumanRejectedException\u001b[0m: Inputs ls /private to tool {'name': 'terminal', 'description': 'Run shell commands on this MacOS machine.'} were rejected."
+     ]
+    }
+   ],
+   "source": [
+    "agent.run(\"list all directories in /private\", callbacks=callbacks)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c0b47e26",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "venv",
+   "language": "python",
+   "name": "venv"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/callbacks/examples/argilla.ipynb

@@ -0,0 +1,423 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Argilla\n",
+    "\n",
+    "![Argilla - Open-source data platform for LLMs](https://argilla.io/og.png)\n",
+    "\n",
+    ">[Argilla](https://argilla.io/) is an open-source data curation platform for LLMs.\n",
+    "> Using Argilla, everyone can build robust language models through faster data curation \n",
+    "> using both human and machine feedback. We provide support for each step in the MLOps cycle, \n",
+    "> from data labeling to model monitoring.\n",
+    "\n",
+    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/modules/callbacks/examples/argilla.ipynb\">\n",
+    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
+    "</a>"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this guide we will demonstrate how to track the inputs and reponses of your LLM to generate a dataset in Argilla, using the `ArgillaCallbackHandler`.\n",
+    "\n",
+    "It's useful to keep track of the inputs and outputs of your LLMs to generate datasets for future fine-tuning. This is especially useful when you're using a LLM to generate data for a specific task, such as question answering, summarization, or translation."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## Installation and Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install argilla --upgrade\n",
+    "!pip install openai"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Getting API Credentials\n",
+    "\n",
+    "To get the Argilla API credentials, follow the next steps:\n",
+    "\n",
+    "1. Go to your Argilla UI.\n",
+    "2. Click on your profile picture and go to \"My settings\".\n",
+    "3. Then copy the API Key.\n",
+    "\n",
+    "In Argilla the API URL will be the same as the URL of your Argilla UI.\n",
+    "\n",
+    "To get the OpenAI API credentials, please visit https://platform.openai.com/account/api-keys"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"ARGILLA_API_URL\"] = \"...\"\n",
+    "os.environ[\"ARGILLA_API_KEY\"] = \"...\"\n",
+    "\n",
+    "os.environ[\"OPENAI_API_KEY\"] = \"...\""
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Setup Argilla\n",
+    "\n",
+    "To use the `ArgillaCallbackHandler` we will need to create a new `FeedbackDataset` in Argilla to keep track of your LLM experiments. To do so, please use the following code:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import argilla as rg"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from packaging.version import parse as parse_version\n",
+    "\n",
+    "if parse_version(rg.__version__) < parse_version(\"1.8.0\"):\n",
+    "    raise RuntimeError(\n",
+    "        \"`FeedbackDataset` is only available in Argilla v1.8.0 or higher, please \"\n",
+    "        \"upgrade `argilla` as `pip install argilla --upgrade`.\"\n",
+    "    )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset = rg.FeedbackDataset(\n",
+    "    fields=[\n",
+    "        rg.TextField(name=\"prompt\"),\n",
+    "        rg.TextField(name=\"response\"),\n",
+    "    ],\n",
+    "    questions=[\n",
+    "        rg.RatingQuestion(\n",
+    "            name=\"response-rating\",\n",
+    "            description=\"How would you rate the quality of the response?\",\n",
+    "            values=[1, 2, 3, 4, 5],\n",
+    "            required=True,\n",
+    "        ),\n",
+    "        rg.TextQuestion(\n",
+    "            name=\"response-feedback\",\n",
+    "            description=\"What feedback do you have for the response?\",\n",
+    "            required=False,\n",
+    "        ),\n",
+    "    ],\n",
+    "    guidelines=\"You're asked to rate the quality of the response and provide feedback.\",\n",
+    ")\n",
+    "\n",
+    "rg.init(\n",
+    "    api_url=os.environ[\"ARGILLA_API_URL\"],\n",
+    "    api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
+    ")\n",
+    "\n",
+    "dataset.push_to_argilla(\"langchain-dataset\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> 📌 NOTE: at the moment, just the prompt-response pairs are supported as `FeedbackDataset.fields`, so the `ArgillaCallbackHandler` will just track the prompt i.e. the LLM input, and the response i.e. the LLM output."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Tracking"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "To use the `ArgillaCallbackHandler` you can either use the following code, or just reproduce one of the examples presented in the following sections."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.callbacks import ArgillaCallbackHandler\n",
+    "\n",
+    "argilla_callback = ArgillaCallbackHandler(\n",
+    "    dataset_name=\"langchain-dataset\",\n",
+    "    api_url=os.environ[\"ARGILLA_API_URL\"],\n",
+    "    api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
+    ")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Scenario 1: Tracking an LLM\n",
+    "\n",
+    "First, let's just run a single LLM a few times and capture the resulting prompt-response pairs in Argilla."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "LLMResult(generations=[[Generation(text='\\n\\nQ: What did the fish say when he hit the wall? \\nA: Dam.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nThe Moon \\n\\nThe moon is high in the midnight sky,\\nSparkling like a star above.\\nThe night so peaceful, so serene,\\nFilling up the air with love.\\n\\nEver changing and renewing,\\nA never-ending light of grace.\\nThe moon remains a constant view,\\nA reminder of life’s gentle pace.\\n\\nThrough time and space it guides us on,\\nA never-fading beacon of hope.\\nThe moon shines down on us all,\\nAs it continues to rise and elope.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nQ. What did one magnet say to the other magnet?\\nA. \"I find you very attractive!\"', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=\"\\n\\nThe world is charged with the grandeur of God.\\nIt will flame out, like shining from shook foil;\\nIt gathers to a greatness, like the ooze of oil\\nCrushed. Why do men then now not reck his rod?\\n\\nGenerations have trod, have trod, have trod;\\nAnd all is seared with trade; bleared, smeared with toil;\\nAnd wears man's smudge and shares man's smell: the soil\\nIs bare now, nor can foot feel, being shod.\\n\\nAnd for all this, nature is never spent;\\nThere lives the dearest freshness deep down things;\\nAnd though the last lights off the black West went\\nOh, morning, at the brown brink eastward, springs —\\n\\nBecause the Holy Ghost over the bent\\nWorld broods with warm breast and with ah! bright wings.\\n\\n~Gerard Manley Hopkins\", generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nQ: What did one ocean say to the other ocean?\\nA: Nothing, they just waved.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=\"\\n\\nA poem for you\\n\\nOn a field of green\\n\\nThe sky so blue\\n\\nA gentle breeze, the sun above\\n\\nA beautiful world, for us to love\\n\\nLife is a journey, full of surprise\\n\\nFull of joy and full of surprise\\n\\nBe brave and take small steps\\n\\nThe future will be revealed with depth\\n\\nIn the morning, when dawn arrives\\n\\nA fresh start, no reason to hide\\n\\nSomewhere down the road, there's a heart that beats\\n\\nBelieve in yourself, you'll always succeed.\", generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {'completion_tokens': 504, 'total_tokens': 528, 'prompt_tokens': 24}, 'model_name': 'text-davinci-003'})"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.llms import OpenAI\n",
+    "\n",
+    "argilla_callback = ArgillaCallbackHandler(\n",
+    "    dataset_name=\"langchain-dataset\",\n",
+    "    api_url=os.environ[\"ARGILLA_API_URL\"],\n",
+    "    api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
+    ")\n",
+    "callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
+    "\n",
+    "llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
+    "llm.generate([\"Tell me a joke\", \"Tell me a poem\"] * 3)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![Argilla UI with LangChain LLM input-response](https://docs.argilla.io/en/latest/_images/llm.png)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Scenario 2: Tracking an LLM in a chain\n",
+    "\n",
+    "Then we can create a chain using a prompt template, and then track the initial prompt and the final response in Argilla."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3mYou are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
+      "Title: Documentary about Bigfoot in Paris\n",
+      "Playwright: This is a synopsis for the above play:\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[{'text': \"\\n\\nDocumentary about Bigfoot in Paris focuses on the story of a documentary filmmaker and their search for evidence of the legendary Bigfoot creature in the city of Paris. The play follows the filmmaker as they explore the city, meeting people from all walks of life who have had encounters with the mysterious creature. Through their conversations, the filmmaker unravels the story of Bigfoot and finds out the truth about the creature's presence in Paris. As the story progresses, the filmmaker learns more and more about the mysterious creature, as well as the different perspectives of the people living in the city, and what they think of the creature. In the end, the filmmaker's findings lead them to some surprising and heartwarming conclusions about the creature's existence and the importance it holds in the lives of the people in Paris.\"}]"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.chains import LLMChain\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "\n",
+    "argilla_callback = ArgillaCallbackHandler(\n",
+    "    dataset_name=\"langchain-dataset\",\n",
+    "    api_url=os.environ[\"ARGILLA_API_URL\"],\n",
+    "    api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
+    ")\n",
+    "callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
+    "llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
+    "\n",
+    "template = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
+    "Title: {title}\n",
+    "Playwright: This is a synopsis for the above play:\"\"\"\n",
+    "prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
+    "synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)\n",
+    "\n",
+    "test_prompts = [{\"title\": \"Documentary about Bigfoot in Paris\"}]\n",
+    "synopsis_chain.apply(test_prompts)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![Argilla UI with LangChain Chain input-response](https://docs.argilla.io/en/latest/_images/chain.png)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Scenario 3: Using an Agent with Tools\n",
+    "\n",
+    "Finally, as a more advanced workflow, you can create an agent that uses some tools. So that `ArgillaCallbackHandler` will keep track of the input and the output, but not about the intermediate steps/thoughts, so that given a prompt we log the original prompt and the final response to that given prompt."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> Note that for this scenario we'll be using Google Search API (Serp API) so you will need to both install `google-search-results` as `pip install google-search-results`, and to set the Serp API Key as `os.environ[\"SERPAPI_API_KEY\"] = \"...\"` (you can find it at https://serpapi.com/dashboard), otherwise the example below won't work."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3m I need to answer a historical question\n",
+      "Action: Search\n",
+      "Action Input: \"who was the first president of the United States of America\" \u001b[0m\n",
+      "Observation: \u001b[36;1m\u001b[1;3mGeorge Washington\u001b[0m\n",
+      "Thought:\u001b[32;1m\u001b[1;3m George Washington was the first president\n",
+      "Final Answer: George Washington was the first president of the United States of America.\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'George Washington was the first president of the United States of America.'"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from langchain.agents import AgentType, initialize_agent, load_tools\n",
+    "from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
+    "from langchain.llms import OpenAI\n",
+    "\n",
+    "argilla_callback = ArgillaCallbackHandler(\n",
+    "    dataset_name=\"langchain-dataset\",\n",
+    "    api_url=os.environ[\"ARGILLA_API_URL\"],\n",
+    "    api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
+    ")\n",
+    "callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
+    "llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
+    "\n",
+    "tools = load_tools([\"serpapi\"], llm=llm, callbacks=callbacks)\n",
+    "agent = initialize_agent(\n",
+    "    tools,\n",
+    "    llm,\n",
+    "    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
+    "    callbacks=callbacks,\n",
+    ")\n",
+    "agent.run(\"Who was the first president of the United States of America?\")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![Argilla UI with LangChain Agent input-response](https://docs.argilla.io/en/latest/_images/agent.png)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.3"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "a53ebf4a859167383b364e7e7521d0add3c2dbbdecce4edf676e8c4634ff3fbb"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

docs/modules/callbacks/filecallbackhandler.ipynb

@@ -0,0 +1,175 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "63b87b91",
+   "metadata": {},
+   "source": [
+    "# Logging to file\n",
+    "This example shows how to print logs to file. It shows how to use the `FileCallbackHandler`, which does the same thing as [`StdOutCallbackHandler`](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#using-an-existing-handler), but instead writes the output to file. It also uses the `loguru` library to log other outputs that are not captured by the handler."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "6cb156cc",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
+      "Prompt after formatting:\n",
+      "\u001b[32;1m\u001b[1;3m1 + 2 = \u001b[0m\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\u001b[32m2023-06-01 18:36:38.929\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36m__main__\u001b[0m:\u001b[36m<module>\u001b[0m:\u001b[36m20\u001b[0m - \u001b[1m\n",
+      "\n",
+      "3\u001b[0m\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "from loguru import logger\n",
+    "\n",
+    "from langchain.callbacks import FileCallbackHandler\n",
+    "from langchain.chains import LLMChain\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "\n",
+    "logfile = 'output.log'\n",
+    "\n",
+    "logger.add(logfile, colorize=True, enqueue=True)\n",
+    "handler = FileCallbackHandler(logfile)\n",
+    "\n",
+    "llm = OpenAI()\n",
+    "prompt = PromptTemplate.from_template(\"1 + {number} = \")\n",
+    "\n",
+    "# this chain will both print to stdout (because verbose=True) and write to 'output.log'\n",
+    "# if verbose=False, the FileCallbackHandler will still write to 'output.log'\n",
+    "chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler], verbose=True)\n",
+    "answer = chain.run(number=2)\n",
+    "logger.info(answer)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9c50d54f",
+   "metadata": {},
+   "source": [
+    "Now we can open the file `output.log` to see that the output has been captured."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "aa32dc0a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install ansi2html > /dev/null"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "4af00719",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">\n",
+       "<html>\n",
+       "<head>\n",
+       "<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\">\n",
+       "<title></title>\n",
+       "<style type=\"text/css\">\n",
+       ".ansi2html-content { display: inline; white-space: pre-wrap; word-wrap: break-word; }\n",
+       ".body_foreground { color: #AAAAAA; }\n",
+       ".body_background { background-color: #000000; }\n",
+       ".inv_foreground { color: #000000; }\n",
+       ".inv_background { background-color: #AAAAAA; }\n",
+       ".ansi1 { font-weight: bold; }\n",
+       ".ansi3 { font-style: italic; }\n",
+       ".ansi32 { color: #00aa00; }\n",
+       ".ansi36 { color: #00aaaa; }\n",
+       "</style>\n",
+       "</head>\n",
+       "<body class=\"body_foreground body_background\" style=\"font-size: normal;\" >\n",
+       "<pre class=\"ansi2html-content\">\n",
+       "\n",
+       "\n",
+       "<span class=\"ansi1\">&gt; Entering new LLMChain chain...</span>\n",
+       "Prompt after formatting:\n",
+       "<span class=\"ansi1 ansi32\"></span><span class=\"ansi1 ansi3 ansi32\">1 + 2 = </span>\n",
+       "\n",
+       "<span class=\"ansi1\">&gt; Finished chain.</span>\n",
+       "<span class=\"ansi32\">2023-06-01 18:36:38.929</span> | <span class=\"ansi1\">INFO    </span> | <span class=\"ansi36\">__main__</span>:<span class=\"ansi36\">&lt;module&gt;</span>:<span class=\"ansi36\">20</span> - <span class=\"ansi1\">\n",
+       "\n",
+       "3</span>\n",
+       "\n",
+       "</pre>\n",
+       "</body>\n",
+       "\n",
+       "</html>\n"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from IPython.display import display, HTML\n",
+    "from ansi2html import Ansi2HTMLConverter\n",
+    "\n",
+    "with open('output.log', 'r') as f:\n",
+    "    content = f.read()\n",
+    "\n",
+    "conv = Ansi2HTMLConverter()\n",
+    "html = conv.convert(content, full=True)\n",
+    "\n",
+    "display(HTML(html))"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/chains.rst

@@ -6,14 +6,13 @@ Chains
 
 
 Using an LLM in isolation is fine for some simple applications,
-but many more complex ones require chaining LLMs - either with each other or with other experts.
-LangChain provides a standard interface for Chains, as well as some common implementations of chains for ease of use.
+but more complex applications require chaining LLMs - either with each other or with other experts.
+LangChain provides a standard interface for **Chains**, as well as several common implementations of chains.
 
-The following sections of documentation are provided:
+|
+- `Getting Started <./chains/getting_started.html>`_: An overview of chains.
 
-- `Getting Started <./chains/getting_started.html>`_: A getting started guide for chains, to get you up and running quickly.
-
-- `How-To Guides <./chains/how_to_guides.html>`_: A collection of how-to guides. These highlight how to use various types of chains.
+- `How-To Guides <./chains/how_to_guides.html>`_: How-to guides about various types of chains.
 
 - `Reference <../reference/modules/chains.html>`_: API reference documentation for all Chain classes.
 

docs/modules/chains/examples/graph_cypher_qa.ipynb

@@ -177,7 +177,7 @@
       "\u001b[32;1m\u001b[1;3mMATCH (a:Actor)-[:ACTED_IN]->(m:Movie {name: 'Top Gun'})\n",
       "RETURN a.name\u001b[0m\n",
       "Full Context:\n",
-      "\u001b[32;1m\u001b[1;3m[{'a.name': 'Tom Cruise'}, {'a.name': 'Val Kilmer'}, {'a.name': 'Anthony Edwards'}, {'a.name': 'Meg Ryan'}]\u001b[0m\n",
+      "\u001b[32;1m\u001b[1;3m[{'a.name': 'Val Kilmer'}, {'a.name': 'Anthony Edwards'}, {'a.name': 'Meg Ryan'}, {'a.name': 'Tom Cruise'}]\u001b[0m\n",
       "\n",
       "\u001b[1m> Finished chain.\u001b[0m\n"
      ]
@@ -185,7 +185,7 @@
     {
      "data": {
       "text/plain": [
-       "'Tom Cruise, Val Kilmer, Anthony Edwards, and Meg Ryan played in Top Gun.'"
+       "'Val Kilmer, Anthony Edwards, Meg Ryan, and Tom Cruise played in Top Gun.'"
       ]
      },
      "execution_count": 7,
@@ -197,10 +197,180 @@
     "chain.run(\"Who played in Top Gun?\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "2d28c4df",
+   "metadata": {},
+   "source": [
+    "## Limit the number of results\n",
+    "You can limit the number of results from the Cypher QA Chain using the `top_k` parameter.\n",
+    "The default is 10."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "df230946",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = GraphCypherQAChain.from_llm(\n",
+    "    ChatOpenAI(temperature=0), graph=graph, verbose=True, top_k=2\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "3f1600ee",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new GraphCypherQAChain chain...\u001b[0m\n",
+      "Generated Cypher:\n",
+      "\u001b[32;1m\u001b[1;3mMATCH (a:Actor)-[:ACTED_IN]->(m:Movie {name: 'Top Gun'})\n",
+      "RETURN a.name\u001b[0m\n",
+      "Full Context:\n",
+      "\u001b[32;1m\u001b[1;3m[{'a.name': 'Val Kilmer'}, {'a.name': 'Anthony Edwards'}]\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'Val Kilmer and Anthony Edwards played in Top Gun.'"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run(\"Who played in Top Gun?\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88c16206",
+   "metadata": {},
+   "source": [
+    "## Return intermediate results\n",
+    "You can return intermediate steps from the Cypher QA Chain using the `return_intermediate_steps` parameter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "e412f36b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = GraphCypherQAChain.from_llm(\n",
+    "    ChatOpenAI(temperature=0), graph=graph, verbose=True, return_intermediate_steps=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "4f4699dc",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new GraphCypherQAChain chain...\u001b[0m\n",
+      "Generated Cypher:\n",
+      "\u001b[32;1m\u001b[1;3mMATCH (a:Actor)-[:ACTED_IN]->(m:Movie {name: 'Top Gun'})\n",
+      "RETURN a.name\u001b[0m\n",
+      "Full Context:\n",
+      "\u001b[32;1m\u001b[1;3m[{'a.name': 'Val Kilmer'}, {'a.name': 'Anthony Edwards'}, {'a.name': 'Meg Ryan'}, {'a.name': 'Tom Cruise'}]\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n",
+      "Intermediate steps: [{'query': \"MATCH (a:Actor)-[:ACTED_IN]->(m:Movie {name: 'Top Gun'})\\nRETURN a.name\"}, {'context': [{'a.name': 'Val Kilmer'}, {'a.name': 'Anthony Edwards'}, {'a.name': 'Meg Ryan'}, {'a.name': 'Tom Cruise'}]}]\n",
+      "Final answer: Val Kilmer, Anthony Edwards, Meg Ryan, and Tom Cruise played in Top Gun.\n"
+     ]
+    }
+   ],
+   "source": [
+    "result = chain(\"Who played in Top Gun?\")\n",
+    "print(f\"Intermediate steps: {result['intermediate_steps']}\")\n",
+    "print(f\"Final answer: {result['result']}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d6e1b054",
+   "metadata": {},
+   "source": [
+    "## Return direct results\n",
+    "You can return direct results from the Cypher QA Chain using the `return_direct` parameter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "2d3acf10",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = GraphCypherQAChain.from_llm(\n",
+    "    ChatOpenAI(temperature=0), graph=graph, verbose=True, return_direct=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "b0a9d143",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new GraphCypherQAChain chain...\u001b[0m\n",
+      "Generated Cypher:\n",
+      "\u001b[32;1m\u001b[1;3mMATCH (a:Actor)-[:ACTED_IN]->(m:Movie {name: 'Top Gun'})\n",
+      "RETURN a.name\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[{'a.name': 'Val Kilmer'},\n",
+       " {'a.name': 'Anthony Edwards'},\n",
+       " {'a.name': 'Meg Ryan'},\n",
+       " {'a.name': 'Tom Cruise'}]"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run(\"Who played in Top Gun?\")"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "b4825316",
+   "id": "74d0a36f",
    "metadata": {},
    "outputs": [],
    "source": []
@@ -222,7 +392,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.1"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,

docs/modules/chains/examples/graph_nebula_qa.ipynb

@@ -0,0 +1,270 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "c94240f5",
+   "metadata": {},
+   "source": [
+    "# NebulaGraphQAChain\n",
+    "\n",
+    "This notebook shows how to use LLMs to provide a natural language interface to NebulaGraph database."
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "dbc0ee68",
+   "metadata": {},
+   "source": [
+    "You will need to have a running NebulaGraph cluster, for which you can run a containerized cluster by running the following script:\n",
+    "\n",
+    "```bash\n",
+    "curl -fsSL nebula-up.siwei.io/install.sh | bash\n",
+    "```\n",
+    "\n",
+    "Other options are:\n",
+    "- Install as a [Docker Desktop Extension](https://www.docker.com/blog/distributed-cloud-native-graph-database-nebulagraph-docker-extension/). See [here](https://docs.nebula-graph.io/3.5.0/2.quick-start/1.quick-start-workflow/)\n",
+    "- NebulaGraph Cloud Service. See [here](https://www.nebula-graph.io/cloud)\n",
+    "- Deploy from package, source code, or via Kubernetes. See [here](https://docs.nebula-graph.io/)\n",
+    "\n",
+    "Once the cluster is running, we could create the SPACE and SCHEMA for the database."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c82f4141",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install ipython-ngql\n",
+    "%load_ext ngql\n",
+    "\n",
+    "# connect ngql jupyter extension to nebulagraph\n",
+    "%ngql --address 127.0.0.1 --port 9669 --user root --password nebula\n",
+    "# create a new space\n",
+    "%ngql CREATE SPACE IF NOT EXISTS langchain(partition_num=1, replica_factor=1, vid_type=fixed_string(128));\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "eda0809a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Wait for a few seconds for the space to be created.\n",
+    "%ngql USE langchain;"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "119fe35c",
+   "metadata": {},
+   "source": [
+    "Create the schema, for full dataset, refer [here](https://www.siwei.io/en/nebulagraph-etl-dbt/)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5aa796ee",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%ngql\n",
+    "CREATE TAG IF NOT EXISTS movie(name string);\n",
+    "CREATE TAG IF NOT EXISTS person(name string, birthdate string);\n",
+    "CREATE EDGE IF NOT EXISTS acted_in();\n",
+    "CREATE TAG INDEX IF NOT EXISTS person_index ON person(name(128));\n",
+    "CREATE TAG INDEX IF NOT EXISTS movie_index ON movie(name(128));"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "66e4799a",
+   "metadata": {},
+   "source": [
+    "Wait for schema creation to complete, then we can insert some data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "d8eea530",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "UsageError: Cell magic `%%ngql` not found.\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%ngql\n",
+    "INSERT VERTEX person(name, birthdate) VALUES \"Al Pacino\":(\"Al Pacino\", \"1940-04-25\");\n",
+    "INSERT VERTEX movie(name) VALUES \"The Godfather II\":(\"The Godfather II\");\n",
+    "INSERT VERTEX movie(name) VALUES \"The Godfather Coda: The Death of Michael Corleone\":(\"The Godfather Coda: The Death of Michael Corleone\");\n",
+    "INSERT EDGE acted_in() VALUES \"Al Pacino\"->\"The Godfather II\":();\n",
+    "INSERT EDGE acted_in() VALUES \"Al Pacino\"->\"The Godfather Coda: The Death of Michael Corleone\":();"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "62812aad",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.chat_models import ChatOpenAI\n",
+    "from langchain.chains import NebulaGraphQAChain\n",
+    "from langchain.graphs import NebulaGraph"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "0928915d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "graph = NebulaGraph(\n",
+    "    space=\"langchain\",\n",
+    "    username=\"root\",\n",
+    "    password=\"nebula\",\n",
+    "    address=\"127.0.0.1\",\n",
+    "    port=9669,\n",
+    "    session_pool_size=30,\n",
+    ")"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "58c1a8ea",
+   "metadata": {},
+   "source": [
+    "## Refresh graph schema information\n",
+    "\n",
+    "If the schema of database changes, you can refresh the schema information needed to generate nGQL statements."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4e3de44f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# graph.refresh_schema()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "1fe76ccd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Node properties: [{'tag': 'movie', 'properties': [('name', 'string')]}, {'tag': 'person', 'properties': [('name', 'string'), ('birthdate', 'string')]}]\n",
+      "Edge properties: [{'edge': 'acted_in', 'properties': []}]\n",
+      "Relationships: ['(:person)-[:acted_in]->(:movie)']\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(graph.get_schema)"
+   ]
+  },
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "id": "68a3c677",
+   "metadata": {},
+   "source": [
+    "## Querying the graph\n",
+    "\n",
+    "We can now use the graph cypher QA chain to ask question of the graph"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "7476ce98",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "chain = NebulaGraphQAChain.from_llm(\n",
+    "    ChatOpenAI(temperature=0), graph=graph, verbose=True\n",
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "ef8ee27b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\u001b[1m> Entering new NebulaGraphQAChain chain...\u001b[0m\n",
+      "Generated nGQL:\n",
+      "\u001b[32;1m\u001b[1;3mMATCH (p:`person`)-[:acted_in]->(m:`movie`) WHERE m.`movie`.`name` == 'The Godfather II'\n",
+      "RETURN p.`person`.`name`\u001b[0m\n",
+      "Full Context:\n",
+      "\u001b[32;1m\u001b[1;3m{'p.person.name': ['Al Pacino']}\u001b[0m\n",
+      "\n",
+      "\u001b[1m> Finished chain.\u001b[0m\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'Al Pacino played in The Godfather II.'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "chain.run(\"Who played in The Godfather II?\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}