github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2026-01-06 08:28:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add OpenLLM wrapper(#6578)

Browse Source 
LLM wrapper for models served with OpenLLM

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
Authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>
Co-authored-by: Chaoyu <paranoyang@gmail.com>
This commit is contained in:
Davis Chase
2023-06-22 01:18:14 -07:00
committed by GitHub
parent d718f3b6d0
commit 4fabd02d25
9 changed files with 1227 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/extras/guides/deployments/index.mdx
View File

@@ -21,7 +21,8 @@ This guide aims to provide a comprehensive overview of the requirements for depl
Understanding these components is crucial when assessing serving systems. LangChain integrates with several open-source projects designed to tackle these issues, providing a robust framework for productionizing your LLM applications. Some notable frameworks include:
- [Ray Serve](/docs/ecosystem/integrations/ray_serve.html)
- [BentoML](https://github.com/ssheng/BentoChain)
- [BentoML](https://github.com/bentoml/BentoML)
- [OpenLLM](/docs/ecosystem/integrations/openllm.html)
- [Modal](/docs/ecosystem/integrations/modal.html)
These links will provide further information on each ecosystem, assisting you in finding the best fit for your LLM deployment needs.
@@ -110,4 +111,4 @@ Rapid iteration also involves the ability to recreate your infrastructure quickl
## CI/CD
In a fast-paced environment, implementing CI/CD pipelines can significantly speed up the iteration process. They help automate the testing and deployment of your LLM applications, reducing the risk of errors and enabling faster feedback and iteration.
In a fast-paced environment, implementing CI/CD pipelines can significantly speed up the iteration process. They help automate the testing and deployment of your LLM applications, reducing the risk of errors and enabling faster feedback and iteration.

5
docs/extras/guides/deployments/template_repos.mdx
View File

@@ -67,6 +67,11 @@ This repository allows users to serve local chains and agents as RESTful, gRPC,
This repository provides an example of how to deploy a LangChain application with [BentoML](https://github.com/bentoml/BentoML). BentoML is a framework that enables the containerization of machine learning applications as standard OCI images. BentoML also allows for the automatic generation of OpenAPI and gRPC endpoints. With BentoML, you can integrate models from all popular ML frameworks and deploy them as microservices running on the most optimal hardware and scaling independently.
## [OpenLLM](https://github.com/bentoml/OpenLLM)
OpenLLM is a platform for operating large language models (LLMs) in production. With OpenLLM, you can run inference with any open-source LLM, deploy to the cloud or on-premises, and build powerful AI apps. It supports a wide range of open-source LLMs, offers flexible APIs, and first-class support for LangChain and BentoML.
See OpenLLM's [integration doc](https://github.com/bentoml/OpenLLM#%EF%B8%8F-integrations) for usage with LangChain.
## [Databutton](https://databutton.com/home?new-data-app=true)
These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include a Chatbot interface with conversational memory, a Personal search engine, and a starter template for LangChain apps. Deploying and sharing is just one click away.