github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-06-02 04:58:46 +00:00
Code Issues Packages Projects Releases Wiki Activity
1b0bfa54cf
langchain/docs/extras/integrations/providers
History
Jiayi Ni 1efb9bae5f
FEAT: Integrate Xinference LLMs and Embeddings (#8171) 
- [Xorbits
Inference(Xinference)](https://github.com/xorbitsai/inference) is a
powerful and versatile library designed to serve language, speech
recognition, and multimodal models. Xinference supports a variety of
GGML-compatible models including chatglm, whisper, and vicuna, and
utilizes heterogeneous hardware and a distributed architecture for
seamless cross-device and cross-server model deployment.
- This PR integrates Xinference models and Xinference embeddings into
LangChain.
- Dependencies: To install the depenedencies for this integration, run
    
    `pip install "xinference[all]"`
    
- Example Usage:

To start a local instance of Xinference, run `xinference`.

To deploy Xinference in a distributed cluster, first start an Xinference
supervisor using `xinference-supervisor`:

`xinference-supervisor -H "${supervisor_host}"`

Then, start the Xinference workers using `xinference-worker` on each
server you want to run them on.

`xinference-worker -e "http://${supervisor_host}:9997"`

To use Xinference with LangChain, you also need to launch a model. You
can use command line interface (CLI) to do so. Fo example: `xinference
launch -n vicuna-v1.3 -f ggmlv3 -q q4_0`. This launches a model named
vicuna-v1.3 with `model_format="ggmlv3"` and `quantization="q4_0"`. A
model UID is returned for you to use.

Now you can use Xinference with LangChain:

```python
from langchain.llms import Xinference

llm = Xinference(
    server_url="http://0.0.0.0:9997", # suppose the supervisor_host is "0.0.0.0"
    model_uid = {model_uid} # model UID returned from launching a model
)

llm(
    prompt="Q: where can we visit in the capital of France? A:",
    generate_config={"max_tokens": 1024},
)
```

You can also use RESTful client to launch a model:
```python
from xinference.client import RESTfulClient

client = RESTfulClient("http://0.0.0.0:9997")

model_uid = client.launch_model(model_name="vicuna-v1.3", model_size_in_billions=7, quantization="q4_0")
```

The following code block demonstrates how to use Xinference embeddings
with LangChain:
```python
from langchain.embeddings import XinferenceEmbeddings

xinference = XinferenceEmbeddings(
    server_url="http://0.0.0.0:9997",
    model_uid = model_uid
)
```

```python
query_result = xinference.embed_query("This is a test query")
```

```python
doc_result = xinference.embed_documents(["text A", "text B"])
```

Xinference is still under rapid development. Feel free to [join our
Slack
community](https://xorbitsio.slack.com/join/shared_invite/zt-1z3zsm9ep-87yI9YZ_B79HLB2ccTq4WA)
to get the latest updates!

- Request for review: @hwchase17, @baskaryan
- Twitter handle: https://twitter.com/Xorbitsio

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-07-27 21:23:19 -07:00
..
portkey mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
vectara mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
agent_with_wandb_tracing.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
ai21.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
aim_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
airbyte.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
airtable.md Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
aleph_alpha.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
alibabacloud_opensearch.md mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
amazon_api_gateway.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
analyticdb.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
annoy.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
anyscale.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
apify.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
arangodb.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
argilla.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
arthur_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
arxiv.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
atlas.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
awadb.md Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
aws_s3.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
azlyrics.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
azure_blob_storage.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
azure_cognitive_search_.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
azure_openai.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
bananadev.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
baseten.md mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
beam.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
bedrock.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
bilibili.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
blackboard.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
brave_search.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
cassandra.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
cerebriumai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
chaindesk.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
chroma.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
clarifai.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
clearml_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
cnosdb.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
cohere.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
college_confidential.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
comet_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
confluence.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
ctransformers.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
databricks.ipynb mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
databricks.md mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
datadog_logs.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
datadog.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
dataforseo.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
deepinfra.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
deeplake.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
diffbot.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
discord.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
docugami.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
duckdb.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
elasticsearch.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
evernote.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
facebook_chat.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
figma.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
flyte.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
forefrontai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
git.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
gitbook.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
golden.mdx update golden-query notebook and fix typo in golden docs (#8253) 2023-07-25 18:15:48 -07:00
google_bigquery.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
google_cloud_storage.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
google_drive.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
google_search.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
google_serper.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
gooseai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
gpt4all.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
graphsignal.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
grobid.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
gutenberg.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
hacker_news.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
hazy_research.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
helicone.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
hologres.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
huggingface.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
ifixit.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
imsdb.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
index.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
infino.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
jina.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
lancedb.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
langchain_decorators.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
llamacpp.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
marqo.md Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
mediawikidump.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
metal.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
microsoft_onedrive.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
microsoft_powerpoint.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
microsoft_word.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
milvus.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
mlflow_ai_gateway.mdx Bagatur/gateway chat (#8198) 2023-07-24 12:17:00 -07:00
mlflow_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
modal.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
modelscope.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
modern_treasury.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
momento.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
motherduck.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
myscale.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
nlpcloud.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
notion.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
obsidian.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
openai.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
openllm.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
opensearch.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
openweathermap.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
petals.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
pgvector.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
pinecone.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
pipelineai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
predibase.md mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
predictionguard.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
promptlayer.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
psychic.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
qdrant.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
ray_serve.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
rebuff.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
reddit.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
redis.mdx mv popular and additional chains to use cases (#8242) 2023-07-27 12:55:13 -07:00
replicate.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
roam.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
rockset.mdx Docs: Fix Rockset links (#8214) 2023-07-26 10:38:37 -07:00
runhouse.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
rwkv.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
sagemaker_endpoint.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
searx.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
serpapi.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
shaleprotocol.md mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
singlestoredb.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
sklearn.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
slack.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
spacy.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
spreedly.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
starrocks.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
stochasticai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
stripe.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
tair.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
telegram.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
tigris.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
tomarkdown.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
trello.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
trulens.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
twitter.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
typesense.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
unstructured.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
vespa.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
wandb_tracking.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
weather.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
weaviate.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
whatsapp.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
whylabs_profiling.ipynb mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
wikipedia.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
wolfram_alpha.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
writer.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
xinference.mdx FEAT: Integrate Xinference LLMs and Embeddings (#8171) 2023-07-27 21:23:19 -07:00
yeagerai.mdx mv module integrations docs (#8101) 2023-07-23 23:23:16 -07:00
youtube.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
zep.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00
zilliz.mdx Update Integrations links (#8206) 2023-07-24 21:20:32 -07:00