mirror of
https://github.com/hwchase17/langchain.git
synced 2026-04-03 19:04:23 +00:00
Use docusaurus versioning with a callout, merged master as well @hwchase17 @baskaryan --------- Signed-off-by: Weichen Xu <weichen.xu@databricks.com> Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Leonid Ganeline <leo.gan.57@gmail.com> Co-authored-by: Leonid Kuligin <lkuligin@yandex.ru> Co-authored-by: Averi Kitsch <akitsch@google.com> Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Nuno Campos <nuno@langchain.dev> Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com> Co-authored-by: Martín Gotelli Ferenaz <martingotelliferenaz@gmail.com> Co-authored-by: Fayfox <admin@fayfox.com> Co-authored-by: Eugene Yurtsev <eugene@langchain.dev> Co-authored-by: Dawson Bauer <105886620+djbauer2@users.noreply.github.com> Co-authored-by: Ravindu Somawansa <ravindu.somawansa@gmail.com> Co-authored-by: Dhruv Chawla <43818888+Dominastorm@users.noreply.github.com> Co-authored-by: ccurme <chester.curme@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: WeichenXu <weichen.xu@databricks.com> Co-authored-by: Benito Geordie <89472452+benitoThree@users.noreply.github.com> Co-authored-by: kartikTAI <129414343+kartikTAI@users.noreply.github.com> Co-authored-by: Kartik Sarangmath <kartik@thirdai.com> Co-authored-by: Sevin F. Varoglu <sfvaroglu@octoml.ai> Co-authored-by: MacanPN <martin.triska@gmail.com> Co-authored-by: Prashanth Rao <35005448+prrao87@users.noreply.github.com> Co-authored-by: Hyeongchan Kim <kozistr@gmail.com> Co-authored-by: sdan <git@sdan.io> Co-authored-by: Guangdong Liu <liugddx@gmail.com> Co-authored-by: Rahul Triptahi <rahul.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: pjb157 <84070455+pjb157@users.noreply.github.com> Co-authored-by: Eun Hye Kim <ehkim1440@gmail.com> Co-authored-by: kaijietti <43436010+kaijietti@users.noreply.github.com> Co-authored-by: Pengcheng Liu <pcliu.fd@gmail.com> Co-authored-by: Tomer Cagan <tomer@tomercagan.com> Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
162 lines
4.6 KiB
Plaintext
162 lines
4.6 KiB
Plaintext
# Hugging Face
|
|
|
|
All functionality related to the [Hugging Face Platform](https://huggingface.co/).
|
|
|
|
## Chat models
|
|
|
|
### Models from Hugging Face
|
|
|
|
We can use the `Hugging Face` LLM classes or directly use the `ChatHuggingFace` class.
|
|
|
|
We need to install several python packages.
|
|
|
|
```bash
|
|
pip install huggingface_hub
|
|
pip install transformers
|
|
```
|
|
See a [usage example](/docs/integrations/chat/huggingface).
|
|
|
|
```python
|
|
from langchain_community.chat_models.huggingface import ChatHuggingFace
|
|
```
|
|
|
|
## LLMs
|
|
|
|
### Hugging Face Local Pipelines
|
|
|
|
Hugging Face models can be run locally through the `HuggingFacePipeline` class.
|
|
|
|
We need to install `transformers` python package.
|
|
|
|
```bash
|
|
pip install transformers
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/llms/huggingface_pipelines).
|
|
|
|
```python
|
|
from langchain_community.llms.huggingface_pipeline import HuggingFacePipeline
|
|
```
|
|
|
|
To use the OpenVINO backend in local pipeline wrapper, please install the optimum library and set HuggingFacePipeline's backend as `openvino`:
|
|
|
|
```bash
|
|
pip install --upgrade-strategy eager "optimum[openvino,nncf]"
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/llms/huggingface_pipelines)
|
|
|
|
To export your model to the OpenVINO IR format with the CLI:
|
|
|
|
```bash
|
|
optimum-cli export openvino --model gpt2 ov_model
|
|
```
|
|
|
|
To apply [weight-only quantization](https://github.com/huggingface/optimum-intel?tab=readme-ov-file#export) when exporting your model.
|
|
|
|
|
|
## Embedding Models
|
|
|
|
### Hugging Face Hub
|
|
|
|
>The [Hugging Face Hub](https://huggingface.co/docs/hub/index) is a platform
|
|
> with over 350k models, 75k datasets, and 150k demo apps (Spaces), all open source
|
|
> and publicly available, in an online platform where people can easily
|
|
> collaborate and build ML together. The Hub works as a central place where anyone
|
|
> can explore, experiment, collaborate, and build technology with Machine Learning.
|
|
|
|
We need to install the `sentence_transformers` python package.
|
|
|
|
```bash
|
|
pip install sentence_transformers
|
|
```
|
|
|
|
|
|
#### HuggingFaceEmbeddings
|
|
|
|
See a [usage example](/docs/integrations/text_embedding/huggingfacehub).
|
|
|
|
```python
|
|
from langchain_community.embeddings import HuggingFaceEmbeddings
|
|
```
|
|
#### HuggingFaceInstructEmbeddings
|
|
|
|
See a [usage example](/docs/integrations/text_embedding/instruct_embeddings).
|
|
|
|
```python
|
|
from langchain_community.embeddings import HuggingFaceInstructEmbeddings
|
|
```
|
|
|
|
#### HuggingFaceBgeEmbeddings
|
|
|
|
>[BGE models on the HuggingFace](https://huggingface.co/BAAI/bge-large-en) are [the best open-source embedding models](https://huggingface.co/spaces/mteb/leaderboard).
|
|
>BGE model is created by the [Beijing Academy of Artificial Intelligence (BAAI)](https://en.wikipedia.org/wiki/Beijing_Academy_of_Artificial_Intelligence). `BAAI` is a private non-profit organization engaged in AI research and development.
|
|
|
|
See a [usage example](/docs/integrations/text_embedding/bge_huggingface).
|
|
|
|
```python
|
|
from langchain_community.embeddings import HuggingFaceBgeEmbeddings
|
|
```
|
|
|
|
### Hugging Face Text Embeddings Inference (TEI)
|
|
|
|
>[Hugging Face Text Embeddings Inference (TEI)](https://huggingface.co/docs/text-generation-inference/index) is a toolkit for deploying and serving open-source
|
|
> text embeddings and sequence classification models. `TEI` enables high-performance extraction for the most popular models,
|
|
>including `FlagEmbedding`, `Ember`, `GTE` and `E5`.
|
|
|
|
We need to install `huggingface-hub` python package.
|
|
|
|
```bash
|
|
pip install huggingface-hub
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/text_embedding/text_embeddings_inference).
|
|
|
|
```python
|
|
from langchain_community.embeddings import HuggingFaceHubEmbeddings
|
|
```
|
|
|
|
|
|
## Document Loaders
|
|
|
|
### Hugging Face dataset
|
|
|
|
>[Hugging Face Hub](https://huggingface.co/docs/hub/index) is home to over 75,000
|
|
> [datasets](https://huggingface.co/docs/hub/index#datasets) in more than 100 languages
|
|
> that can be used for a broad range of tasks across NLP, Computer Vision, and Audio.
|
|
> They used for a diverse range of tasks such as translation, automatic speech
|
|
> recognition, and image classification.
|
|
|
|
We need to install `datasets` python package.
|
|
|
|
```bash
|
|
pip install datasets
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/document_loaders/hugging_face_dataset).
|
|
|
|
```python
|
|
from langchain_community.document_loaders.hugging_face_dataset import HuggingFaceDatasetLoader
|
|
```
|
|
|
|
|
|
|
|
## Tools
|
|
|
|
### Hugging Face Hub Tools
|
|
|
|
>[Hugging Face Tools](https://huggingface.co/docs/transformers/v4.29.0/en/custom_tools)
|
|
> support text I/O and are loaded using the `load_huggingface_tool` function.
|
|
|
|
We need to install several python packages.
|
|
|
|
```bash
|
|
pip install transformers huggingface_hub
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/tools/huggingface_tools).
|
|
|
|
```python
|
|
from langchain.agents import load_huggingface_tool
|
|
```
|