mirror of
https://github.com/hwchase17/langchain.git
synced 2026-04-03 10:55:08 +00:00
Use docusaurus versioning with a callout, merged master as well @hwchase17 @baskaryan --------- Signed-off-by: Weichen Xu <weichen.xu@databricks.com> Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Leonid Ganeline <leo.gan.57@gmail.com> Co-authored-by: Leonid Kuligin <lkuligin@yandex.ru> Co-authored-by: Averi Kitsch <akitsch@google.com> Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Nuno Campos <nuno@langchain.dev> Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com> Co-authored-by: Martín Gotelli Ferenaz <martingotelliferenaz@gmail.com> Co-authored-by: Fayfox <admin@fayfox.com> Co-authored-by: Eugene Yurtsev <eugene@langchain.dev> Co-authored-by: Dawson Bauer <105886620+djbauer2@users.noreply.github.com> Co-authored-by: Ravindu Somawansa <ravindu.somawansa@gmail.com> Co-authored-by: Dhruv Chawla <43818888+Dominastorm@users.noreply.github.com> Co-authored-by: ccurme <chester.curme@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: WeichenXu <weichen.xu@databricks.com> Co-authored-by: Benito Geordie <89472452+benitoThree@users.noreply.github.com> Co-authored-by: kartikTAI <129414343+kartikTAI@users.noreply.github.com> Co-authored-by: Kartik Sarangmath <kartik@thirdai.com> Co-authored-by: Sevin F. Varoglu <sfvaroglu@octoml.ai> Co-authored-by: MacanPN <martin.triska@gmail.com> Co-authored-by: Prashanth Rao <35005448+prrao87@users.noreply.github.com> Co-authored-by: Hyeongchan Kim <kozistr@gmail.com> Co-authored-by: sdan <git@sdan.io> Co-authored-by: Guangdong Liu <liugddx@gmail.com> Co-authored-by: Rahul Triptahi <rahul.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: pjb157 <84070455+pjb157@users.noreply.github.com> Co-authored-by: Eun Hye Kim <ehkim1440@gmail.com> Co-authored-by: kaijietti <43436010+kaijietti@users.noreply.github.com> Co-authored-by: Pengcheng Liu <pcliu.fd@gmail.com> Co-authored-by: Tomer Cagan <tomer@tomercagan.com> Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
58 lines
1.7 KiB
Plaintext
58 lines
1.7 KiB
Plaintext
# C Transformers
|
|
|
|
This page covers how to use the [C Transformers](https://github.com/marella/ctransformers) library within LangChain.
|
|
It is broken into two parts: installation and setup, and then references to specific C Transformers wrappers.
|
|
|
|
## Installation and Setup
|
|
|
|
- Install the Python package with `pip install ctransformers`
|
|
- Download a supported [GGML model](https://huggingface.co/TheBloke) (see [Supported Models](https://github.com/marella/ctransformers#supported-models))
|
|
|
|
## Wrappers
|
|
|
|
### LLM
|
|
|
|
There exists a CTransformers LLM wrapper, which you can access with:
|
|
|
|
```python
|
|
from langchain_community.llms import CTransformers
|
|
```
|
|
|
|
It provides a unified interface for all models:
|
|
|
|
```python
|
|
llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2')
|
|
|
|
print(llm('AI is going to'))
|
|
```
|
|
|
|
If you are getting `illegal instruction` error, try using `lib='avx'` or `lib='basic'`:
|
|
|
|
```py
|
|
llm = CTransformers(model='/path/to/ggml-gpt-2.bin', model_type='gpt2', lib='avx')
|
|
```
|
|
|
|
It can be used with models hosted on the Hugging Face Hub:
|
|
|
|
```py
|
|
llm = CTransformers(model='marella/gpt-2-ggml')
|
|
```
|
|
|
|
If a model repo has multiple model files (`.bin` files), specify a model file using:
|
|
|
|
```py
|
|
llm = CTransformers(model='marella/gpt-2-ggml', model_file='ggml-model.bin')
|
|
```
|
|
|
|
Additional parameters can be passed using the `config` parameter:
|
|
|
|
```py
|
|
config = {'max_new_tokens': 256, 'repetition_penalty': 1.1}
|
|
|
|
llm = CTransformers(model='marella/gpt-2-ggml', config=config)
|
|
```
|
|
|
|
See [Documentation](https://github.com/marella/ctransformers#config) for a list of available parameters.
|
|
|
|
For a more detailed walkthrough of this, see [this notebook](/docs/integrations/llms/ctransformers).
|