mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-06 13:33:37 +00:00
Add Baidu Qianfan endpoint for LLM (#10496)
- Description： * Baidu AI Cloud's [Qianfan Platform](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) is an all-in-one platform for large model development and service deployment, catering to enterprise developers in China. Qianfan Platform offers a wide range of resources, including the Wenxin Yiyan model (ERNIE-Bot) and various third-party open-source models. - Issue: none - Dependencies: * qianfan - Tag maintainer: @baskaryan - Twitter handle: --------- Co-authored-by: Bagatur <baskaryan@gmail.com>
This commit is contained in:
stonekim
GitHub
181
docs/extras/integrations/chat/baidu_qianfan_endpoint.ipynb
Normal file
181
docs/extras/integrations/chat/baidu_qianfan_endpoint.ipynb
Normal file
@@ -0,0 +1,181 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Baidu Qianfan\n",
|
||||
"\n",
|
||||
"Baidu AI Cloud Qianfan Platform is a one-stop large model development and service operation platform for enterprise developers. Qianfan not only provides including the model of Wenxin Yiyan (ERNIE-Bot) and the third-party open source models, but also provides various AI development tools and the whole set of development environment, which facilitates customers to use and develop large model applications easily.\n",
|
||||
"\n",
|
||||
"Basically, those model are split into the following type:\n",
|
||||
"\n",
|
||||
"- Embedding\n",
|
||||
"- Chat\n",
|
||||
"- Completion\n",
|
||||
"\n",
|
||||
"In this notebook, we will introduce how to use langchain with [Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) mainly in `Chat` corresponding\n",
|
||||
" to the package `langchain/chat_models` in langchain:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## API Initialization\n",
|
||||
"\n",
|
||||
"To use the LLM services based on Baidu Qianfan, you have to initialize these parameters:\n",
|
||||
"\n",
|
||||
"You could either choose to init the AK,SK in enviroment variables or init params:\n",
|
||||
"\n",
|
||||
"```base\n",
|
||||
"export QIANFAN_AK=XXX\n",
|
||||
"export QIANFAN_SK=XXX\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"## Current supported models:\n",
|
||||
"\n",
|
||||
"- ERNIE-Bot-turbo (default models)\n",
|
||||
"- ERNIE-Bot\n",
|
||||
"- BLOOMZ-7B\n",
|
||||
"- Llama-2-7b-chat\n",
|
||||
"- Llama-2-13b-chat\n",
|
||||
"- Llama-2-70b-chat\n",
|
||||
"- Qianfan-BLOOMZ-7B-compressed\n",
|
||||
"- Qianfan-Chinese-Llama-2-7B\n",
|
||||
"- ChatGLM2-6B-32K\n",
|
||||
"- AquilaChat-7B"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\"\"\"For basic init and call\"\"\"\n",
|
||||
"from langchain.chat_models.baidu_qianfan_endpoint import QianfanChatEndpoint \n",
|
||||
"from langchain.chat_models.base import HumanMessage\n",
|
||||
"import os\n",
|
||||
"os.environ[\"QIAFAN_AK\"] = \"xxx\"\n",
|
||||
"os.environ[\"QIAFAN_AK\"] = \"xxx\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"chat = QianfanChatEndpoint(\n",
|
||||
" qianfan_ak=\"xxx\",\n",
|
||||
" qianfan_sk=\"xxx\",\n",
|
||||
" streaming=True, \n",
|
||||
" )\n",
|
||||
"res = chat([HumanMessage(content=\"write a funny joke\")])\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
" \n",
|
||||
"from langchain.chat_models.baidu_qianfan_endpoint import QianfanChatEndpoint\n",
|
||||
"from langchain.schema import HumanMessage\n",
|
||||
"import asyncio\n",
|
||||
"\n",
|
||||
"chatLLM = QianfanChatEndpoint(\n",
|
||||
" streaming=True,\n",
|
||||
")\n",
|
||||
"res = chatLLM.stream([HumanMessage(content=\"hi\")], streaming=True)\n",
|
||||
"for r in res:\n",
|
||||
" print(\"chat resp1:\", r)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"async def run_aio_generate():\n",
|
||||
" resp = await chatLLM.agenerate(messages=[[HumanMessage(content=\"write a 20 words sentence about sea.\")]])\n",
|
||||
" print(resp)\n",
|
||||
" \n",
|
||||
"await run_aio_generate()\n",
|
||||
"\n",
|
||||
"async def run_aio_stream():\n",
|
||||
" async for res in chatLLM.astream([HumanMessage(content=\"write a 20 words sentence about sea.\")]):\n",
|
||||
" print(\"astream\", res)\n",
|
||||
" \n",
|
||||
"await run_aio_stream()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Use different models in Qianfan\n",
|
||||
"\n",
|
||||
"In the case you want to deploy your own model based on Ernie Bot or third-party open sources model, you could follow these steps:\n",
|
||||
"\n",
|
||||
"- 1. （Optional, if the model are included in the default models, skip it）Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
|
||||
"- 2. Set up the field called `endpoint` in the initlization:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"chatBloom = QianfanChatEndpoint(\n",
|
||||
" streaming=True, \n",
|
||||
" model=\"BLOOMZ-7B\",\n",
|
||||
" )\n",
|
||||
"res = chatBloom([HumanMessage(content=\"hi\")])\n",
|
||||
"print(res)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Model Params:\n",
|
||||
"\n",
|
||||
"For now, only `ERNIE-Bot` and `ERNIE-Bot-turbo` support model params below, we might support more models in the future.\n",
|
||||
"\n",
|
||||
"- temperature\n",
|
||||
"- top_p\n",
|
||||
"- penalty_score\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"res = chat.stream([HumanMessage(content=\"hi\")], **{'top_p': 0.4, 'temperature': 0.1, 'penalty_score': 1})\n",
|
||||
"\n",
|
||||
"for r in res:\n",
|
||||
" print(r)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.2"
|
||||
},
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "2d8226dd90b7dc6e8932aea372a8bf9fc71abac4be3cdd5a63a36c2a19e3700f"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
177
docs/extras/integrations/llms/baidu_qianfan_endpoint.ipynb
Normal file
177
docs/extras/integrations/llms/baidu_qianfan_endpoint.ipynb
Normal file
@@ -0,0 +1,177 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Baidu Qianfan\n",
|
||||
"\n",
|
||||
"Baidu AI Cloud Qianfan Platform is a one-stop large model development and service operation platform for enterprise developers. Qianfan not only provides including the model of Wenxin Yiyan (ERNIE-Bot) and the third-party open source models, but also provides various AI development tools and the whole set of development environment, which facilitates customers to use and develop large model applications easily.\n",
|
||||
"\n",
|
||||
"Basically, those model are split into the following type:\n",
|
||||
"\n",
|
||||
"- Embedding\n",
|
||||
"- Chat\n",
|
||||
"- Coompletion\n",
|
||||
"\n",
|
||||
"In this notebook, we will introduce how to use langchain with [Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) mainly in `Completion` corresponding\n",
|
||||
" to the package `langchain/llms` in langchain:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## API Initialization\n",
|
||||
"\n",
|
||||
"To use the LLM services based on Baidu Qianfan, you have to initialize these parameters:\n",
|
||||
"\n",
|
||||
"You could either choose to init the AK,SK in enviroment variables or init params:\n",
|
||||
"\n",
|
||||
"```base\n",
|
||||
"export QIANFAN_AK=XXX\n",
|
||||
"export QIANFAN_SK=XXX\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"## Current supported models:\n",
|
||||
"\n",
|
||||
"- ERNIE-Bot-turbo (default models)\n",
|
||||
"- ERNIE-Bot\n",
|
||||
"- BLOOMZ-7B\n",
|
||||
"- Llama-2-7b-chat\n",
|
||||
"- Llama-2-13b-chat\n",
|
||||
"- Llama-2-70b-chat\n",
|
||||
"- Qianfan-BLOOMZ-7B-compressed\n",
|
||||
"- Qianfan-Chinese-Llama-2-7B\n",
|
||||
"- ChatGLM2-6B-32K\n",
|
||||
"- AquilaChat-7B"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\"\"\"For basic init and call\"\"\"\n",
|
||||
"from langchain.llms.baidu_qianfan_endpoint import QianfanLLMEndpoint\n",
|
||||
"\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"os.environ[\"QIANFAN_AK\"] = \"xx\"\n",
|
||||
"os.environ[\"QIANFAN_SK\"] = \"xx\"\n",
|
||||
"\n",
|
||||
"llm = QianfanLLMEndpoint(streaming=True, ak=\"xx\", sk=\"xx\")\n",
|
||||
"res = llm(\"hi\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\n",
|
||||
"\"\"\"Test for llm generate \"\"\"\n",
|
||||
"res = llm.generate(prompts=[\"hillo?\"])\n",
|
||||
"import asyncio\n",
|
||||
"\"\"\"Test for llm aio generate\"\"\"\n",
|
||||
"async def run_aio_generate():\n",
|
||||
" resp = await llm.agenerate(prompts=[\"Write a 20-word article about rivers.\"])\n",
|
||||
" print(resp)\n",
|
||||
"\n",
|
||||
"await run_aio_generate()\n",
|
||||
"\n",
|
||||
"\"\"\"Test for llm stream\"\"\"\n",
|
||||
"for res in llm.stream(\"write a joke.\"):\n",
|
||||
" print(res)\n",
|
||||
"\n",
|
||||
"\"\"\"Test for llm aio stream\"\"\"\n",
|
||||
"async def run_aio_stream():\n",
|
||||
" async for res in llm.astream(\"Write a 20-word article about mountains\"):\n",
|
||||
" print(res)\n",
|
||||
"\n",
|
||||
"await run_aio_stream()\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Use different models in Qianfan\n",
|
||||
"\n",
|
||||
"In the case you want to deploy your own model based on EB or serval open sources model, you could follow these steps:\n",
|
||||
"\n",
|
||||
"- 1. （Optional, if the model are included in the default models, skip it）Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
|
||||
"- 2. Set up the field called `endpoint` in the initlization:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"llm = QianfanLLMEndpoint(qianfan_ak='xxx', \n",
|
||||
" qianfan_sk='xxx', \n",
|
||||
" streaming=True, \n",
|
||||
" model=\"ERNIE-Bot-turbo\",\n",
|
||||
" endpoint=\"eb-instant\",\n",
|
||||
" )\n",
|
||||
"res = llm(\"hi\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Model Params:\n",
|
||||
"\n",
|
||||
"For now, only `ERNIE-Bot` and `ERNIE-Bot-turbo` support model params below, we might support more models in the future.\n",
|
||||
"\n",
|
||||
"- temperature\n",
|
||||
"- top_p\n",
|
||||
"- penalty_score\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"res = llm.generate(prompts=[\"hi\"], streaming=True, **{'top_p': 0.4, 'temperature': 0.1, 'penalty_score': 1})\n",
|
||||
"\n",
|
||||
"for r in res:\n",
|
||||
" print(r)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "base",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "6fa70026b407ae751a5c9e6bd7f7d482379da8ad616f98512780b705c84ee157"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Baidu Qianfan\n",
|
||||
"\n",
|
||||
"Baidu AI Cloud Qianfan Platform is a one-stop large model development and service operation platform for enterprise developers. Qianfan not only provides including the model of Wenxin Yiyan (ERNIE-Bot) and the third-party open source models, but also provides various AI development tools and the whole set of development environment, which facilitates customers to use and develop large model applications easily.\n",
|
||||
"\n",
|
||||
"Basically, those model are split into the following type:\n",
|
||||
"\n",
|
||||
"- Embedding\n",
|
||||
"- Chat\n",
|
||||
"- Completion\n",
|
||||
"\n",
|
||||
"In this notebook, we will introduce how to use langchain with [Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) mainly in `Embedding` corresponding\n",
|
||||
" to the package `langchain/embeddings` in langchain:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## API Initialization\n",
|
||||
"\n",
|
||||
"To use the LLM services based on Baidu Qianfan, you have to initialize these parameters:\n",
|
||||
"\n",
|
||||
"You could either choose to init the AK,SK in enviroment variables or init params:\n",
|
||||
"\n",
|
||||
"```base\n",
|
||||
"export QIANFAN_AK=XXX\n",
|
||||
"export QIANFAN_SK=XXX\n",
|
||||
"```\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"\"\"\"For basic init and call\"\"\"\n",
|
||||
"from langchain.embeddings.baidu_qianfan_endpoint import QianfanEmbeddingsEndpoint \n",
|
||||
"\n",
|
||||
"import os\n",
|
||||
"os.environ[\"QIANFAN_AK\"] = \"xx\"\n",
|
||||
"os.environ[\"QIANFAN_SK\"] = \"xx\"\n",
|
||||
"\n",
|
||||
"embed = QianfanEmbeddingsEndpoint(qianfan_ak='xxx', \n",
|
||||
" qianfan_sk='xxx')\n",
|
||||
"res = embed.embed_documents([\"hi\", \"world\"])\n",
|
||||
"\n",
|
||||
"import asyncio\n",
|
||||
"\n",
|
||||
"async def aioEmbed():\n",
|
||||
" res = await embed.aembed_query(\"qianfan\")\n",
|
||||
" print(res)\n",
|
||||
"await aioEmbed()\n",
|
||||
"\n",
|
||||
"import asyncio\n",
|
||||
"async def aioEmbedDocs():\n",
|
||||
" res = await embed.aembed_documents([\"hi\", \"world\"])\n",
|
||||
" for r in res:\n",
|
||||
" print(\"\", r[:8])\n",
|
||||
"await aioEmbedDocs()\n",
|
||||
"\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Use different models in Qianfan\n",
|
||||
"\n",
|
||||
"In the case you want to deploy your own model based on Ernie Bot or third-party open sources model, you could follow these steps:\n",
|
||||
"\n",
|
||||
"- 1. （Optional, if the model are included in the default models, skip it）Deploy your model in Qianfan Console, get your own customized deploy endpoint.\n",
|
||||
"- 2. Set up the field called `endpoint` in the initlization:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"embed = QianfanEmbeddingsEndpoint(qianfan_ak='xxx', \n",
|
||||
" qianfan_sk='xxx',\n",
|
||||
" model=\"bge_large_zh\",\n",
|
||||
" endpoint=\"bge_large_zh\")\n",
|
||||
"\n",
|
||||
"res = embed.embed_documents([\"hi\", \"world\"])"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "base",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
"interpreter": {
|
||||
"hash": "6fa70026b407ae751a5c9e6bd7f7d482379da8ad616f98512780b705c84ee157"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in New Issue
Block a user