langchain/docs/versioned_docs/version-0.2.x/integrations/llms/yuan2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Yuan2.0\n",
    "\n",
    "[Yuan2.0](https://github.com/IEIT-Yuan/Yuan-2.0) is a new generation Fundamental Large Language Model developed by IEIT System. We have published all three models, Yuan 2.0-102B, Yuan 2.0-51B, and Yuan 2.0-2B. And we provide relevant scripts for pretraining, fine-tuning, and inference services for other developers. Yuan2.0 is based on Yuan1.0, utilizing a wider range of high-quality pre training data and instruction fine-tuning datasets to enhance the model's understanding of semantics, mathematics, reasoning, code, knowledge, and other aspects.\n",
    "\n",
    "This example goes over how to use LangChain to interact with `Yuan2.0`(2B/51B/102B) Inference for text generation.\n",
    "\n",
    "Yuan2.0 set up an inference service so user just need request the inference api to get result, which is introduced in [Yuan2.0 Inference-Server](https://github.com/IEIT-Yuan/Yuan-2.0/blob/main/docs/inference_server.md)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from langchain.chains import LLMChain\n",
    "from langchain_community.llms.yuan2 import Yuan2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# default infer_api for a local deployed Yuan2.0 inference server\n",
    "infer_api = \"http://127.0.0.1:8000/yuan\"\n",
    "\n",
    "# direct access endpoint in a proxied environment\n",
    "# import os\n",
    "# os.environ[\"no_proxy\"]=\"localhost,127.0.0.1,::1\"\n",
    "\n",
    "yuan_llm = Yuan2(\n",
    "    infer_api=infer_api,\n",
    "    max_tokens=2048,\n",
    "    temp=1.0,\n",
    "    top_p=0.9,\n",
    "    use_history=False,\n",
    ")\n",
    "\n",
    "# turn on use_history only when you want the Yuan2.0 to keep track of the conversation history\n",
    "# and send the accumulated context to the backend model api, which make it stateful. By default it is stateless.\n",
    "# llm.use_history = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "question = \"请介绍一下中国。\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "print(yuan_llm.invoke(question))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain-dev",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}