mirror of
https://github.com/hwchase17/langchain.git
synced 2025-09-04 04:28:58 +00:00
Takeoff pro support (#12070)
**Description:** This PR adds support for the [Pro version of Titan Takeoff Server](https://docs.titanml.co/docs/category/pro-features). Users of the Pro version will have to import the TitanTakeoffPro model, which is different from TitanTakeoff. **Issue:** Also minor fixes to docs for Titan Takeoff (Community version) **Dependencies:** No additional dependencies **Twitter handle:** @becoming_blake @baskaryan @hwchase17
This commit is contained in:
committed by
GitHub
parent
4e47fe1dce
commit
b9410f2b6f
@@ -108,7 +108,7 @@
|
||||
"from langchain.llms import TitanTakeoff\n",
|
||||
"\n",
|
||||
"llm = TitanTakeoff(\n",
|
||||
" baseURL=\"http://localhost:8000\",\n",
|
||||
" base_url=\"http://localhost:8000\",\n",
|
||||
" generate_max_length=128,\n",
|
||||
" temperature=1.0\n",
|
||||
")\n",
|
||||
|
100
docs/docs/integrations/llms/titan_takeoff_pro.ipynb
Normal file
100
docs/docs/integrations/llms/titan_takeoff_pro.ipynb
Normal file
@@ -0,0 +1,100 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Titan Takeoff Pro\n",
|
||||
"\n",
|
||||
"`TitanML` helps businesses build and deploy better, smaller, cheaper, and faster NLP models through our training, compression, and inference optimization platform.\n",
|
||||
"\n",
|
||||
">Note: These docs are for the Pro version of Titan Takeoff. For the community version, see the page for Titan Takeoff.\n",
|
||||
"\n",
|
||||
"Our inference server, [Titan Takeoff (Pro Version)](https://docs.titanml.co/docs/titan-takeoff/pro-features/feature-comparison) enables deployment of LLMs locally on your hardware in a single command. Most generative model architectures are supported, such as Falcon, Llama 2, GPT2, T5 and many more."
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Example usage\n",
|
||||
"Here are some helpful examples to get started using the Pro version of Titan Takeoff Server.\n",
|
||||
"No parameters are needed by default, but a baseURL that points to your desired URL where Takeoff is running can be specified and generation parameters can be supplied."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain.llms import TitanTakeoffPro\n",
|
||||
"from langchain.prompts import PromptTemplate\n",
|
||||
"from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
|
||||
"from langchain.callbacks.manager import CallbackManager\n",
|
||||
"\n",
|
||||
"# Example 1: Basic use\n",
|
||||
"llm = TitanTakeoffPro()\n",
|
||||
"output = llm(\"What is the weather in London in August?\")\n",
|
||||
"print(output)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Example 2: Specifying a port and other generation parameters\n",
|
||||
"llm = TitanTakeoffPro(\n",
|
||||
" base_url=\"http://localhost:3000\",\n",
|
||||
" min_new_tokens=128,\n",
|
||||
" max_new_tokens=512,\n",
|
||||
" no_repeat_ngram_size=2,\n",
|
||||
" sampling_topk= 1,\n",
|
||||
" sampling_topp= 1.0,\n",
|
||||
" sampling_temperature= 1.0,\n",
|
||||
" repetition_penalty= 1.0,\n",
|
||||
" regex_string= \"\",\n",
|
||||
")\n",
|
||||
"output = llm(\"What is the largest rainforest in the world?\")\n",
|
||||
"print(output)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Example 3: Using generate for multiple inputs\n",
|
||||
"llm = TitanTakeoffPro()\n",
|
||||
"rich_output = llm.generate([\"What is Deep Learning?\", \"What is Machine Learning?\"])\n",
|
||||
"print(rich_output.generations)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# Example 4: Streaming output\n",
|
||||
"llm = TitanTakeoffPro(streaming=True, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]))\n",
|
||||
"prompt = \"What is the capital of France?\"\n",
|
||||
"llm(prompt)\n",
|
||||
"\n",
|
||||
"# Example 5: Using LCEL\n",
|
||||
"llm = TitanTakeoffPro()\n",
|
||||
"prompt = PromptTemplate.from_template(\"Tell me about {topic}\")\n",
|
||||
"chain = prompt | llm\n",
|
||||
"chain.invoke({\"topic\": \"the universe\"})"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.12"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
Reference in New Issue
Block a user