github/langchain
1
0
Fork 0
mirror of https://github.com/hwchase17/langchain.git synced 2025-09-03 20:16:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix typo (#3728)

Browse Source
This commit is contained in:
Alan Cha
2023-04-28 16:01:09 -04:00
committed by GitHub
parent 5042bd40d3
commit e3b7a20454
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/modules/indexes/text_splitters/examples/tiktoken.ipynb
View File

@@ -6,7 +6,7 @@
"metadata": {},
"source": [
"# tiktoken (OpenAI) Length Function\n",
"You can also use tiktoken, a open source tokenizer package from OpenAI to estimate tokens used. Will probably be more accurate for their models.\n",
"You can also use tiktoken, an open source tokenizer package from OpenAI to estimate tokens used. Will probably be more accurate for their models.\n",
"\n",
"1. How the text is split: by character passed in\n",
"2. How the chunk size is measured: by `tiktoken` tokenizer"