Updated ui.py to include a small sleep timer while building the stream deltas. This recursive function fires off so quickly to eats up too much of the CPU. This small sleep frees up the CPU to not be bottlenecked. This value can go lower/shorter. But 0.02 or 0.025 seems to work well. (#1589)
Co-authored-by: root <root@wesgitlabdemo.icl.gtri.org>
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.
Implements #1424
I was misled into believing I could install using python 3.12 whereas the pyproject.toml explicitly states otherwise. This PR only removes this comment to make sure other people are not also trapped 😄
As discussed on Discord, the decision has been made to remove the system prompts by default, to better segregate the API and the UI usages.
A concurrent PR (#1353) is enabling the dynamic setting of a system prompt in the UI.
Therefore, if UI users want to use a custom system prompt, they can specify one directly in the UI.
If the API users want to use a custom prompt, they can pass it directly into their messages that they are passing to the API.
In the highlight of the two use case above, it becomes clear that default system_prompt does not need to exist.
* Fix the parallel ingestion mode, and make it available through conf
Also updated the documentation to show how to configure the ingest mode.
* PR feedback: redirect to documentation
* added max_new_tokens as a configuration option to the llm block in settings
* Update fern/docs/pages/manual/settings.mdx
Co-authored-by: lopagela <lpglm@orange.fr>
* Update private_gpt/settings/settings.py
Add default value for max_new_tokens = 256
Co-authored-by: lopagela <lpglm@orange.fr>
* Addressed location of docs comment
* reformatting from running 'make check'
* remove default config value from settings.yaml
---------
Co-authored-by: lopagela <lpglm@orange.fr>
* Update llama_index to 0.9.3
Had to change some imports because of breaking change durin the llama_index update to 0.9.0
* Update poetry.lock after update of llama_index
* Refactor documentation architecture
Split into several `tab` and sections
* Fix Fern's docs.yml after PR review
Thank you Danny!
Co-authored-by: dannysheridan <danny@buildwithfern.com>
* Re-add quickstart in the overview tab
It went missing after a refactoring of the doc architecture
* Documentation writing
* Adapt Makefile to fern documentation
* Do not create overlapping page names in fern documentation
This is causing 500. Thank you to @dsinghvi for the troubleshooting and the help!
* Add a readme to help to understand how fern documentation work and how to add new pages
* Rework the welcome view
Redirects directly users to installation guide with links for people that are not familiar with documentation browsing.
* Simplify the quickstart guide
* PR feedback on installation guide
A ton of refactoring can still be made there
* PR feedback on ingestion
* PR feedback on ingestion splitting
* Rename section on LLM
* Fix missing word in list of LLMs
---------
Co-authored-by: dannysheridan <danny@buildwithfern.com>
