74 Commits

Author SHA1 Message Date
Javier Martinez
21d42fd97a feat: code execution v4 (#2295)
* feat: add bundle to remove

* fix: spaces

* feat: add xml render as skill spec

* feat: add skill volume root to cache

* fix: deduplicate values

* fix: change the mount path to skill_id

* fix: use different paths

* feat: add principal

(cherry picked from commit 5db64fe721d5706440ce9f342b1388ffe742bc16)

# Conflicts:
#	private_gpt/components/code_execution/base.py
#	private_gpt/components/code_execution/code_execution_component.py
#	private_gpt/components/code_execution/local.py
#	private_gpt/components/environment/manager.py
#	private_gpt/components/tools/builders/bash_tool_builder.py
#	private_gpt/components/tools/builders/text_editor_tool_builder.py
#	private_gpt/components/tools/processors/bash_processor.py

* fix: mypy

...

* fix: config

* fix: sandbox config

* feat: add forward cookies

* fix: loop

* feat: add present server

* feat: add feature flag for tools

* refactor: move principal to another better place

* feat: add api key principal

* docs: fix docs

* docs: add present server

* fix: principal

* fix: mypy

* fix: avoid to block the loop

* fix: blocks in expansion

* fix: remove maximum concurrent users

...

* fix: multiplexer

* fix: readers

* fix: more fixes

...

* fix: impl

* feat: tool scheduler

* feat: add adaptative

* feat: add chat worker

* fix: config

* fix: max

* feat: add chat/tools workers

* fix: mypy

* feat: add generic scheduler

* fix: get result

* feat: do serializable the tool executor

* fix: tools

* fix: config

* fix: config

* fix: args

* fix: config

* fix: serializer

* Revert "fix: blocks in expansion"

This reverts commit a2110f94a8.

* fix: unify all logic

* feat: add ingestion scheduler

* fix: settings

* fix: config

* feat: add arq worker to chat

* fix: arq worker

* fix: add nest

* fix: mypy

* fix: await

* fix: script stress

* fix: tokenizer

* fix: chat scheduler

* fix: mypy

* fix: add async tokenizer

* fix: improve condense

* fix: tool scheduler

* feat: add initial real async chat worker

* fix: mypy

* fix: do resumable local executor

...

...

...

fix: revert usleess changes

fix: remove parent chat job

fix: refactor

fix: loop

ref: rename models

fix: chat engine

fix: mypy

...

...

...

fix: fix deps

* fix: tests

* fix: tests

* ...

* fix: stream

* fix: config

* fix: scheduler

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Handle PGPT_WORKER_MODE=celery in health check worker status

* fix: cancel

* fix: arch

* fix: test ingestion

* fix: deserialization of chat messages

* fix: broken results

* fix: mypy

* fix: test

* fix: config

* fix: remove arq tool worker

* fix: output cls

* fix: preserve early resumable tool callbacks

* fix: preserve async tool result order

* refactor: address worker PR review comments

* fix: mypy

* test: colocate ARQ chat enqueue coverage

* fix: remove redis from tests

* fix: mypy

* fix: tests

* test: isolate chat mocks and cancellation timing

* fix: tests

* fix: tests

* fix: test

(cherry picked from commit f8ee460af2)

* fix: worker config

* test: remove flaky chat cancellation assertion

(cherry picked from commit 1115ff2349)

# Conflicts:
#	tests/server/chat/test_chat_routes.py

* fix: emit chat pings from stream listeners

* fix: don't duplciate the output

* fix: rss memory

...

* fix: pass the args

* fix: threads

* fix: websearch

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-07-15 13:05:23 +02:00
Javier Martinez
cd8ca2214a feat: resumable chat worker + tool worker + async tokenizer (#2298)
* fix: avoid to block the loop

* fix: blocks in expansion

* fix: remove maximum concurrent users

...

* fix: multiplexer

* fix: readers

* fix: more fixes

...

* fix: impl

* feat: tool scheduler

* feat: add adaptative

* feat: add chat worker

* fix: max

* feat: add chat/tools workers

* fix: mypy

* feat: add generic scheduler

* fix: get result

* feat: do serializable the tool executor

* fix: tools

* fix: config

* fix: config

* fix: args

* fix: config

* fix: serializer

* Revert "fix: blocks in expansion"

This reverts commit a2110f94a8.

* fix: unify all logic

* feat: add ingestion scheduler

* fix: settings

* fix: config

* feat: add arq worker to chat

* fix: arq worker

* fix: add nest

* fix: mypy

* fix: await

* fix: script stress

* fix: tokenizer

* fix: chat scheduler

* fix: mypy

* fix: add async tokenizer

* fix: improve condense

* fix: tool scheduler

* feat: add initial real async chat worker

* fix: mypy

* fix: do resumable local executor

...

...

...

fix: revert usleess changes

fix: remove parent chat job

fix: refactor

fix: loop

ref: rename models

fix: chat engine

fix: mypy

...

...

...

fix: fix deps

* fix: tests

* fix: tests

* ...

* fix: stream

* fix: config

* fix: scheduler

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Handle PGPT_WORKER_MODE=celery in health check worker status

* fix: cancel

* fix: arch

* fix: test ingestion

* fix: deserialization of chat messages

* fix: broken results

* fix: mypy

* fix: test

* fix: config

* fix: remove arq tool worker

* fix: output cls

* fix: preserve early resumable tool callbacks

* fix: preserve async tool result order

* refactor: address worker PR review comments

* fix: mypy

* test: colocate ARQ chat enqueue coverage

* fix: remove redis from tests

* test: isolate chat mocks and cancellation timing

* fix: tests

(cherry picked from commit 218b599c66)

# Conflicts:
#	tests/server/chat/anthropic/test_anthropic_client.py
#	tests/server/chat/anthropic/test_langchain_anthropic.py
#	tests/server/chat/test_chat_knowledge_revamp.py
#	tests/server/chat/test_chat_routes.py
#	tests/server/chat/test_chat_routes_skills_integration.py

* fix: tests

(cherry picked from commit fc5ec0f72a)

* fix: ruff

* fix: test

* fix: worker config

(cherry picked from commit 1371c275a1)

* fix: principal

* test: remove flaky chat cancellation assertion

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-07-15 12:12:45 +02:00
Javier Martinez
bcadb418f3 docs: add community forks (#2290) 2026-07-06 11:20:32 +02:00
Javier Martinez
f3d24b5413 fix: update fern & resolve fern preview (#2288)
* chore: update fern cli

* docs: add examples

* fix: update doc
2026-07-06 11:20:05 +02:00
Javier Martinez
603152a62a docs: code execution (#2279)
* fix: preview

* docs: add doc
2026-06-29 18:32:01 +02:00
Alfonso Lozana
78a1e8b9f6 feat: control docs parser (#2273)
* fix: ensure that the publisher is ready to publish new messages

* fix: dockerfile

* fix: clones

(cherry picked from commit bc0a77e050)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: add ExtractionUnsuccessfulError in docling reader

* feat: add vision pipeline

* feat: add fall back with vision

* feat: add settings of enable_vision_fallback

* doc: update doc

* chore: code refactor

* feat: remove pypdfium2 dependency

* fix: add settings and change only doc vision behaviour

* feat: change extraction_type_override to skip_strategy_inference (disable strategy on vision transform parser)

* chore: make check

---------

Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-24 08:22:17 +02:00
zylon-ci
88d7e259a4 chore(main): release 1.0.1 (#2263)
* chore(main): release 1.0.1

* build: update uv.lock for 1.0.1

* docs: update OpenAPI spec for 1.0.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Zylon CI <ci@zylon.ai>
2026-06-18 15:55:36 +02:00
Javier Martinez
4021cf4e20 fix: use PGPT_HOME for local data, caches, and cleanup paths (#2267)
* feat: use default user folder

* docs: update references to local paths

* fix: windows

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: align make test and wipe with PGPT_HOME paths

* fix: align wipe target with PGPT_HOME local_data

* fix: folders

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-10 09:06:02 +02:00
github-actions[bot]
7595d25a4c chore(main): release 1.0.0 (#2231)
* chore(main): release 1.0.0

* build: update uv.lock for 1.0.0

* docs: update OpenAPI spec for 1.0.0

---------

Co-authored-by: zylon-ci <dev+ci@zylon.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Zylon CI <ci@zylon.ai>
2026-06-03 17:25:13 +02:00
Daniel Gallego Vico
349cec5a64 Main README update 2026-06-03 17:12:04 +02:00
Javier Martinez
80ff34b423 docs: add required libraries (#2256)
* docs: add instructions to install deps

* docs: update order
2026-06-03 15:44:59 +02:00
Javier Martinez
a6ab856c2b fix: claude desktop after changes yesterday (#2245)
* fix: claude desktop after changes yesterday

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-03 09:55:59 +02:00
Alfonso Lozana
53ea754fb3 docs: add python version (#2241) 2026-06-02 21:04:17 +02:00
Javier Martinez
65f717e046 docs: fix index url (#2238)
(cherry picked from commit 6eb12d658f04e89b7b512a362f083418023cde2f)
2026-06-02 20:40:40 +02:00
Javier Martinez
8a5c36725d docs: fix broken links (#2236)
* chore: add docs redirects

(cherry picked from commit 29fec7af0e)

* chore: fix links on doc

(cherry picked from commit ba56846b36a0e985e5efe5b38b09c1dfdf3e9acc)
(cherry picked from commit 2f86726aab656f12e809aa2bc83f67d8d08c83d8)

* fix: use current host instead hardcoded

(cherry picked from commit e32cac5afb13d49fba25101bd86afb1694044434)

* docs: fix links of doc

(cherry picked from commit 165f4c1a2abf65ba70ba2ba783ff644badfdf4a2)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* docs: update index link

---------

Co-authored-by: Alfonso Lozana <alfonsolozana@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-02 20:20:09 +02:00
Javier Martinez
183cd03857 feat!: PrivateGPT revamp v1 (#2230)
* feat!: PrivateGPT revamp v1

* chore(docs): update nodejs
2026-06-02 16:55:46 +02:00
Javier Martinez
4ca6d0cb55 fix: add numpy issue to troubleshooting (#2048)
* docs: add numpy issue to troubleshooting

* fix: troubleshooting link

...
2024-08-07 12:16:03 +02:00
Javier Martinez
f09f6dd255 fix: add built image from DockerHub (#2042)
* chore: update docker-compose with profiles

* docs: add quick start doc

* chore: generate docker release when new version is released

* chore: add dockerhub image in docker-compose

* docs: update quickstart with local/remote images

* chore: update docker tag

* chore: refactor dockerfile names

* chore: update docker-compose names

* docs: update llamacpp naming

* fix: naming

* docs: fix llamacpp command
2024-08-05 17:15:38 +02:00
Javier Martinez
dae0727a1b fix(deploy): improve Docker-Compose and quickstart on Docker (#2037)
* chore: update docker-compose with profiles

* docs: add quick start doc
2024-08-05 16:28:19 +02:00
Javier Martinez
50b3027a24 docs: update docs and capture (#2029)
* docs: update Readme

* style: refactor image

* docs: change important to tip
2024-08-01 10:01:22 +02:00
Javier Martinez
8119842ae6 feat(recipe): add our first recipe Summarize (#2028)
* feat: add summary recipe

* test: add summary tests

* docs: move all recipes docs

* docs: add recipes and summarize doc

* docs: update openapi reference

* refactor: split method in two method (summary)

* feat: add initial summarize ui

* feat: add mode explanation

* fix: mypy

* feat: allow to configure async property in summarize

* refactor: move modes to enum and update mode explanations

* docs: fix url

* docs: remove list-llm pages

* docs: remove double header

* fix: summary description
2024-07-31 16:53:27 +02:00
Javier Martinez
40638a18a5 fix: unify embedding models (#2027)
* feat: unify embedding model to nomic

* docs: add embedding dimensions mismatch

* docs: fix fern
2024-07-31 14:35:46 +02:00
Javier Martinez
9027d695c1 feat: make llama3.1 as default (#2022)
* feat: change ollama default model to llama3.1

* chore: bump versions

* feat: Change default model in local mode to llama3.1

* chore: make sure last poetry version is used

* fix: mypy

* fix: do not add BOS (with last llamacpp-python version)
2024-07-31 14:35:36 +02:00
Javier Martinez
e54a8fe043 fix: prevent to ingest local files (by default) (#2010)
* feat: prevent to local ingestion (by default) and add white-list

* docs: add local ingestion warning

* docs: add missing comment

* fix: update exception error

* fix: black
2024-07-31 14:33:46 +02:00
Javier Martinez
20bad17c98 feat(llm): autopull ollama models (#2019)
* chore: update ollama (llm)

* feat: allow to autopull ollama models

* fix: mypy

* chore: install always ollama client

* refactor: check connection and pull ollama method to utils

* docs: update ollama config with autopulling info
2024-07-29 13:25:42 +02:00
Iván Martínez
05a986231c Add proper param to demo urls (#2007) 2024-07-22 14:44:03 +02:00
Javier Martinez
b62669784b docs: update welcome page (#2004) 2024-07-18 14:42:39 +02:00
Jackson
43cc31f740 feat(vectordb): Milvus vector db Integration (#1996)
* integrate Milvus into Private GPT

* adjust milvus settings

* update doc info and reformat

* adjust milvus initialization

* adjust import error

* mionr update

* adjust format

* adjust the db storing path

* update doc
2024-07-18 10:55:45 +02:00
Javier Martinez
4523a30c8f feat(docs): update documentation and fix preview-docs (#2000)
* docs: add missing configurations

* docs: change HF embeddings by ollama

* docs: add disclaimer about Gradio UI

* docs: improve readability in concepts

* docs: reorder `Fully Local Setups`

* docs: improve setup instructions

* docs: prevent have duplicate documentation and use table to show different options

* docs: rename privateGpt to PrivateGPT

* docs: update ui image

* docs: remove useless header

* docs: convert to alerts ingestion disclaimers

* docs: add UI alternatives

* docs: reference UI alternatives in disclaimers

* docs: fix table

* chore: update doc preview version

* chore: add permissions

* chore: remove useless line

* docs: fixes

...
2024-07-18 10:06:51 +02:00
Javier Martinez
01b7ccd064 fix(config): make tokenizer optional and include a troubleshooting doc (#1998)
* docs: add troubleshooting

* fix: pass HF token to setup script and prevent to download tokenizer when it is empty

* fix: improve log and disable specific tokenizer by default

* chore: change HF_TOKEN environment to be aligned with default config

* ifx: mypy
2024-07-17 10:06:27 +02:00
Javier Martinez
15f73dbc48 docs: update repo links, citations (#1990)
* docs: update project links

...

* docs: update citation
2024-07-09 10:03:57 +02:00
fern
187bc9320e (feat): add github button (#1989)
Co-authored-by: chdeskur <chdeskur@gmail.com>
2024-07-09 08:48:47 +02:00
Marco Braga
dde02245bc fix(docs): Fix concepts.mdx referencing to installation page (#1779)
* Fix/update concepts.mdx referencing to installation page

The link for `/installation` is broken in the "Main Concepts" page.

The correct path would be `./installation` or  maybe `/installation/getting-started/installation`

* fix: docs

---------

Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 16:19:50 +02:00
Mart
067a5f144c feat(docs): Fix setup docu (#1926)
* Update settings.mdx

* docs: add cmd

---------

Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 16:19:16 +02:00
Proger666
2612928839 feat(vectorstore): Add clickhouse support as vectore store (#1883)
* Added ClickHouse vector sotre support

* port fix

* updated lock file

* fix: mypy

* fix: mypy

---------

Co-authored-by: Valery Denisov <valerydenisov@double.cloud>
Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 16:18:22 +02:00
uw4
fc13368bc7 feat(llm): Support for Google Gemini LLMs and Embeddings (#1965)
* Support for Google Gemini LLMs and Embeddings

Initial support for Gemini, enables usage of Google LLMs and embedding models (see settings-gemini.yaml)

Install via
poetry install --extras "llms-gemini embeddings-gemini"

Notes:
* had to bump llama-index-core to later version that supports Gemini
* poetry --no-update did not work: Gemini/llama_index seem to require more (transient) updates to make it work...

* fix: crash when gemini is not selected

* docs: add gemini llm

---------

Co-authored-by: Javier Martinez <javiermartinezalvarez98@gmail.com>
2024-07-08 11:47:36 +02:00
Shengsheng Huang
19a7c065ef feat(docs): update doc for ipex-llm (#1968) 2024-07-08 09:42:44 +02:00
Fran García
d13029a046 feat(docs): add privategpt-ts sdk (#1924) 2024-05-10 14:13:15 +02:00
Daniel Gallego Vico
c1802e7cf0 fix(docs): Update installation.mdx (#1866)
Update repo url
2024-04-19 17:10:58 +02:00
Иван
8a836e4651 feat(docs): Add guide Llama-CPP Linux AMD GPU support (#1782) 2024-04-02 16:55:05 +02:00
machatschek
83adc12a8e feat(RAG): Introduce SentenceTransformer Reranker (#1810) 2024-04-02 10:29:51 +02:00
Iván Martínez
572518143a feat(docs): Feature/upgrade docs (#1741)
* Upgrade fern version

* Add info about SDKs
2024-03-19 21:26:53 +01:00
Brett England
134fc54d7d feat(ingest): Created a faster ingestion mode - pipeline (#1750)
* Unify pgvector and postgres connection settings

* Remove local changes

* Update file pgvector->postgres

* postgresql should be postgres

* Adding pipeline ingestion mode

* disable hugging face parallelism.  Continue on file to doc transform failure

* Semaphore to limit docq async workers. ETA reporting
2024-03-19 21:24:46 +01:00
Otto L
1efac6a3fe feat(llm - embed): Add support for Azure OpenAI (#1698)
* Add support for Azure OpenAI

* fix: wrong default api_version

Should be dashes instead of underscores.
see: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference

* fix: code styling

applied "make check" changes

* refactor: extend documentation

* mention azopenai as available option and extras
* add recommended section
* include settings-azopenai.yaml configuration file

* fix: documentation
2024-03-15 16:49:50 +01:00
Brett England
258d02d87c fix(docs): Minor documentation amendment (#1739)
* Unify pgvector and postgres connection settings

* Remove local changes

* Update file pgvector->postgres

* postgresql should be postgres
2024-03-15 16:36:32 +01:00
Brett England
63de7e4930 feat: unify settings for vector and nodestore connections to PostgreSQL (#1730)
* Unify pgvector and postgres connection settings

* Remove local changes

* Update file pgvector->postgres
2024-03-15 09:55:17 +01:00
Brett England
68b3a34b03 feat(nodestore): add Postgres for the doc and index store (#1706)
* Adding Postgres for the doc and index store

* Adding documentation.  Rename postgres database local->simple.  Postgres storage dependencies

* Update documentation for postgres storage

* Renaming feature to nodestore

* update docstore -> nodestore in doc

* missed some docstore changes in doc

* Updated poetry.lock

* Formatting updates to pass ruff/black checks

* Correction to unreachable code!

* Format adjustment to pass black test

* Adjust extra inclusion name for vector pg

* extra dep change for pg vector

* storage-postgres -> storage-nodestore-postgres

* Hash change on poetry lock
2024-03-14 17:12:33 +01:00
Andrew Jiang
84ad16af80 feat(docs): upgrade fern (#1596) 2024-03-11 23:02:56 +01:00
Iván Martínez
45f05711eb feat: Upgrade to LlamaIndex to 0.10 (#1663)
* Extract optional dependencies

* Separate local mode into llms-llama-cpp and embeddings-huggingface for clarity

* Support Ollama embeddings

* Upgrade to llamaindex 0.10.14. Remove legacy use of ServiceContext in ContextChatEngine

* Fix vector retriever filters
2024-03-06 17:51:30 +01:00
TQ
cd40e3982b feat(Vector): support pgvector (#1624) 2024-02-20 15:29:26 +01:00