11 Commits

Author SHA1 Message Date
Javier Martinez
20e3f6fbc8 chore: update typechecking deps (#2302) 2026-07-16 13:36:11 +02:00
Javier Martinez
091d5f7020 fix: random bugs (#2301)
* fix: celery callbacks

* fix: s3 + skill creator

* fix: resumable when there's params

* fix: add distributed cache

* fix: do durable context stack

* fix: mcp tools

* fix: present server tool

* fix: add cache to the skills

* fix: mcp

* fix: mypy
2026-07-16 09:14:43 +02:00
Javier Martinez
576589bdc5 feat: add websearch in a sandbox (#2300)
* feat: add websearch in a sandbox

* fix: celery callbacks

(cherry picked from commit fe7d0ce9d5)

* fix: mypy
2026-07-16 08:39:52 +02:00
Javier Martinez
21d42fd97a feat: code execution v4 (#2295)
* feat: add bundle to remove

* fix: spaces

* feat: add xml render as skill spec

* feat: add skill volume root to cache

* fix: deduplicate values

* fix: change the mount path to skill_id

* fix: use different paths

* feat: add principal

(cherry picked from commit 5db64fe721d5706440ce9f342b1388ffe742bc16)

# Conflicts:
#	private_gpt/components/code_execution/base.py
#	private_gpt/components/code_execution/code_execution_component.py
#	private_gpt/components/code_execution/local.py
#	private_gpt/components/environment/manager.py
#	private_gpt/components/tools/builders/bash_tool_builder.py
#	private_gpt/components/tools/builders/text_editor_tool_builder.py
#	private_gpt/components/tools/processors/bash_processor.py

* fix: mypy

...

* fix: config

* fix: sandbox config

* feat: add forward cookies

* fix: loop

* feat: add present server

* feat: add feature flag for tools

* refactor: move principal to another better place

* feat: add api key principal

* docs: fix docs

* docs: add present server

* fix: principal

* fix: mypy

* fix: avoid to block the loop

* fix: blocks in expansion

* fix: remove maximum concurrent users

...

* fix: multiplexer

* fix: readers

* fix: more fixes

...

* fix: impl

* feat: tool scheduler

* feat: add adaptative

* feat: add chat worker

* fix: config

* fix: max

* feat: add chat/tools workers

* fix: mypy

* feat: add generic scheduler

* fix: get result

* feat: do serializable the tool executor

* fix: tools

* fix: config

* fix: config

* fix: args

* fix: config

* fix: serializer

* Revert "fix: blocks in expansion"

This reverts commit a2110f94a8.

* fix: unify all logic

* feat: add ingestion scheduler

* fix: settings

* fix: config

* feat: add arq worker to chat

* fix: arq worker

* fix: add nest

* fix: mypy

* fix: await

* fix: script stress

* fix: tokenizer

* fix: chat scheduler

* fix: mypy

* fix: add async tokenizer

* fix: improve condense

* fix: tool scheduler

* feat: add initial real async chat worker

* fix: mypy

* fix: do resumable local executor

...

...

...

fix: revert usleess changes

fix: remove parent chat job

fix: refactor

fix: loop

ref: rename models

fix: chat engine

fix: mypy

...

...

...

fix: fix deps

* fix: tests

* fix: tests

* ...

* fix: stream

* fix: config

* fix: scheduler

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Handle PGPT_WORKER_MODE=celery in health check worker status

* fix: cancel

* fix: arch

* fix: test ingestion

* fix: deserialization of chat messages

* fix: broken results

* fix: mypy

* fix: test

* fix: config

* fix: remove arq tool worker

* fix: output cls

* fix: preserve early resumable tool callbacks

* fix: preserve async tool result order

* refactor: address worker PR review comments

* fix: mypy

* test: colocate ARQ chat enqueue coverage

* fix: remove redis from tests

* fix: mypy

* fix: tests

* test: isolate chat mocks and cancellation timing

* fix: tests

* fix: tests

* fix: test

(cherry picked from commit f8ee460af2)

* fix: worker config

* test: remove flaky chat cancellation assertion

(cherry picked from commit 1115ff2349)

# Conflicts:
#	tests/server/chat/test_chat_routes.py

* fix: emit chat pings from stream listeners

* fix: don't duplciate the output

* fix: rss memory

...

* fix: pass the args

* fix: threads

* fix: websearch

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-07-15 13:05:23 +02:00
Javier Martinez
cd8ca2214a feat: resumable chat worker + tool worker + async tokenizer (#2298)
* fix: avoid to block the loop

* fix: blocks in expansion

* fix: remove maximum concurrent users

...

* fix: multiplexer

* fix: readers

* fix: more fixes

...

* fix: impl

* feat: tool scheduler

* feat: add adaptative

* feat: add chat worker

* fix: max

* feat: add chat/tools workers

* fix: mypy

* feat: add generic scheduler

* fix: get result

* feat: do serializable the tool executor

* fix: tools

* fix: config

* fix: config

* fix: args

* fix: config

* fix: serializer

* Revert "fix: blocks in expansion"

This reverts commit a2110f94a8.

* fix: unify all logic

* feat: add ingestion scheduler

* fix: settings

* fix: config

* feat: add arq worker to chat

* fix: arq worker

* fix: add nest

* fix: mypy

* fix: await

* fix: script stress

* fix: tokenizer

* fix: chat scheduler

* fix: mypy

* fix: add async tokenizer

* fix: improve condense

* fix: tool scheduler

* feat: add initial real async chat worker

* fix: mypy

* fix: do resumable local executor

...

...

...

fix: revert usleess changes

fix: remove parent chat job

fix: refactor

fix: loop

ref: rename models

fix: chat engine

fix: mypy

...

...

...

fix: fix deps

* fix: tests

* fix: tests

* ...

* fix: stream

* fix: config

* fix: scheduler

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Handle PGPT_WORKER_MODE=celery in health check worker status

* fix: cancel

* fix: arch

* fix: test ingestion

* fix: deserialization of chat messages

* fix: broken results

* fix: mypy

* fix: test

* fix: config

* fix: remove arq tool worker

* fix: output cls

* fix: preserve early resumable tool callbacks

* fix: preserve async tool result order

* refactor: address worker PR review comments

* fix: mypy

* test: colocate ARQ chat enqueue coverage

* fix: remove redis from tests

* test: isolate chat mocks and cancellation timing

* fix: tests

(cherry picked from commit 218b599c66)

# Conflicts:
#	tests/server/chat/anthropic/test_anthropic_client.py
#	tests/server/chat/anthropic/test_langchain_anthropic.py
#	tests/server/chat/test_chat_knowledge_revamp.py
#	tests/server/chat/test_chat_routes.py
#	tests/server/chat/test_chat_routes_skills_integration.py

* fix: tests

(cherry picked from commit fc5ec0f72a)

* fix: ruff

* fix: test

* fix: worker config

(cherry picked from commit 1371c275a1)

* fix: principal

* test: remove flaky chat cancellation assertion

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-07-15 12:12:45 +02:00
Javier Martinez
f2cffd1ab9 feat: code execution v2 (#2278)
* fix: ensure that the publisher is ready to publish new messages

* fix: dockerfile

* fix: clones

(cherry picked from commit bc0a77e050)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* feat: M3 persistent sessions + M4 isolated bash + OpenSandbox provider

Introduces mount-aware sessions with canonical path abstraction:
- ContentBundle / SessionMount abstraction for skill/plugin mounting
- PathTranslator: rewrites commands and scrubs output (canonical ↔ real paths)
- LocalMount + ReadOnlyMount: local FS-backed mounts; read-only cache shared across sessions
- BashExecutor: asyncio subprocess with setsid + setrlimit isolation + killpg on timeout
- SkillLoader: downloads skill files from object storage as ContentBundles
- LocalCodeExecutionProvider: rewritten to use mounts, TTL reaper, BashExecutor
- OpenSandboxCodeExecutionProvider: new Docker/K8s backend via opensandbox SDK
- Async cascade: create_session / get_or_create_session / build_tool all async
- ObjectStorage.list_files() added to ABC + both implementations
- New settings: session_ttl_seconds, bash rlimit fields, OpenSandboxSettings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: generic ContentBundle with BundledFile, remove skill_filter from code_execution layer

- Replace ContentBundle/dataclass with pydantic BaseModel; add BundledFile with path, content, permissions
- Update ReadOnlyMount to use list[BundledFile] with per-file chmod
- Remove skill_filter and SkillLoader from create_session() in base, local, and code_execution_component
- Move skill-to-bundle resolution into BashToolBuilder (inject SkillLoader there)
- Update SkillLoader.load() to return list[BundledFile] objects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove opensandbox

* feat: use sandbox inside of code executor

* fix: mypy

* feat: add workspace manager

* feat: improvements

* feat: add env abstraction

* feat: remove leftovers

* feat: improve skill content

* fix: macos

* fix: mypy

* feat: add code executor prompt

* feat: add content bundle in stack

* fix: mypy

* fix: layout

* feat: add skill prompt

* fix: order

* feat: allow to present final files

* feat: add container block

* fix: remove default config

* fix: session pers

* fix: do lazy env

* fix: mounter

* feat: refactor mounter

* fix: container

* feat: add requirements

* fix: add container registry

* fix: move to be lazy

* fix: mypy

* fix: current folder

* fix: stop sandbox in tabular

* fi: ensure to use absolute paths in bash & text editor

* feat: add files router

* feat: simplify

* feat: update present files

* fix: download files

* fix: mypy

* fix: tests

* fix: tests

* fix: bash executor in linux

* Revert "fix: bash executor in linux"

This reverts commit 483e208a96.

* Revert "fix: tests"

This reverts commit 50d9288f5e.

* test: remove test in ci

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 18:29:29 +02:00
Javier Martinez
4cca7d0b97 feat: allow to attach files as Anthropic does (#2265)
* feat: add convert/parser service

(cherry picked from commit 2350ad0a3e60291fbb0ded1d1f1d1c784dcc6b3c)

* fix: mypy & test

(cherry picked from commit 4e5358ee156dc11400b6f1fdbcc71a58220a581c)

* feat: add initial content

(cherry picked from commit 6eaeb4f1a8e20ce09fb208aca8468185d3866e3c)

* fix: return content as tree

* docs: add convert examples

* feat: add parallel support

* feat: add concurrency in multimodal interceptor

* refactor: move config to a common class

* fix: apply preprocessors only over the last user message

* fix: final tweaks to allow to configure feature flag

* fix: support legacy blocks

* test: fix

* fix: mypy

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: image/audio in the chat

* fix: copilot

* fix: name

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-25 15:35:46 +02:00
Javier Martinez
42a6c047d0 feat: add validation skill endpoint + skill prompt (#2272)
* feat: add skill validator endpoint

* fix: skills

* feat: add skill prompt

(cherry picked from commit c36d27f008)

# Conflicts:
#	private_gpt/chat/input_models.py
#	private_gpt/components/prompts/prompt_builder.py
#	private_gpt/server/chat/interceptors/platform_guidelines_interceptor.py

* fix: test

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: clarify skill visibility comment

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: rename has_loaded_non_eager variable to snake_case

* test: cover skill validation route

* test: fix invalid skill validation case

* test: cover skill tool visibility states

* feat: improve skill tools + prompt

* fix: mypy

* feat: allow to limit the skill maximum size

* feat: add validation error codes

* fix: tests

* fix: invalid frontmatter

* fix: edge cases

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-25 15:35:26 +02:00
JulianS1987
c120a90603 fix: make embedding API key optional for local OpenAI-compatible engines (#2261)
Ingesting documents with a local OpenAI-compatible embedding engine (Ollama,
vLLM, ...) failed with `openai.OpenAIError: Missing credentials` when no API
key was configured. The OpenAI-like embedding factory forwarded an empty
api_key straight to the OpenAI client, which rejects empty credentials at
request time.

Chat already tolerates a missing key (completions/openailike.py uses
`... or "default"`) and so does the embedding discovery probe
(`... or "no-key"`); only the embedding factory lacked the fallback, so model
discovery and chat worked while the first real embedding call (ingestion)
crashed.

Mirror the chat behaviour with a placeholder key on the local
OpenAI-compatible path. The strict api.openai.com path is unchanged and still
requires a real key.

Fixes #2260

Co-authored-by: Julian Siegrist <jsiegrist@immomio.de>
2026-06-08 08:09:15 +02:00
Javier Martinez
c438ba183f fix: citation/thinking prompts (#2257)
* fix: platform guidelines

(cherry picked from commit e341525b3ad5b3000bc5c6fd7b5f2d6ef90d8403)

* fix: remove dummy file

(cherry picked from commit 44b46758af7db803059831fb4bc57b15a486bf65)

* fix: tests

(cherry picked from commit 631d796175ab81f33f0e5bd264add3b1babf6743)
2026-06-03 17:00:13 +02:00
Javier Martinez
183cd03857 feat!: PrivateGPT revamp v1 (#2230)
* feat!: PrivateGPT revamp v1

* chore(docs): update nodejs
2026-06-02 16:55:46 +02:00