Add a symbolic link to our actual docs for the gpt4all-chat app.

Signed-off-by: Adam Treat <treat.adam@gmail.com>

gpt4all-chat/CHANGELOG.md

@@ -4,6 +4,22 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [Unreleased]
+
+### Fixed
+- Fix API server ignoring assistant messages in history after v3.5.0 ([#3256](https://github.com/nomic-ai/gpt4all/pull/3256))
+- Fix API server replying with incorrect token counts and stop reason after v3.5.0 ([#3256](https://github.com/nomic-ai/gpt4all/pull/3256))
+- Fix API server remembering previous, unrelated conversations after v3.5.0 ([#3256](https://github.com/nomic-ai/gpt4all/pull/3256))
+- Fix mishandling of default chat template and system message of cloned models in v3.5.0 ([#3262](https://github.com/nomic-ai/gpt4all/pull/3262))
+
+## [3.5.1] - 2024-12-10
+
+### Fixed
+- Fix an incorrect value for currentResponse ([#3245](https://github.com/nomic-ai/gpt4all/pull/3245))
+- Fix the default model button so it works again after 3.5.0 ([#3246](https://github.com/nomic-ai/gpt4all/pull/3246))
+- Fix chat templates for Nous Hermes 2 Mistral, Mistral OpenOrca, Qwen 2, and remote models ([#3250](https://github.com/nomic-ai/gpt4all/pull/3250))
+- Fix chat templates for Llama 3.2 models ([#3251](https://github.com/nomic-ai/gpt4all/pull/3251))
+
 ## [3.5.0] - 2024-12-09
 
 ### Changed
@@ -203,6 +219,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 - Fix several Vulkan resource management issues ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
 - Fix crash/hang when some models stop generating, by showing special tokens ([#2701](https://github.com/nomic-ai/gpt4all/pull/2701))
 
+[Unreleased]: https://github.com/nomic-ai/gpt4all/compare/v3.5.1...HEAD
+[3.5.1]: https://github.com/nomic-ai/gpt4all/compare/v3.5.0...v3.5.1
 [3.5.0]: https://github.com/nomic-ai/gpt4all/compare/v3.5.0-rc2...v3.5.0
 [3.5.0-rc2]: https://github.com/nomic-ai/gpt4all/compare/v3.5.0-rc1...v3.5.0-rc2
 [3.5.0-rc1]: https://github.com/nomic-ai/gpt4all/compare/v3.4.2...v3.5.0-rc1

gpt4all-chat/CMakeLists.txt

@@ -4,9 +4,9 @@ include(../common/common.cmake)
 
 set(APP_VERSION_MAJOR 3)
 set(APP_VERSION_MINOR 5)
-set(APP_VERSION_PATCH 0)
+set(APP_VERSION_PATCH 2)
 set(APP_VERSION_BASE "${APP_VERSION_MAJOR}.${APP_VERSION_MINOR}.${APP_VERSION_PATCH}")
-set(APP_VERSION "${APP_VERSION_BASE}")
+set(APP_VERSION "${APP_VERSION_BASE}-dev0")
 
 project(gpt4all VERSION ${APP_VERSION_BASE} LANGUAGES CXX C)
 

gpt4all-chat/docs

@@ -0,0 +1 @@
+../gpt4all-bindings/python/docs/gpt4all_desktop

gpt4all-chat/metadata/latestnews.md

@@ -1,20 +1,18 @@
 ## Latest News
 
-GPT4All v3.4.1 was released on October 11th, and fixes several issues with LocalDocs from the previous release.
-
-GPT4All v3.4.2 was released on October 16th, and fixes more issues with LocalDocs.
-
-**IMPORTANT NOTE:** If you are coming from v3.4.0, be sure to "Rebuild" your collections at least once after updating!
+GPT4All v3.5.1 was released on December 10th and fixes several issues with the new chat templates. Additionally, it fixes a bug with the default model button as well an issue with remote models.
 
 ---
 
-GPT4All v3.4.0 was released on October 8th. Changes include:
+GPT4All v3.5.0 was released on December 9th. Changes include:
 
-* **Attached Files:** You can now attach a small Microsoft Excel spreadsheet (.xlsx) to a chat message and ask the model about it.
-* **LocalDocs Accuracy:** The LocalDocs algorithm has been enhanced to find more accurate references for some queries.
-* **Word Document Support:** LocalDocs now supports Microsoft Word (.docx) documents natively.
-  * **IMPORTANT NOTE:** If .docx files are not found, make sure Settings > LocalDocs > Allowed File Extensions includes "docx".
-* **Forgetful Model Fixes:** Issues with the "Redo last chat response" button, and with continuing chats from previous sessions, have been fixed.
-* **Chat Saving Improvements:** On exit, GPT4All will no longer save chats that are not new or modified. As a bonus, downgrading without losing access to all chats will be possible in the future, should the need arise.
-* **UI Fixes:** The model list no longer scrolls to the top when you start downloading a model.
-* **New Models:** LLama 3.2 Instruct 3B and 1B models now available in model list.
+* **Message Editing:**
+  * You can now edit any message you've sent by clicking the pencil icon below it.
+  * You can now redo earlier responses in the conversation.
+* **Templates:** Chat templates have been completely overhauled! They now use Jinja-style syntax. You may notice warnings or errors in the UI. Read the linked docs, and if you have any questions, please ask on the Discord.
+* **File Attachments:** Markdown and plain text files are now supported as file attachments.
+* **System Tray:** There is now an option in Application Settings to allow GPT4All to minimize to the system tray instead of closing.
+* **Local API Server:**
+  * The API server now supports system messages from the client and no longer uses the system message in settings.
+  * You can now send messages to the API server in any order supported by the model instead of just user/assistant pairs.
+* **Translations:** The Italian and Romanian translations have been improved.

gpt4all-chat/metadata/models3.json

@@ -30,7 +30,7 @@
     "url": "https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_0.gguf",
     "promptTemplate": "<|start_header_id|>user<|end_header_id|>\n\n%1<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n%2",
     "systemPrompt": "<|start_header_id|>system<|end_header_id|>\nCutting Knowledge Date: December 2023\n\nYou are a helpful assistant.<|eot_id|>",
-    "chatTemplate": "{{- bos_token }}\n{%- if not date_string is defined %}\n    {%- if strftime_now is defined %}\n        {%- set date_string = strftime_now(\"%d %b %Y\") %}\n    {%- else %}\n        {%- set date_string = \"26 Jul 2024\" %}\n    {%- endif %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n"
+    "chatTemplate": "{{- bos_token }}\n{%- if not date_string is defined %}\n    {%- if strftime_now is defined %}\n        {%- set date_string = strftime_now('%d %b %Y') %}\n    {%- else %}\n        {%- set date_string = '26 Jul 2024' %}\n    {%- endif %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set loop_start = 1 %}\n{%- else %}\n    {%- set system_message = '' %}\n    {%- set loop_start = 0 %}\n{%- endif %}\n\n{#- System message #}\n{{- '<|start_header_id|>system<|end_header_id|>\\n\\n' }}\n{{- 'Cutting Knowledge Date: December 2023\\n' }}\n{{- 'Today Date: ' + date_string + '\\n\\n' }}\n{{- system_message }}\n{{- '<|eot_id|>' }}\n\n{%- for message in messages %}\n    {%- if loop.index0 >= loop_start %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n' + message['content'] | trim + '<|eot_id|>' }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}"
   },
   {
     "order": "c",
@@ -47,7 +47,7 @@
     "url": "https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_0.gguf",
     "promptTemplate": "<|start_header_id|>user<|end_header_id|>\n\n%1<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n%2",
     "systemPrompt": "<|start_header_id|>system<|end_header_id|>\nCutting Knowledge Date: December 2023\n\nYou are a helpful assistant.<|eot_id|>",
-    "chatTemplate": "{{- bos_token }}\n{%- if not date_string is defined %}\n    {%- if strftime_now is defined %}\n        {%- set date_string = strftime_now(\"%d %b %Y\") %}\n    {%- else %}\n        {%- set date_string = \"26 Jul 2024\" %}\n    {%- endif %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n"
+    "chatTemplate": "{{- bos_token }}\n{%- if not date_string is defined %}\n    {%- if strftime_now is defined %}\n        {%- set date_string = strftime_now('%d %b %Y') %}\n    {%- else %}\n        {%- set date_string = '26 Jul 2024' %}\n    {%- endif %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set loop_start = 1 %}\n{%- else %}\n    {%- set system_message = '' %}\n    {%- set loop_start = 0 %}\n{%- endif %}\n\n{#- System message #}\n{{- '<|start_header_id|>system<|end_header_id|>\\n\\n' }}\n{{- 'Cutting Knowledge Date: December 2023\\n' }}\n{{- 'Today Date: ' + date_string + '\\n\\n' }}\n{{- system_message }}\n{{- '<|eot_id|>' }}\n\n{%- for message in messages %}\n    {%- if loop.index0 >= loop_start %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n' + message['content'] | trim + '<|eot_id|>' }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}"
   },
   {
     "order": "d",
@@ -63,7 +63,8 @@
     "description": "<strong>Good overall fast chat model</strong><br><ul><li>Fast responses</li><li>Chat based model</li><li>Accepts system prompts in ChatML format</li><li>Trained by Mistral AI<li>Finetuned by Nous Research on the OpenHermes-2.5 dataset<li>Licensed for commercial use</ul>",
     "url": "https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF/resolve/main/Nous-Hermes-2-Mistral-7B-DPO.Q4_0.gguf",
     "promptTemplate": "<|im_start|>user\n%1<|im_end|>\n<|im_start|>assistant\n%2<|im_end|>\n",
-    "systemPrompt": ""
+    "systemPrompt": "",
+    "chatTemplate": "{%- for message in messages %}\n    {{- '<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}"
   },
   {
     "order": "e",
@@ -112,7 +113,8 @@
     "description": "<strong>Strong overall fast chat model</strong><br><ul><li>Fast responses</li><li>Chat based model</li><li>Trained by Mistral AI<li>Finetuned on OpenOrca dataset curated via <a href=\"https://atlas.nomic.ai/\">Nomic Atlas</a><li>Licensed for commercial use</ul>",
     "url": "https://gpt4all.io/models/gguf/mistral-7b-openorca.gguf2.Q4_0.gguf",
     "promptTemplate": "<|im_start|>user\n%1<|im_end|>\n<|im_start|>assistant\n%2<|im_end|>\n",
-    "systemPrompt": "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI.\n<|im_end|>\n"
+    "systemPrompt": "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI.\n<|im_end|>\n",
+    "chatTemplate": "{%- if not add_generation_prompt is defined %}\n    {%- set add_generation_prompt = false %}\n{%- endif %}\n{%- for message in messages %}\n    {{- '<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}"
   },
   {
     "order": "h",
@@ -457,6 +459,7 @@
     "description": "<ul><li>Very fast responses</li><li>Instruction based model</li><li>Usage of LocalDocs (RAG): Highly recommended</li><li>Supports context length of up to 32768</li><li>Trained and finetuned by Qwen (Alibaba Cloud)</li><li>License: <a href=\"https://www.apache.org/licenses/LICENSE-2.0.html/\">Apache 2.0</a></li></ul>",
     "url": "https://huggingface.co/Qwen/Qwen2-1.5B-Instruct-GGUF/resolve/main/qwen2-1_5b-instruct-q4_0.gguf",
     "promptTemplate": "<|im_start|>user\n%1<|im_end|>\n<|im_start|>assistant\n%2<|im_end|>",
-    "systemPrompt": "<|im_start|>system\nBelow is an instruction that describes a task. Write a response that appropriately completes the request.<|im_end|>\n"
+    "systemPrompt": "<|im_start|>system\nBelow is an instruction that describes a task. Write a response that appropriately completes the request.<|im_end|>\n",
+    "chatTemplate": "{%- for message in messages %}\n    {%- if loop.first and messages[0]['role'] != 'system' %}\n        {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n    {{- '<|im_start|>' + message['role'] + '\\n' + message['content'] + '<|im_end|>' + '\\n' }}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}"
   }
 ]

gpt4all-chat/metadata/release.json

@@ -228,5 +228,15 @@
     "version": "3.4.2",
     "notes": "* **LocalDocs Fixes:** Several issues with LocalDocs, some of which were introduced in v3.4.0, have been fixed.\n  * Fixed the possible use of references from unselected collections.\n  * Fixed unnecessary reindexing of files with uppercase extensions.\n  * Fixed hybrid search failure due to inconsistent database state.\n  * Fully fixed the blank Embeddings Device selection in LocalDocs settings.\n  * Fixed LocalDocs indexing of large PDFs making very slow progress or even stalling.\n",
     "contributors": "* Adam Treat (Nomic AI)\n* Jared Van Bortel (Nomic AI)"
+  },
+  {
+    "version": "3.5.0",
+    "notes": "* **Message Editing:**\n  * You can now edit any message you've sent by clicking the pencil icon below it.\n  * You can now redo earlier responses in the conversation.\n* **Templates:** Chat templates have been completely overhauled! They now use Jinja-style syntax. You may notice warnings or errors in the UI. Read the linked docs, and if you have any questions, please ask on the Discord.\n* **File Attachments:** Markdown and plain text files are now supported as file attachments.\n* **System Tray:** There is now an option in Application Settings to allow GPT4All to minimize to the system tray instead of closing.\n* **Local API Server:**\n  * The API server now supports system messages from the client and no longer uses the system message in settings.\n  * You can now send messages to the API server in any order supported by the model instead of just user/assistant pairs.\n* **Translations:** The Italian and Romanian translations have been improved.\n",
+    "contributors": "* Jared Van Bortel (Nomic AI)\n* Adam Treat (Nomic AI)\n* Benjamin Gallois (`@bgallois`)\n* Riccardo Giovanetti (`@Harvester62`)\n* Victor Emanuel (`@SINAPSA-IC`)"
+  },
+  {
+    "version": "3.5.1",
+    "notes": "* **Chat template fixes:** Llama 3.2 models, Nous Hermes 2 Mistral, Mistral OpenOrca, Qwen 2 and remote models\n* **Bugfix:** Fix the default model button so it works again after 3.5.0\n",
+    "contributors": "* Jared Van Bortel (Nomic AI)\n* Adam Treat (Nomic AI)"
   }
 ]

gpt4all-chat/qml/ChatView.qml

@@ -668,6 +668,7 @@ Rectangle {
                         id: homePage
                         color: "transparent"
                         anchors.fill: parent
+                        z: 200
                         visible: !currentChat.isModelLoaded && (ModelList.selectableModels.count === 0 || currentModelName() === "") && !currentChat.isServer
 
                         ColumnLayout {

gpt4all-chat/src/chatllm.cpp

@@ -851,39 +851,44 @@ std::string ChatLLM::applyJinjaTemplate(std::span<const ChatItem> items) const
     return *maybeRendered;
 }
 
-auto ChatLLM::promptInternalChat(const QStringList &enabledCollections, const LLModel::PromptContext &ctx)
-    -> ChatPromptResult
+auto ChatLLM::promptInternalChat(const QStringList &enabledCollections, const LLModel::PromptContext &ctx,
+                                 std::optional<QList<ChatItem>> chat) -> ChatPromptResult
 {
     Q_ASSERT(isModelLoaded());
     Q_ASSERT(m_chatModel);
 
-    QList<ResultInfo> databaseResults;
-    const int retrievalSize = MySettings::globalInstance()->localDocsRetrievalSize();
-    if (!enabledCollections.isEmpty()) {
-        std::optional<std::pair<int, QString>> query;
-        {
-            // Find the prompt that represents the query. Server chats are flexible and may not have one.
-            auto items = m_chatModel->chatItems(); // holds lock
-            Q_ASSERT(items);
-            auto response = items.end() - 1;
-            if (auto peer = m_chatModel->getPeerUnlocked(response))
-                query = {*peer - items.begin(), (*peer)->value};
-        }
-        if (query) {
-            auto &[promptIndex, queryStr] = *query;
-            emit requestRetrieveFromDB(enabledCollections, queryStr, retrievalSize, &databaseResults); // blocks
-            m_chatModel->updateSources(promptIndex, databaseResults);
-            emit databaseResultsChanged(databaseResults);
-        }
-    }
-
     // copy messages for safety (since we can't hold the lock the whole time)
+    std::optional<std::pair<int, QString>> query;
     std::vector<ChatItem> chatItems;
     {
-        auto items = m_chatModel->chatItems(); // holds lock
-        Q_ASSERT(items.size() >= 2); // should be prompt/response pairs
-        chatItems.assign(items.begin(), items.end() - 1); // exclude last
+        std::optional<ChatModelAccessor> items;
+        std::span<const ChatItem> view;
+        if (chat) {
+            view = *chat;
+        } else {
+            items = m_chatModel->chatItems(); // holds lock
+            Q_ASSERT(!items->empty());
+            view = *items;
+        }
+        Q_ASSERT(view.size() >= 2); // should be prompt/response pairs
+
+        // Find the prompt that represents the query. Server chats are flexible and may not have one.
+        auto response = view.end() - 1;
+        if (auto peer = m_chatModel->getPeer(view, response))
+            query = { *peer - view.begin(), (*peer)->value };
+
+        chatItems.assign(view.begin(), view.end() - 1); // exclude last
     }
+
+    QList<ResultInfo> databaseResults;
+    if (query && !enabledCollections.isEmpty()) {
+        auto &[promptIndex, queryStr] = *query;
+        const int retrievalSize = MySettings::globalInstance()->localDocsRetrievalSize();
+        emit requestRetrieveFromDB(enabledCollections, queryStr, retrievalSize, &databaseResults); // blocks
+        m_chatModel->updateSources(promptIndex, databaseResults);
+        emit databaseResultsChanged(databaseResults);
+    }
+
     auto result = promptInternal(chatItems, ctx, !databaseResults.isEmpty());
     return {
         /*PromptResult*/ {
@@ -933,7 +938,7 @@ auto ChatLLM::promptInternal(
         }
     }
 
-    PromptResult result;
+    PromptResult result {};
 
     auto handlePrompt = [this, &result](std::span<const LLModel::Token> batch, bool cached) -> bool {
         Q_UNUSED(cached)

gpt4all-chat/src/chatllm.h

@@ -250,7 +250,8 @@ protected:
         QList<ResultInfo> databaseResults;
     };
 
-    ChatPromptResult promptInternalChat(const QStringList &enabledCollections, const LLModel::PromptContext &ctx);
+    ChatPromptResult promptInternalChat(const QStringList &enabledCollections, const LLModel::PromptContext &ctx,
+                                        std::optional<QList<ChatItem>> chat = {});
     // passing a string_view directly skips templating and uses the raw string
     PromptResult promptInternal(const std::variant<std::span<const ChatItem>, std::string_view> &prompt,
                                 const LLModel::PromptContext &ctx,

gpt4all-chat/src/chatmodel.h

@@ -111,8 +111,13 @@ public:
     ChatItem(prompt_tag_t, const QString &value, const QList<PromptAttachment> &attachments = {})
         : name(u"Prompt: "_s), value(value), promptAttachments(attachments) {}
 
-    ChatItem(response_tag_t, bool isCurrentResponse = true)
-        : name(u"Response: "_s), isCurrentResponse(isCurrentResponse) {}
+    // A new response, to be filled in
+    ChatItem(response_tag_t)
+        : name(u"Response: "_s), isCurrentResponse(true) {}
+
+    // An existing response, from Server
+    ChatItem(response_tag_t, const QString &value)
+        : name(u"Response: "_s), value(value) {}
 
     Type type() const
     {
@@ -165,9 +170,9 @@ public:
 };
 Q_DECLARE_METATYPE(ChatItem)
 
-class ChatModelAccessor : public ranges::subrange<QList<ChatItem>::const_iterator> {
+class ChatModelAccessor : public std::span<const ChatItem> {
 private:
-    using Super = ranges::subrange<QList<ChatItem>::const_iterator>;
+    using Super = std::span<const ChatItem>;
 
 public:
     template <typename... T>
@@ -193,9 +198,6 @@ public:
         NameRole = Qt::UserRole + 1,
         ValueRole,
 
-        // prompts and responses
-        PeerRole,
-
         // prompts
         PromptAttachmentsRole,
 
@@ -222,38 +224,38 @@ public:
 
     /* a "peer" is a bidirectional 1:1 link between a prompt and the response that would cite its LocalDocs
      * sources. Return std::nullopt if there is none, which is possible for e.g. server chats. */
-    auto getPeerUnlocked(QList<ChatItem>::const_iterator item) const
-        -> std::optional<QList<ChatItem>::const_iterator>
+    static std::optional<qsizetype> getPeer(const ChatItem *arr, qsizetype size, qsizetype index)
     {
-        switch (item->type()) {
+        Q_ASSERT(index >= 0);
+        Q_ASSERT(index < size);
+        qsizetype peer;
+        ChatItem::Type expected;
+        switch (arr[index].type()) {
             using enum ChatItem::Type;
-        case Prompt:
-            {
-                auto peer = std::next(item);
-                if (peer < m_chatItems.cend() && peer->type() == Response)
-                    return peer;
-                break;
-            }
-        case Response:
-            {
-                if (item > m_chatItems.cbegin()) {
-                    if (auto peer = std::prev(item); peer->type() == Prompt)
-                        return peer;
-                }
-                break;
-            }
-        default:
-            throw std::invalid_argument("getPeer() called on item that is not a prompt or response");
+            case Prompt:   peer = index + 1; expected = Response; break;
+            case Response: peer = index - 1; expected = Prompt;   break;
+            default: throw std::invalid_argument("getPeer() called on item that is not a prompt or response");
         }
+        if (peer >= 0 && peer < size && arr[peer].type() == expected)
+            return peer;
         return std::nullopt;
     }
 
-    auto getPeerUnlocked(int index) const -> std::optional<int>
+    template <ranges::contiguous_range R>
+        requires std::same_as<ranges::range_value_t<R>, ChatItem>
+    static auto getPeer(R &&range, ranges::iterator_t<R> item) -> std::optional<ranges::iterator_t<R>>
     {
-        return getPeerUnlocked(m_chatItems.cbegin() + index)
-            .transform([&](auto &&i) { return i - m_chatItems.cbegin(); } );
+        auto begin = ranges::begin(range);
+        return getPeer(ranges::data(range), ranges::size(range), item - begin)
+            .transform([&](auto i) { return begin + i; });
     }
 
+    auto getPeerUnlocked(QList<ChatItem>::const_iterator item) const -> std::optional<QList<ChatItem>::const_iterator>
+    { return getPeer(m_chatItems, item); }
+
+    std::optional<qsizetype> getPeerUnlocked(qsizetype index) const
+    { return getPeer(m_chatItems.constData(), m_chatItems.size(), index); }
+
     QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override
     {
         QMutexLocker locker(&m_mutex);
@@ -266,18 +268,6 @@ public:
                 return item->name;
             case ValueRole:
                 return item->value;
-            case PeerRole:
-                switch (item->type()) {
-                    using enum ChatItem::Type;
-                case Prompt:
-                case Response:
-                    {
-                        auto peer = getPeerUnlocked(item);
-                        return peer ? QVariant::fromValue(**peer) : QVariant::fromValue(nullptr);
-                    }
-                default:
-                    return QVariant();
-                }
             case PromptAttachmentsRole:
                 return QVariant::fromValue(item->promptAttachments);
             case SourcesRole:
@@ -320,7 +310,6 @@ public:
         return {
             { NameRole,                "name"                },
             { ValueRole,               "value"               },
-            { PeerRole,                "peer"                },
             { PromptAttachmentsRole,   "promptAttachments"   },
             { SourcesRole,             "sources"             },
             { ConsolidatedSourcesRole, "consolidatedSources" },
@@ -362,41 +351,39 @@ public:
             count = m_chatItems.count();
         }
 
-        int promptIndex = 0;
         beginInsertRows(QModelIndex(), count, count);
         {
             QMutexLocker locker(&m_mutex);
-            m_chatItems.emplace_back(ChatItem::response_tag, promptIndex);
-            if (auto pi = getPeerUnlocked(m_chatItems.size() - 1))
-                promptIndex = *pi;
+            m_chatItems.emplace_back(ChatItem::response_tag);
         }
         endInsertRows();
         emit countChanged();
-        if (promptIndex >= 0)
-            emit dataChanged(createIndex(promptIndex, 0), createIndex(promptIndex, 0), {PeerRole});
     }
 
     // Used by Server to append a new conversation to the chat log.
-    void appendResponseWithHistory(std::span<const ChatItem> history)
+    // Appends a new, blank response to the end of the input list.
+    void appendResponseWithHistory(QList<ChatItem> &history)
     {
         if (history.empty())
             throw std::invalid_argument("at least one message is required");
 
+        // add an empty response to prepare for generation
+        history.emplace_back(ChatItem::response_tag);
+
         m_mutex.lock();
-        qsizetype startIndex = m_chatItems.count();
+        qsizetype startIndex = m_chatItems.size();
         m_mutex.unlock();
 
-        qsizetype nNewItems = history.size() + 1;
-        qsizetype endIndex  = startIndex + nNewItems;
+        qsizetype endIndex = startIndex + history.size();
         beginInsertRows(QModelIndex(), startIndex, endIndex - 1 /*inclusive*/);
         bool hadError;
+        QList<ChatItem> newItems;
         {
             QMutexLocker locker(&m_mutex);
             hadError = hasErrorUnlocked();
-            m_chatItems.reserve(m_chatItems.count() + nNewItems);
+            m_chatItems.reserve(m_chatItems.size() + history.size());
             for (auto &item : history)
                 m_chatItems << item;
-            m_chatItems.emplace_back(ChatItem::response_tag);
         }
         endInsertRows();
         emit countChanged();

gpt4all-chat/src/modellist.cpp

@@ -53,7 +53,9 @@ static const QStringList FILENAME_BLACKLIST { u"gpt4all-nomic-embed-text-v1.rmod
 static const QString RMODEL_CHAT_TEMPLATE = uR"(<chat>
 {%- set loop_messages = messages %}
 {%- for message in loop_messages %}
-    {{- raise_exception('Unknown role: ' + messages['role']) }}
+    {%- if not message['role'] in ['user', 'assistant', 'system'] %}
+        {{- raise_exception('Unknown role: ' + message['role']) }}
+    {%- endif %}
     {{- '<' + message['role'] + '>' }}
     {%- if message['role'] == 'user' %}
         {%- for source in message.sources %}
@@ -1139,18 +1141,20 @@ bool ModelList::isUniqueName(const QString &name) const
 
 QString ModelList::clone(const ModelInfo &model)
 {
+    auto *mySettings = MySettings::globalInstance();
+
     const QString id = Network::globalInstance()->generateUniqueId();
     addModel(id);
 
-    QString chatTemplate, systemMessage;
+    QString tmplSetting, sysmsgSetting;
     if (auto tmpl = model.chatTemplate().asModern()) {
-        chatTemplate = *tmpl;
+        tmplSetting = *tmpl;
     } else {
         qWarning("ModelList Warning: attempted to clone model with legacy chat template");
         return {};
     }
     if (auto msg = model.systemMessage().asModern()) {
-        systemMessage = *msg;
+        sysmsgSetting = *msg;
     } else {
         qWarning("ModelList Warning: attempted to clone model with legacy system message");
         return {};
@@ -1175,12 +1179,22 @@ QString ModelList::clone(const ModelInfo &model)
         { ModelList::GpuLayersRole, model.gpuLayers() },
         { ModelList::RepeatPenaltyRole, model.repeatPenalty() },
         { ModelList::RepeatPenaltyTokensRole, model.repeatPenaltyTokens() },
-        { ModelList::ChatTemplateRole,  chatTemplate  },
-        { ModelList::SystemMessageRole, systemMessage },
+        { ModelList::SystemMessageRole, model.m_systemMessage },
         { ModelList::ChatNamePromptRole, model.chatNamePrompt() },
         { ModelList::SuggestedFollowUpPromptRole, model.suggestedFollowUpPrompt() },
     };
+    if (auto tmpl = model.m_chatTemplate)
+        data.emplace_back(ModelList::ChatTemplateRole, *tmpl); // copy default chat template, if known
     updateData(id, data);
+
+    // Ensure setting overrides are copied in case the base model overrides change.
+    // This is necessary because setting these roles on ModelInfo above does not write to settings.
+    auto cloneInfo = modelInfo(id);
+    if (mySettings->isModelChatTemplateSet (model))
+        mySettings->setModelChatTemplate (cloneInfo, tmplSetting  );
+    if (mySettings->isModelSystemMessageSet(model))
+        mySettings->setModelSystemMessage(cloneInfo, sysmsgSetting);
+
     return id;
 }
 

gpt4all-chat/src/server.cpp

@@ -771,13 +771,13 @@ auto Server::handleChatRequest(const ChatRequest &request)
     Q_ASSERT(!request.messages.isEmpty());
 
     // adds prompt/response items to GUI
-    std::vector<ChatItem> chatItems;
+    QList<ChatItem> chatItems;
     for (auto &message : request.messages) {
         using enum ChatRequest::Message::Role;
         switch (message.role) {
-            case System:    chatItems.emplace_back(ChatItem::system_tag, message.content); break;
-            case User:      chatItems.emplace_back(ChatItem::prompt_tag, message.content); break;
-            case Assistant: chatItems.emplace_back(ChatItem::response_tag, /*currentResponse*/ false); break;
+            case System:    chatItems.emplace_back(ChatItem::system_tag,   message.content); break;
+            case User:      chatItems.emplace_back(ChatItem::prompt_tag,   message.content); break;
+            case Assistant: chatItems.emplace_back(ChatItem::response_tag, message.content); break;
         }
     }
     m_chatModel->appendResponseWithHistory(chatItems);
@@ -800,7 +800,7 @@ auto Server::handleChatRequest(const ChatRequest &request)
     for (int i = 0; i < request.n; ++i) {
         ChatPromptResult result;
         try {
-            result = promptInternalChat(m_collections, promptCtx);
+            result = promptInternalChat(m_collections, promptCtx, chatItems);
         } catch (const std::exception &e) {
             emit responseChanged(e.what());
             emit responseStopped(0);

gpt4all-chat/tests/python/test_server_api.py

@@ -252,8 +252,8 @@ def test_with_models(chat_server_with_model: None) -> None:
     assert response == EXPECTED_COMPLETIONS_RESPONSE
 
 
-@pytest.mark.xfail(reason='Assertion failure in GPT4All. See nomic-ai/gpt4all#3133')
 def test_with_models_temperature(chat_server_with_model: None) -> None:
+    """Fixed by nomic-ai/gpt4all#3202."""
     data = {
         'model': 'Llama 3.2 1B Instruct',
         'prompt': 'The quick brown fox',