python: bump bindings version for AMD fixes

Update to llama.cpp
Bump version to v2.5.3 and release notes.
2026-07-17 10:58:08 +00:00 · 2023-10-30 17:00:05 -04:00 · 2023-10-30 11:40:16 -04:00 · 2023-10-30 11:15:12 -04:00 · 2023-10-30 10:37:32 -04:00 · 2023-10-30 10:37:05 -04:00
62 changed files with 280 additions and 1222 deletions
--- a/.codespellrc
+++ b/.codespellrc
@@ -1,3 +1,3 @@
 [codespell]
-ignore-words-list = blong, belong, afterall, som, assistent
+ignore-words-list = blong, afterall, som, assistent, crasher
 skip = .git,*.pdf,*.svg,*.lock
--- a/README.md
+++ b/README.md
@@ -1,11 +1,9 @@
 <h1 align="center">GPT4All</h1>

-<p align="center">Open-source assistant-style large language models that run locally on your CPU</p>
-
-<p align="center"><strong>New</strong>: Now with Nomic Vulkan Universal GPU support. <a href="https://blog.nomic.ai/posts/gpt4all-gpu-inference-with-vulkan">Learn more</a>.</p>
+<p align="center">Open-source large language models that run locally on your CPU and nearly any GPU</p>

 <p align="center">
-<a href="https://gpt4all.io">GPT4All Website</a>
+<a href="https://gpt4all.io">GPT4All Website and Models</a>
 </p>

 <p align="center">
@@ -32,13 +30,24 @@ Run on an M1 macOS Device (not sped up!)
 </p>

 ## GPT4All: An ecosystem of open-source on-edge large language models.
-GPT4All is an ecosystem to train and deploy **powerful** and **customized** large language models that run locally on consumer grade CPUs. Note that your CPU needs to support [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions).
+
+> [!IMPORTANT]
+> GPT4All v2.5.0 and newer only supports models in GGUF format (.gguf). Models used with a previous version of GPT4All (.bin extension) will no longer work.
+
+GPT4All is an ecosystem to run **powerful** and **customized** large language models that work locally on consumer grade CPUs and any GPU. Note that your CPU needs to support [AVX or AVX2 instructions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions).

 Learn more in the [documentation](https://docs.gpt4all.io).

-The goal is simple - be the best instruction tuned assistant-style language model that any person or enterprise can freely use, distribute and build on.
+A GPT4All model is a 3GB - 8GB file that you can download and plug into the GPT4All open-source ecosystem software. **Nomic AI** supports and maintains this software ecosystem to enforce quality and security alongside spearheading the effort to allow any person or enterprise to easily train and deploy their own on-edge large language models.

-A GPT4All model is a 3GB - 8GB file that you can download and plug into the GPT4All open-source ecosystem software. **Nomic AI** supports and maintains this software ecosystem to enforce quality and security alongside spearheading the effort to allow any person or enterprise to easily train and deploy their own on-edge large language models. 
+### What's New ([Issue Tracker](https://github.com/orgs/nomic-ai/projects/2))
+- **October 19th, 2023**: GGUF Support Launches with Support for:
+    - Mistral 7b base model, an updated model gallery on [gpt4all.io](https://gpt4all.io), several new local code models including Rift Coder v1.5
+    - [Nomic Vulkan](https://blog.nomic.ai/posts/gpt4all-gpu-inference-with-vulkan) support for Q4_0, Q6 quantizations in GGUF.
+    - Offline build support for running old versions of the GPT4All Local LLM Chat Client.
+- **September 18th, 2023**: [Nomic Vulkan](https://blog.nomic.ai/posts/gpt4all-gpu-inference-with-vulkan) launches supporting local LLM inference on AMD, Intel, Samsung, Qualcomm and NVIDIA GPUs.
+- **August 15th, 2023**: GPT4All API launches allowing inference of local LLMs from docker containers.
+- **July 2023**: Stable support for LocalDocs, a GPT4All Plugin that allows you to privately and locally chat with your data.


 ### Chat Client
--- a/gpt4all-backend/CMakeLists.txt
+++ b/gpt4all-backend/CMakeLists.txt
@@ -102,10 +102,6 @@ foreach(BUILD_VARIANT IN LISTS BUILD_VARIANTS)
            gptj.cpp utils.h utils.cpp llmodel_shared.cpp llmodel_shared.h)
        prepare_target(gptj llama-mainline)

-        add_library(mpt-${BUILD_VARIANT} SHARED
-            mpt.cpp utils.h utils.cpp llmodel_shared.cpp llmodel_shared.h)
-        prepare_target(mpt llama-mainline)
-
        add_library(bert-${BUILD_VARIANT} SHARED
            bert.cpp utils.h utils.cpp llmodel_shared.cpp llmodel_shared.h)
        target_compile_definitions(bert-${BUILD_VARIANT} PRIVATE LLAMA_VERSIONS=>=3 LLAMA_DATE=999999)
--- a/gpt4all-backend/bert.cpp
+++ b/gpt4all-backend/bert.cpp
@@ -884,7 +884,7 @@ DLL_EXPORT bool magic_match(const char * fname) {
    if (!ctx_gguf)
        return false;

-    bool isValid = gguf_get_version(ctx_gguf) <= 2;
+    bool isValid = gguf_get_version(ctx_gguf) <= 3;
    isValid = isValid && get_arch_name(ctx_gguf) == "bert";

    gguf_free(ctx_gguf);
--- a/gpt4all-backend/gptj.cpp
+++ b/gpt4all-backend/gptj.cpp
@@ -806,7 +806,7 @@ DLL_EXPORT bool magic_match(const char * fname) {
    if (!ctx_gguf)
        return false;

-    bool isValid = gguf_get_version(ctx_gguf) <= 2;
+    bool isValid = gguf_get_version(ctx_gguf) <= 3;
    isValid = isValid && get_arch_name(ctx_gguf) == "gptj";

    gguf_free(ctx_gguf);
--- a/gpt4all-backend/llama.cpp-mainline
+++ b/gpt4all-backend/llama.cpp-mainline
--- a/gpt4all-backend/llamamodel.cpp
+++ b/gpt4all-backend/llamamodel.cpp
@@ -395,9 +395,9 @@ DLL_EXPORT bool magic_match(const char * fname) {
    if (!ctx_gguf)
        return false;

-    bool isValid = gguf_get_version(ctx_gguf) <= 2;
+    bool isValid = gguf_get_version(ctx_gguf) <= 3;
    auto arch = get_arch_name(ctx_gguf);
-    isValid = isValid && (arch == "llama" || arch == "starcoder" || arch == "falcon");
+    isValid = isValid && (arch == "llama" || arch == "starcoder" || arch == "falcon" || arch == "mpt");

    gguf_free(ctx_gguf);
    return isValid;
--- a/gpt4all-backend/llmodel.cpp
+++ b/gpt4all-backend/llmodel.cpp
@@ -10,6 +10,7 @@
 #include <cassert>
 #include <cstdlib>
 #include <sstream>
+#include <regex>
 #ifdef _MSC_VER
 #include <intrin.h>
 #endif
@@ -81,6 +82,13 @@ const std::vector<LLModel::Implementation> &LLModel::Implementation::implementat
    static auto* libs = new std::vector<Implementation>([] () {
        std::vector<Implementation> fres;

+        std::string impl_name_re = "(bert|llama|gptj|llamamodel-mainline)";
+        if (requires_avxonly()) {
+            impl_name_re += "-avxonly";
+        } else {
+            impl_name_re += "-(default|metal)";
+        }
+        std::regex re(impl_name_re);
        auto search_in_directory = [&](const std::string& paths) {
            std::stringstream ss(paths);
            std::string path;
@@ -90,7 +98,10 @@ const std::vector<LLModel::Implementation> &LLModel::Implementation::implementat
                // Iterate over all libraries
                for (const auto& f : std::filesystem::directory_iterator(fs_path)) {
                    const std::filesystem::path& p = f.path();
+
                    if (p.extension() != LIB_FILE_EXT) continue;
+                    if (!std::regex_search(p.stem().string(), re)) continue;
+
                    // Add to list if model implementation
                    try {
                        Dlhandle dl(p.string());
--- a/gpt4all-backend/mpt.cpp
+++ b/gpt4all-backend/mpt.cpp
@@ -1,969 +0,0 @@
-#define MPT_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
-#include "mpt_impl.h"
-
-#include "utils.h"
-#include "llmodel_shared.h"
-
-#include <cassert>
-#include <cinttypes>
-#include <cmath>
-#include <cstdio>
-#include <cstring>
-#include <map>
-#include <random>
-#include <string>
-#include <vector>
-#include <iostream>
-#if defined(_WIN32) && defined(_MSC_VER)
-    #define WIN32_LEAN_AND_MEAN
-    #ifndef NOMINMAX
-        #define NOMINMAX
-    #endif
-    #include <windows.h>
-    #include <io.h>
-    #include <stdio.h>
-#else
-    #include <unistd.h>
-#endif
-#include <sstream>
-#include <thread>
-#include <unordered_set>
-#include <unordered_map>
-#include <regex>
-#include <ggml.h>
-
-
-namespace {
-const char *modelType_ = "MPT";
-}
-
-// default hparams (MPT 7B)
-struct mpt_hparams {
-    int32_t n_vocab      = 50432;
-    int32_t n_ctx        = 2048;
-    int32_t n_embd       = 4096;
-    int32_t n_head       = 32;
-    int32_t n_layer      = 32;
-    float alibi_bias_max = 8;
-    float clip_qkv       = 0;
-    float norm_eps       = 1e-5;
-    int32_t expand       = 4;
-};
-
-struct mpt_layer {
-    // normalization
-    struct ggml_tensor * norm_1_w;
-    struct ggml_tensor * norm_2_w;
-
-    // attention
-    struct ggml_tensor * attn_Wqkv_w;
-    struct ggml_tensor * attn_out_proj_w;
-
-    // ff
-    struct ggml_tensor * ffn_up_proj_w;
-    struct ggml_tensor * ffn_down_proj_w;
-};
-
-struct mpt_model {
-    mpt_hparams hparams;
-
-    // normalization
-    struct ggml_tensor * norm_f_w;
-
-    struct ggml_tensor * wte; // position embedding
-
-    // mpt does weight tying
-
-    std::vector<mpt_layer> layers;
-
-    struct llm_kv_cache kv_self;
-    struct ggml_context * ctx;
-
-
-    llm_buffer eval_buf;
-    llm_buffer scr0_buf;
-    llm_buffer scr1_buf;
-
-    ~mpt_model() {
-        if (ctx) {
-            ggml_free(ctx);
-        }
-    }
-};
-
-enum mpt_token_type {
-    MPT_TOKEN_TYPE_NORMAL  = 1,
-    MPT_TOKEN_TYPE_CONTROL = 3,
-};
-
-using replit_piece_t = std::pair<std::size_t, float>;
-using replit_piece_map_t = std::unordered_map<std::string, replit_piece_t>;
-
-static const std::string replit_ws_symbol = "\342\226\201";
-
-struct mpt_vocab {
-    bool is_replit = false;
-    gpt_vocab raw;
-    replit_piece_map_t piece_map;
-    std::vector<std::string> vocab;
-
-    const char * end_of_text() const {
-        return is_replit ? "<|endoftext|>" : "<|im_end|>";
-    }
-};
-
-std::pair<std::vector<LLModel::Token>, float> encode_word(const std::string & word, const replit_piece_map_t & model) {
-    std::vector<int> best_segmentations_starts(word.length() + 1, -1);
-    best_segmentations_starts[0] = 0;
-
-    std::vector<float> best_segmentations_scores(word.length() + 1, -std::numeric_limits<float>::infinity());
-    best_segmentations_scores[0] = 1.0;
-
-    for (size_t start_idx = 0; start_idx < word.length(); ++start_idx) {
-        float best_score_at_start = best_segmentations_scores[start_idx];
-        for (size_t end_idx = start_idx + 1; end_idx <= word.length(); ++end_idx) {
-            std::string token = word.substr(start_idx, end_idx - start_idx);
-            if (model.count(token) && best_score_at_start != -std::numeric_limits<float>::infinity()) {
-                float token_score = model.at(token).second;
-                float score = token_score + best_score_at_start;
-                if (best_segmentations_scores[end_idx] == -std::numeric_limits<float>::infinity() ||
-                    best_segmentations_scores[end_idx] > score) {
-                    best_segmentations_starts[end_idx] = start_idx;
-                    best_segmentations_scores[end_idx] = score;
-                }
-            }
-        }
-    }
-
-    if (best_segmentations_scores.back() == -std::numeric_limits<float>::infinity()) {
-        return std::make_pair(std::vector<LLModel::Token>{0}, 0.0f);
-    }
-
-    float score = best_segmentations_scores.back();
-    int start = best_segmentations_starts.back();
-    int end = word.length();
-    std::vector<LLModel::Token> tokens;
-    while (start != 0) {
-        const auto token_id = model.at(word.substr(start, end - start)).first;
-        tokens.insert(tokens.begin(), token_id);
-        int next_start = best_segmentations_starts[start];
-        end = start;
-        start = next_start;
-    }
-    const auto token_id = model.at(word.substr(start, end - start)).first;
-    tokens.insert(tokens.begin(), token_id);
-    return std::make_pair(tokens, score);
-}
-
-bool replit_tokenizer_load(mpt_vocab & tokenizer, gguf_context * ggufctx, int tokens_keyidx, int max_vocab_size) {
-    int scores_keyidx = gguf_find_key(ggufctx, "tokenizer.ggml.scores");
-    if (scores_keyidx == -1) {
-        fprintf(stderr, "%s: llama token scores not found!\n", __func__);
-        return false;
-    }
-    const auto *scores = reinterpret_cast<const float *>(gguf_get_arr_data(ggufctx, scores_keyidx));
-
-    for (LLModel::Token i = 0; i < max_vocab_size; i++) {
-        std::string word = gguf_get_arr_str(ggufctx, tokens_keyidx, i);
-        tokenizer.piece_map[word] = std::make_pair(i, -scores[i]);
-        tokenizer.raw.id_to_token[i] = word;
-        tokenizer.raw.token_to_id[word] = i;
-    }
-
-    return true;
-}
-
-std::string replace_all(const std::string & str,    // where to work
-                        const std::string & find,   // substitute 'find'
-                        const std::string & replace //      by 'replace'
-) {
-    std::string result;
-    size_t find_len = find.size();
-    size_t pos, from = 0;
-    while (std::string::npos != (pos = str.find(find, from))) {
-        result.append(str, from, pos - from);
-        result.append(replace);
-        from = pos + find_len;
-    }
-    result.append(str, from, std::string::npos);
-    return result;
-}
-
-std::vector<LLModel::Token> replit_tokenizer_tokenize(mpt_vocab & tokenizer, const std::string & text) {
-    std::vector<LLModel::Token> tokens;
-    auto normalized_text = replace_all(text, " ", replit_ws_symbol);
-    auto tokenized = encode_word(normalized_text, tokenizer.piece_map);
-
-    return tokenized.first;
-}
-
-std::string replit_tokenizer_detokenize(mpt_vocab & tokenizer, const std::vector<LLModel::Token> & tokens) {
-    std::string text;
-    for (auto token : tokens) {
-        text += tokenizer.raw.id_to_token[token];
-    }
-    return replace_all(text, replit_ws_symbol, " ");
-}
-
-
-static bool kv_cache_init(
-        const struct mpt_hparams & hparams,
-             struct llm_kv_cache & cache,
-                         ggml_type   wtype,
-                               int   n_ctx) {
-    const int n_embd  = hparams.n_embd;
-    const int n_layer = hparams.n_layer;
-
-    const int64_t n_mem      = (int64_t)n_layer*n_ctx;
-    const int64_t n_elements = n_embd*n_mem;
-
-    cache.buf.resize(2u*n_elements*ggml_type_size(wtype) + 2_MiB);
-
-    struct ggml_init_params params;
-    params.mem_size   = cache.buf.size;
-    params.mem_buffer = cache.buf.addr;
-    params.no_alloc   = false;
-
-    cache.ctx = ggml_init(params);
-
-    if (!cache.ctx) {
-        fprintf(stderr, "%s: failed to allocate memory for kv cache\n", __func__);
-        return false;
-    }
-
-    cache.k = ggml_new_tensor_1d(cache.ctx, wtype, n_elements);
-    cache.v = ggml_new_tensor_1d(cache.ctx, wtype, n_elements);
-
-    return true;
-}
-
-// load the model's weights from a file path. if mem_req ptr is passed the model is
-// only partially parsed to estimate required memory
-bool mpt_model_load(const std::string &fname, mpt_model & model, mpt_vocab & vocab, size_t * mem_req) {
-    printf("%s: loading model from '%s' - please wait ...\n", __func__, fname.c_str());
-    if (mem_req != nullptr) {
-        *mem_req = 0;
-    }
-
-    // create the ggml context
-    struct gguf_init_params params = {
-        /*.no_alloc = */ false,
-        /*.ctx      = */ &model.ctx,
-    };
-
-    gguf_context *ggufctx = gguf_init_from_file(fname.c_str(), params);
-    if (!ggufctx) {
-        fprintf(stderr, "%s: gguf_init_from_file() failed\n", __func__);
-        return false;
-    }
-
-    printf("%s: gguf version     = %d\n", __func__, gguf_get_version(ggufctx));
-    printf("%s: gguf alignment   = %zu\n", __func__, gguf_get_alignment(ggufctx));
-    printf("%s: gguf data offset = %zu\n", __func__, gguf_get_data_offset(ggufctx));
-
-    // print some standard metadata
-    {
-        int keyidx;
-
-        keyidx = gguf_find_key(ggufctx, "general.name");
-        if (keyidx != -1) { printf("%s: model name           = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.description");
-        if (keyidx != -1) { printf("%s: model description    = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.author");
-        if (keyidx != -1) { printf("%s: model author         = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.license");
-        if (keyidx != -1) { printf("%s: model license        = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.architecture");
-        if (keyidx != -1) { printf("%s: model architecture   = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.file_type");
-        if (keyidx != -1) { printf("%s: model file type      = %" PRIu32 "\n", __func__, gguf_get_val_u32(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "gptneox.tensor_data_layout");
-        if (keyidx != -1) { printf("%s: model data layout    = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-        keyidx = gguf_find_key(ggufctx, "general.source.huggingface.repository");
-        if (keyidx != -1) { printf("%s: model source HF repo = %s\n", __func__, gguf_get_val_str(ggufctx, keyidx)); }
-    }
-
-    // check required metadata
-    {
-        // check model architecture kv
-        int keyidx = gguf_find_key(ggufctx, "general.architecture");
-        if (keyidx == -1) {
-            fprintf(stderr, "%s: gguf model architecture not found!\n", __func__);
-            return false;
-        }
-        if (strcmp(gguf_get_val_str(ggufctx, keyidx), "mpt") != 0) {
-            fprintf(stderr, "%s: model architecture not supported!\n", __func__);
-            return false;
-        }
-    }
-
-    // load hparams
-    {
-        auto & hparams = model.hparams;
-
-        bool ok = false;
-        int keyidx;
-
-        do {
-            keyidx = gguf_find_key(ggufctx, "mpt.context_length");
-            if (keyidx == -1) { break; }
-            hparams.n_ctx = gguf_get_val_u32(ggufctx, keyidx);
-
-            keyidx = gguf_find_key(ggufctx, "mpt.embedding_length");
-            if (keyidx == -1) { break; }
-            hparams.n_embd = gguf_get_val_u32(ggufctx, keyidx);
-
-            keyidx = gguf_find_key(ggufctx, "mpt.attention.head_count");
-            if (keyidx == -1) { break; }
-            hparams.n_head = gguf_get_val_u32(ggufctx, keyidx);
-
-            keyidx = gguf_find_key(ggufctx, "mpt.block_count");
-            if (keyidx == -1) { break; }
-            hparams.n_layer = gguf_get_val_u32(ggufctx, keyidx);
-
-            keyidx = gguf_find_key(ggufctx, "mpt.attention.max_alibi_bias");
-            if (keyidx == -1) { break; }
-            hparams.alibi_bias_max = gguf_get_val_f32(ggufctx, keyidx);
-
-            keyidx = gguf_find_key(ggufctx, "mpt.attention.clamp_kqv");
-            if (keyidx != -1) { // optional
-                hparams.clip_qkv = gguf_get_val_f32(ggufctx, keyidx);
-            }
-
-            keyidx = gguf_find_key(ggufctx, "mpt.attention.layer_norm_epsilon");
-            if (keyidx == -1) { break; }
-            hparams.norm_eps = gguf_get_val_f32(ggufctx, keyidx);
-
-            ok = true;
-        } while (false);
-
-        if (!ok) {
-            fprintf(stderr, "%s: required hparam missing!\n", __func__);
-            return false;
-        }
-
-        printf("%s: n_ctx          = %d\n", __func__, hparams.n_ctx);
-        printf("%s: n_embd         = %d\n", __func__, hparams.n_embd);
-        printf("%s: n_head         = %d\n", __func__, hparams.n_head);
-        printf("%s: n_layer        = %d\n", __func__, hparams.n_layer);
-        printf("%s: alibi_bias_max = %f\n", __func__, hparams.alibi_bias_max);
-        printf("%s: clip_qkv       = %f\n", __func__, hparams.clip_qkv);
-    }
-
-    // load vocab
-    {
-        auto & hparams = model.hparams;
-
-        int tokens_keyidx = gguf_find_key(ggufctx, "tokenizer.ggml.tokens");
-        if (tokens_keyidx == -1) {
-            fprintf(stderr, "%s: tokenizer vocab not found!\n", __func__);
-            return false;
-        }
-
-        int keyidx = gguf_find_key(ggufctx, "tokenizer.ggml.model");
-        if (keyidx == -1) {
-            fprintf(stderr, "%s: tokenizer model not found!\n", __func__);
-            return false;
-        }
-        std::string tokenizer_model(gguf_get_val_str(ggufctx, keyidx));
-
-        hparams.n_vocab = gguf_get_arr_n(ggufctx, tokens_keyidx);
-        printf("%s: %s tokenizer vocab = %d\n", __func__, tokenizer_model.c_str(), int(hparams.n_vocab));
-
-        if (tokenizer_model == "llama") { // Replit
-            vocab.is_replit = true;
-            if (!replit_tokenizer_load(vocab, ggufctx, tokens_keyidx, hparams.n_vocab)) {
-                return false;
-            }
-        } else if (tokenizer_model == "gpt2") {
-            int toktypes_keyidx = gguf_find_key(ggufctx, "tokenizer.ggml.token_type");
-            if (toktypes_keyidx == -1) {
-                fprintf(stderr, "%s: gpt2 token types not found!\n", __func__);
-                return false;
-            }
-            const auto *toktypes = reinterpret_cast<const uint32_t *>(gguf_get_arr_data(ggufctx, toktypes_keyidx));
-
-            for (int i = 0; i < hparams.n_vocab; i++) {
-                std::string word = gguf_get_arr_str(ggufctx, tokens_keyidx, i);
-
-                bool special = false;
-                if (toktypes[i] == MPT_TOKEN_TYPE_CONTROL) {
-                    special = true;
-                } else if (toktypes[i] != MPT_TOKEN_TYPE_NORMAL) {
-                    fprintf(stderr, "%s: unknown token type: %d\n", __func__, int(toktypes[i]));
-                    return false;
-                }
-
-                vocab.raw.token_to_id[word] = i;
-                vocab.raw.id_to_token[i] = word;
-
-                if (special) {
-                    vocab.raw.add_special_token(word);
-                }
-            }
-        } else {
-            fprintf(stderr, "%s: tokenizer model not supported!\n", __func__);
-            return false;
-        }
-    }
-
-    auto & ctx = model.ctx;
-
-    size_t ctx_size = ggml_get_mem_size(ctx);
-    printf("%s: ggml ctx size = %6.2f MB\n", __func__, ctx_size / (1024.0 * 1024.0));
-
-    if (mem_req != nullptr) {
-        *mem_req = ctx_size;
-        gguf_free(ggufctx);
-        return false;
-    }
-
-    // prepare memory for the weights
-    {
-        const auto & hparams = model.hparams;
-        model.layers.resize(hparams.n_layer);
-
-        model.wte      = ggml_get_tensor(ctx, "token_embd.weight");
-        model.norm_f_w = ggml_get_tensor(ctx, "output_norm.weight");
-
-        auto name = [](int i, std::string n) {
-            static std::string key;
-            key = "blk." + std::to_string(i) + "." + n;
-            return key.c_str();
-        };
-
-        for (int i = 0; i < hparams.n_layer; ++i) {
-            auto &layer = model.layers[i];
-
-            layer.norm_1_w        = ggml_get_tensor(ctx, name(i, "attn_norm.weight"));
-            layer.norm_2_w        = ggml_get_tensor(ctx, name(i, "ffn_norm.weight"));
-
-            layer.attn_Wqkv_w     = ggml_get_tensor(ctx, name(i, "attn_qkv.weight"));
-            layer.attn_out_proj_w = ggml_get_tensor(ctx, name(i, "attn_output.weight"));
-            layer.ffn_up_proj_w   = ggml_get_tensor(ctx, name(i, "ffn_up.weight"));
-            layer.ffn_down_proj_w = ggml_get_tensor(ctx, name(i, "ffn_down.weight"));
-        }
-    }
-
-    // key + value memory
-    {
-        const auto &hparams = model.hparams;
-        if (!kv_cache_init(hparams, model.kv_self, GGML_TYPE_F16, model.hparams.n_ctx)) {
-            fprintf(stderr, "%s: kv_cache_init() failed for self-attention cache\n", __func__);
-            ggml_free(ctx);
-            return false;
-        }
-
-        const size_t memory_size = ggml_nbytes(model.kv_self.k) + ggml_nbytes(model.kv_self.v);
-        printf("%s: kv self size  = %7.2f MB\n", __func__, memory_size / 1024.0 / 1024.0);
-    }
-
-    model.scr0_buf.resize(256u * 1024 * 1024);
-    model.scr1_buf.resize(256u * 1024 * 1024);
-
-    return true;
-}
-
-bool mpt_eval(
-        mpt_model & model,
-        const int n_threads,
-        const int n_past,
-        const std::vector<int>           & embd_inp,
-              std::vector<float>         & embd_w,
-              size_t                     & mem_per_token) {
-    const int N = embd_inp.size();
-
-    const auto & hparams = model.hparams;
-
-    const int n_embd  = hparams.n_embd;
-    const int n_layer = hparams.n_layer;
-    const int n_ctx   = hparams.n_ctx;
-    const int n_head  = hparams.n_head;
-    const int n_vocab = hparams.n_vocab;
-
-    const size_t init_buf_size = 1024_MiB;
-    if (!model.eval_buf.addr || model.eval_buf.size < init_buf_size)
-        model.eval_buf.resize(init_buf_size);
-
-    if (mem_per_token > 0 && mem_per_token*N > model.eval_buf.size) {
-        const size_t buf_size_new = 1.1*(mem_per_token*N); // add 10% to account for ggml object overhead
-        // printf("\n%s: reallocating buffer from %zu to %zu bytes\n", __func__, model.buf.size, buf_size_new);
-
-        // reallocate
-        model.eval_buf.resize(buf_size_new);
-        if (model.eval_buf.addr == nullptr) {
-            fprintf(stderr, "%s: failed to allocate %zu bytes\n", __func__, model.eval_buf.size);
-            return false;
-        }
-    }
-
-    struct ggml_init_params params = {
-        .mem_size   = model.eval_buf.size,
-        .mem_buffer = model.eval_buf.addr,
-        .no_alloc = false
-    };
-
-    struct ggml_context * ctx0 = ggml_init(params);
-    struct ggml_cgraph gf = {};
-
-    struct ggml_tensor * embd = ggml_new_tensor_1d(ctx0, GGML_TYPE_I32, N);
-    memcpy(embd->data, embd_inp.data(), N*ggml_element_size(embd));
-
-    // wte
-    struct ggml_tensor * inpL = ggml_get_rows(ctx0, model.wte, embd);
-
-    for (int il = 0; il < n_layer; ++il) {
-        ggml_set_scratch(ctx0, {0, model.scr0_buf.size, model.scr0_buf.addr, });
-
-        struct ggml_tensor * inpSA = inpL;
-        struct ggml_tensor * cur = inpSA;
-        // self-attention
-        {
-
-            // norm1
-            cur = ggml_norm(ctx0, cur, model.hparams.norm_eps);
-            cur = ggml_mul(ctx0,
-                    ggml_repeat(ctx0, model.layers[il].norm_1_w, cur),
-                    cur);
-            // compute QKV
-            cur = ggml_mul_mat(ctx0,
-                    model.layers[il].attn_Wqkv_w,
-                    cur);
-
-            // TODO: clip_qkv
-            struct ggml_tensor * Qcur = ggml_cont(ctx0, ggml_view_2d(ctx0, cur, n_embd, N, cur->nb[1], 0*ggml_element_size(cur)*n_embd));
-            struct ggml_tensor * Kcur = ggml_cont(ctx0, ggml_view_2d(ctx0, cur, n_embd, N, cur->nb[1], 1*ggml_element_size(cur)*n_embd));
-            struct ggml_tensor * Vcur = ggml_cont(ctx0, ggml_view_2d(ctx0, cur, n_embd, N, cur->nb[1], 2*ggml_element_size(cur)*n_embd));
-
-            // TODO: qk_ln? (seems to be False in MPT-7B configs)
-            {
-                Vcur = ggml_transpose(ctx0, Vcur);
-
-                struct ggml_tensor * k = ggml_view_1d(ctx0, model.kv_self.k, N*n_embd, (ggml_element_size(model.kv_self.k)*n_embd)*(il*n_ctx + n_past));
-                struct ggml_tensor * v = ggml_view_2d(ctx0, model.kv_self.v, N, n_embd,
-                                        (   n_ctx)*ggml_element_size(model.kv_self.v),
-                                        (il*n_ctx)*ggml_element_size(model.kv_self.v)*n_embd + n_past*ggml_element_size(model.kv_self.v));
-
-                ggml_build_forward_expand(&gf, ggml_cpy(ctx0, Kcur, k));
-                ggml_build_forward_expand(&gf, ggml_cpy(ctx0, Vcur, v));
-            }
-            // Q = Qcur.contiguous().view(n_embd/n_head, n_head, N).permute(0, 2, 1, 3)
-            struct ggml_tensor * Q =
-                ggml_permute(ctx0,
-                        ggml_reshape_3d(ctx0, Qcur, n_embd/n_head, n_head, N),
-                        0, 2, 1, 3);
-
-            struct ggml_tensor * K =
-                ggml_permute(ctx0,
-                        ggml_reshape_3d(ctx0,
-                            ggml_view_1d(ctx0, model.kv_self.k, (n_past + N)*n_embd, il*n_ctx*ggml_element_size(model.kv_self.k)*n_embd),
-                            n_embd/n_head, n_head, n_past + N),
-                        0, 2, 1, 3);
-
-            // K * Q
-            struct ggml_tensor * KQ = ggml_mul_mat(ctx0, K, Q);
-
-            // KQ_scaled = KQ / sqrt(n_embd/n_head)
-            struct ggml_tensor * KQ_scaled =
-                ggml_scale(ctx0,
-                        KQ,
-                        ggml_new_f32(ctx0, 1.0f/sqrt(float(n_embd)/n_head))
-                        );
-
-
-            // Alibi
-            struct ggml_tensor * KQ_scaled_biased = ggml_alibi(
-                ctx0, ggml_cont(ctx0, KQ_scaled), n_past, n_head, model.hparams.alibi_bias_max
-            );
-
-            // KQ_masked = mask_past(KQ_scaled)
-            struct ggml_tensor * KQ_masked = ggml_diag_mask_inf(ctx0, KQ_scaled_biased, n_past);
-
-            // KQ = soft_max(KQ_masked)
-            struct ggml_tensor * KQ_soft_max = ggml_soft_max(ctx0, KQ_masked);
-
-            // V_trans = Vmem.view(n_embd/n_head, n_head, n_past + N).permute(1, 2, 0, 3).contiguous()
-            struct ggml_tensor * V =
-                ggml_view_3d(ctx0, model.kv_self.v,
-                        n_past + N, n_embd/n_head, n_head,
-                        n_ctx*ggml_element_size(model.kv_self.v),
-                        n_ctx*ggml_element_size(model.kv_self.v)*n_embd/n_head,
-                        il*n_ctx*ggml_element_size(model.kv_self.v)*n_embd);
-
-            // KQV = transpose(V) * KQ_soft_max
-            struct ggml_tensor * KQV = ggml_mul_mat(ctx0, V, KQ_soft_max);
-
-            // KQV_merged = KQV.permute(0, 2, 1, 3)
-            struct ggml_tensor * KQV_merged = ggml_permute(ctx0, KQV, 0, 2, 1, 3);
-
-            // cur = KQV_merged.contiguous().view(n_embd, N)
-            cur = ggml_cpy(ctx0,
-                    KQV_merged,
-                    ggml_new_tensor_2d(ctx0, GGML_TYPE_F32, n_embd, N));
-
-            // projection (no bias)
-            cur = ggml_mul_mat(ctx0,
-                    model.layers[il].attn_out_proj_w,
-                    cur);
-        }
-
-        ggml_set_scratch(ctx0, {0, model.scr1_buf.size, model.scr1_buf.addr, });
-        // residual
-        struct ggml_tensor * resSA = ggml_add(ctx0, cur, inpSA);
-        // feed-forward network
-        {
-            cur = resSA;
-            // norm2
-            cur = ggml_norm(ctx0, cur, model.hparams.norm_eps);
-            cur = ggml_mul(ctx0,
-                    ggml_repeat(ctx0, model.layers[il].norm_2_w, cur),
-                    cur);
-            // ffn
-            cur = ggml_mul_mat(ctx0,
-                    model.layers[il].ffn_up_proj_w,
-                    cur);
-            cur = ggml_gelu(ctx0, cur);
-            cur = ggml_mul_mat(ctx0,
-                    model.layers[il].ffn_down_proj_w,
-                    cur);
-
-        }
-
-        // self-attention + FF
-        inpL = ggml_add(ctx0, cur, resSA);
-    }
-    ggml_set_scratch(ctx0, {0, model.scr0_buf.size, model.scr0_buf.addr, });
-
-    struct ggml_tensor * out = inpL;
-    // -> logits
-    {
-        out = ggml_norm(ctx0, out, model.hparams.norm_eps);
-        out = ggml_mul(ctx0,
-                    ggml_repeat(ctx0, model.norm_f_w, out),
-                    out);
-        ggml_set_scratch(ctx0, { 0, 0, nullptr, });
-        out = ggml_mul_mat(ctx0, model.wte, out);
-    }
-
-    ggml_build_forward_expand(&gf, out);
-
-
-    // run the computation
-    {
-        std::unique_ptr<uint8_t []> data;
-        auto plan = ggml_graph_plan(&gf, n_threads);
-        if (plan.work_size > 0) {
-            data.reset(new uint8_t[plan.work_size]);
-            plan.work_data = data.get();
-        }
-        ggml_graph_compute(&gf, &plan);
-    }
-
-
-    // return result for just the last token
-    embd_w.resize(n_vocab);
-    memcpy(embd_w.data(), (float *) ggml_get_data(out) + (n_vocab*(N-1)), sizeof(float)*n_vocab);
-
-    if (mem_per_token == 0) {
-        mem_per_token = ggml_used_mem(ctx0)/N;
-    }
-    //printf("used_mem = %zu\n", ggml_used_mem(ctx0));
-
-    ggml_free(ctx0);
-
-    return true;
-}
-
-
-#define MPT_MAX_RNG_STATE 64*1024
-
-size_t mpt_get_state_size(const mpt_model &model)
-{
-    // we don't know size of rng until we actually serialize it. so reserve more than enough memory for its serialized state.
-    // for reference, std::mt19937(1337) serializes to 6701 bytes.
-    const size_t s_rng_size        = sizeof(size_t);
-    const size_t s_rng             = MPT_MAX_RNG_STATE;
-    const size_t s_kv_size         = sizeof(size_t);
-    const size_t s_kv_ntok         = sizeof(int);
-    const size_t s_kv              = model.kv_self.buf.size;
-    const size_t s_total = (
-        + s_rng_size
-        + s_rng
-        + s_kv_size
-        + s_kv_ntok
-        + s_kv
-    );
-    fflush(stdout);
-    return s_total;
-}
-
-size_t mpt_copy_state_data(const mpt_model &model, const std::mt19937 &rng, uint8_t *dest)
-{
-    uint8_t * out = dest;
-    fflush(stdout);
-    // copy rng
-    {
-        std::stringstream rng_ss;
-        rng_ss << rng;
-
-        const size_t rng_size = rng_ss.str().size();
-        char rng_buf[MPT_MAX_RNG_STATE];
-
-        memset(&rng_buf[0], 0, MPT_MAX_RNG_STATE);
-        memcpy(&rng_buf[0], rng_ss.str().data(), rng_ss.str().size());
-
-        memcpy(out, &rng_size,   sizeof(rng_size));   out += sizeof(rng_size);
-        memcpy(out, &rng_buf[0], MPT_MAX_RNG_STATE); out += MPT_MAX_RNG_STATE;
-    }
-
-    // copy kv cache
-    {
-        const size_t kv_size = model.kv_self.buf.size;
-        const int    kv_ntok = model.kv_self.n;
-
-        memcpy(out, &kv_size, sizeof(kv_size)); out += sizeof(kv_size);
-        memcpy(out, &kv_ntok, sizeof(kv_ntok)); out += sizeof(kv_ntok);
-
-        if (kv_size) {
-            memcpy(out, model.kv_self.buf.addr, kv_size); out += kv_size;
-        }
-    }
-
-    const size_t written  = out - dest;
-    assert(written == mpt_get_state_size(model));
-    fflush(stdout);
-    return written;
-}
-
-size_t mpt_set_state_data(mpt_model *model, std::mt19937 *rng, const uint8_t *src)
-{
-    const uint8_t * in = src;
-
-    // set rng
-    {
-        size_t rng_size;
-        char   rng_buf[MPT_MAX_RNG_STATE];
-
-        memcpy(&rng_size,   in, sizeof(rng_size));    in += sizeof(rng_size);
-        memcpy(&rng_buf[0], in, MPT_MAX_RNG_STATE); in += MPT_MAX_RNG_STATE;
-
-        std::stringstream rng_ss;
-        rng_ss.str(std::string(&rng_buf[0], rng_size));
-        rng_ss >> *rng;
-
-        assert(rng_ss.fail() == false);
-    }
-
-    // set kv cache
-    {
-        size_t kv_size;
-        int kv_ntok;
-
-        memcpy(&kv_size, in, sizeof(kv_size)); in += sizeof(kv_size);
-        memcpy(&kv_ntok, in, sizeof(kv_ntok)); in += sizeof(kv_ntok);
-
-        if (kv_size) {
-            assert(model->kv_self.buf.size == kv_size);
-
-            void * k_data = model->kv_self.k->data; // remember data pointers
-            void * v_data = model->kv_self.v->data; // because their value is stored in buf and overwritten by memcpy
-
-            memcpy(model->kv_self.buf.addr, in, kv_size); in += kv_size;
-
-            model->kv_self.k->data = k_data; // restore correct data pointers
-            model->kv_self.v->data = v_data;
-
-        }
-
-        model->kv_self.n = kv_ntok;
-    }
-
-    const size_t nread    = in - src;
-    assert(nread == mpt_get_state_size(*model));
-    fflush(stdout);
-    return nread;
-}
-
-struct MPTPrivate {
-    const std::string modelPath;
-    bool modelLoaded;
-    mpt_vocab vocab;
-    mpt_model *model = nullptr;
-    int64_t n_threads = 0;
-    size_t mem_per_token = 0;
-    std::mt19937 rng;
-    bool has_end_of_text = false;
-};
-
-MPT::MPT()
-    : d_ptr(new MPTPrivate) {
-    d_ptr->model = new mpt_model;
-    d_ptr->model->ctx = nullptr;
-    d_ptr->modelLoaded = false;
-}
-
-size_t MPT::requiredMem(const std::string &modelPath) {
-    mpt_model dummy_model;
-    mpt_vocab dummy_vocab;
-    size_t mem_req;
-    mpt_model_load(modelPath, dummy_model, dummy_vocab, &mem_req);
-    return mem_req;
-}
-
-bool MPT::loadModel(const std::string &modelPath) {
-    std::mt19937 rng(time(NULL));
-    d_ptr->rng = rng;
-
-    // load the model
-    if (!mpt_model_load(modelPath, *d_ptr->model, d_ptr->vocab, nullptr)) {
-        std::cerr << "MPT ERROR: failed to load model from " <<  modelPath;
-        return false;
-    }
-
-    d_ptr->n_threads = std::min(4, (int32_t) std::thread::hardware_concurrency());
-    d_ptr->modelLoaded = true;
-    const auto & vocab = d_ptr->vocab;
-    d_ptr->has_end_of_text = vocab.raw.token_to_id.find(vocab.end_of_text()) != vocab.raw.token_to_id.end();
-    fflush(stdout);
-    return true;
-}
-
-void MPT::setThreadCount(int32_t n_threads) {
-    d_ptr->n_threads = n_threads;
-}
-
-int32_t MPT::threadCount() const
-{
-    return d_ptr->n_threads;
-}
-
-MPT::~MPT()
-{
-    delete d_ptr->model;
-}
-
-bool MPT::isModelLoaded() const
-{
-    return d_ptr->modelLoaded;
-}
-
-size_t MPT::stateSize() const
-{
-    return mpt_get_state_size(*d_ptr->model);
-}
-
-size_t MPT::saveState(uint8_t *dest) const
-{
-    return mpt_copy_state_data(*d_ptr->model, d_ptr->rng, dest);
-}
-
-size_t MPT::restoreState(const uint8_t *src)
-{
-    return mpt_set_state_data(d_ptr->model, &d_ptr->rng, src);
-}
-
-std::vector<LLModel::Token> MPT::tokenize(PromptContext &, const std::string &str) const
-{
-    if (d_ptr->vocab.is_replit) {
-        return replit_tokenizer_tokenize(d_ptr->vocab, str);
-    }
-    return ::gpt_tokenize(d_ptr->vocab.raw, str);
-}
-
-std::string MPT::tokenToString(Token id) const
-{
-    if (d_ptr->vocab.is_replit) {
-        return replit_tokenizer_detokenize(d_ptr->vocab, {id});
-    }
-    return d_ptr->vocab.raw.id_to_token[id];
-}
-
-LLModel::Token MPT::sampleToken(PromptContext &promptCtx) const
-{
-    const size_t n_prev_toks = std::min((size_t) promptCtx.repeat_last_n, promptCtx.tokens.size());
-    return gpt_sample_top_k_top_p(d_ptr->model->hparams.n_vocab,
-        promptCtx.tokens.data() + promptCtx.tokens.size() - n_prev_toks,
-        n_prev_toks,
-        promptCtx.logits,
-        promptCtx.top_k, promptCtx.top_p, promptCtx.temp,
-        promptCtx.repeat_penalty,
-        d_ptr->rng);
-}
-
-bool MPT::evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const
-{
-    // determine the required inference memory per token:
-    static bool initialized = false;
-    if (!initialized) {
-        mpt_eval(*d_ptr->model, d_ptr->n_threads, 0, { 0, 1, 2, 3 }, ctx.logits,
-            d_ptr->mem_per_token);
-        initialized = true;
-    }
-
-    return mpt_eval(*d_ptr->model, d_ptr->n_threads, ctx.n_past, tokens, ctx.logits, d_ptr->mem_per_token);
-}
-
-int32_t MPT::contextLength() const
-{
-    return d_ptr->model->hparams.n_ctx;
-}
-
-const std::vector<LLModel::Token> &MPT::endTokens() const
-{
-    static std::vector<LLModel::Token> fres;
-    if (fres.empty()) {
-        fres = {0, d_ptr->vocab.raw.token_to_id[d_ptr->vocab.end_of_text()]};
-    }
-    return fres;
-}
-
-std::string get_arch_name(gguf_context *ctx_gguf) {
-    std::string arch_name;
-    const int kid = gguf_find_key(ctx_gguf, "general.architecture");
-    enum gguf_type ktype = gguf_get_kv_type(ctx_gguf, kid);
-    if (ktype != GGUF_TYPE_STRING) {
-        throw std::runtime_error("ERROR: Can't get general architecture from gguf file.");
-    }
-    return gguf_get_val_str(ctx_gguf, kid);
-}
-
-#if defined(_WIN32)
-#define DLL_EXPORT __declspec(dllexport)
-#else
-#define DLL_EXPORT __attribute__ ((visibility ("default")))
-#endif
-
-extern "C" {
-DLL_EXPORT bool is_g4a_backend_model_implementation() {
-    return true;
-}
-
-DLL_EXPORT const char *get_model_type() {
-    return modelType_;
-}
-
-DLL_EXPORT const char *get_build_variant() {
-    return GGML_BUILD_VARIANT;
-}
-
-DLL_EXPORT bool magic_match(const char * fname) {
-    struct ggml_context * ctx_meta = NULL;
-    struct gguf_init_params params = {
-        /*.no_alloc = */ true,
-        /*.ctx      = */ &ctx_meta,
-    };
-    gguf_context *ctx_gguf = gguf_init_from_file(fname, params);
-    if (!ctx_gguf)
-        return false;
-
-    bool isValid = gguf_get_version(ctx_gguf) <= 2;
-    isValid = isValid && get_arch_name(ctx_gguf) == "mpt";
-
-    gguf_free(ctx_gguf);
-    return isValid;
-}
-
-DLL_EXPORT LLModel *construct() {
-    return new MPT;
-}
-}
--- a/gpt4all-backend/mpt_impl.h
+++ b/gpt4all-backend/mpt_impl.h
@@ -1,41 +0,0 @@
-#ifndef MPT_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
-#error This file is NOT meant to be included outside of mpt.cpp. Doing so is DANGEROUS. Be sure to know what you are doing before proceeding to #define MPT_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
-#endif
-#ifndef MPT_H
-#define MPT_H
-
-#include <string>
-#include <functional>
-#include <vector>
-#include "llmodel.h"
-
-struct MPTPrivate;
-class MPT : public LLModel {
-public:
-    MPT();
-    ~MPT();
-
-    bool supportsEmbedding() const override { return false; }
-    bool supportsCompletion() const override { return true; }
-    bool loadModel(const std::string &modelPath) override;
-    bool isModelLoaded() const override;
-    size_t requiredMem(const std::string &modelPath) override;
-    size_t stateSize() const override;
-    size_t saveState(uint8_t *dest) const override;
-    size_t restoreState(const uint8_t *src) override;
-    void setThreadCount(int32_t n_threads) override;
-    int32_t threadCount() const override;
-
-private:
-    MPTPrivate *d_ptr;
-
-protected:
-    std::vector<Token> tokenize(PromptContext &, const std::string&) const override;
-    std::string tokenToString(Token) const override;
-    Token sampleToken(PromptContext &ctx) const override;
-    bool evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const override;
-    int32_t contextLength() const override;
-    const std::vector<Token>& endTokens() const override;
-};
-
-#endif // MPT_H
--- a/gpt4all-backend/scripts/convert_mpt_hf_to_gguf.py
+++ b/gpt4all-backend/scripts/convert_mpt_hf_to_gguf.py
@@ -18,7 +18,7 @@ from pathlib import Path
 import gguf
 import numpy as np
 import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig
+from transformers import AutoTokenizer, AutoModelForCausalLM, AutoConfig, MptConfig
 from transformers.models.gpt2 import tokenization_gpt2


@@ -30,7 +30,7 @@ if not 3 <= len(sys.argv) < 5:
    print("  ftype == 1 -> float16")
    sys.exit(1)

-model_name = sys.argv[1]
+dir_model = Path(sys.argv[1])
 dir_out = Path(sys.argv[2])

 # make sure the output directory exists
@@ -50,7 +50,7 @@ if len(sys.argv) > 3:
        print("Invalid ftype: " + str(ftype))
        sys.exit(1)

-fname_out = dir_out / f"ggml-model-{Path(model_name).name}-{ftype_str[ftype]}.gguf"
+fname_out = dir_out / f"ggml-model-{dir_model.name}-{ftype_str[ftype]}.gguf"


 ARCH = gguf.MODEL_ARCH.MPT
@@ -58,7 +58,7 @@ gguf_writer = gguf.GGUFWriter(fname_out, gguf.MODEL_ARCH_NAMES[ARCH])

 print("gguf: get model metadata")

-config = AutoConfig.from_pretrained(model_name)
+config = AutoConfig.from_pretrained(dir_model, trust_remote_code=True)

 block_count = config.n_layers
 gguf_writer.add_name("MPT")
@@ -67,17 +67,19 @@ gguf_writer.add_embedding_length(config.d_model)
 gguf_writer.add_block_count(block_count)
 gguf_writer.add_feed_forward_length(4 * config.d_model)
 gguf_writer.add_head_count(config.n_heads)
-gguf_writer.add_max_alibi_bias(config.attn_config.alibi_bias_max)
-gguf_writer.add_layer_norm_eps(config.layer_norm_epsilon)
+if kv_n_heads := config.attn_config.get('kv_n_heads'):
+    gguf_writer.add_head_count_kv(kv_n_heads)
+gguf_writer.add_max_alibi_bias(config.attn_config['alibi_bias_max'])
+gguf_writer.add_layer_norm_eps(MptConfig().layer_norm_epsilon)  # use default from upstream transformers
 gguf_writer.add_file_type(ftype)

-clip_qkv = config.attn_config.clip_qkv
+clip_qkv = config.attn_config['clip_qkv']
 if clip_qkv is not None:
    gguf_writer.add_clamp_kqv(clip_qkv)

 print("gguf: get gpt2 tokenizer vocab")

-tokenizer = AutoTokenizer.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(dir_model)

 special_ids = tokenizer.all_special_ids

@@ -111,13 +113,17 @@ gguf_writer.add_tokenizer_model("gpt2")
 gguf_writer.add_token_list(tokens)
 gguf_writer.add_token_types(toktypes)

+special_vocab = gguf.SpecialVocab(dir_model, load_merges=True)
+special_vocab.add_to_gguf(gguf_writer)
+
 print("gguf: get tensor metadata")

-print("Loading model:", model_name)
+print("Loading model:", dir_model)
 model = AutoModelForCausalLM.from_pretrained(
-    model_name, config=config, torch_dtype=torch.float16 if ftype == 1 else torch.float32, low_cpu_mem_usage=True,
+    dir_model, config=config, torch_dtype=torch.float16 if ftype == 1 else torch.float32,
+    low_cpu_mem_usage=True, trust_remote_code=True,
 )
-print("Model loaded:", model_name)
+print("Model loaded:", dir_model)

 tensor_map = gguf.get_tensor_name_map(ARCH, block_count)

--- a/gpt4all-bindings/cli/README.md
+++ b/gpt4all-bindings/cli/README.md
@@ -40,5 +40,5 @@ directory, if necessary.
 If you have already saved a model beforehand, specify its path with the `-m`/`--model` argument,
 for example:
 ```shell
-python app.py repl --model /home/user/my-gpt4all-models/GPT4All-13B-snoozy.ggmlv3.q4_0.bin
+python app.py repl --model /home/user/my-gpt4all-models/gpt4all-13b-snoozy-q4_0.gguf
 ```
--- a/gpt4all-bindings/cli/app.py
+++ b/gpt4all-bindings/cli/app.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 """GPT4All CLI

 The GPT4All CLI is a self-contained script based on the `gpt4all` and `typer` packages. It offers a
@@ -53,7 +54,7 @@ def repl(
    model: Annotated[
        str,
        typer.Option("--model", "-m", help="Model to use for chatbot"),
-    ] = "ggml-gpt4all-j-v1.3-groovy",
+    ] = "mistral-7b-instruct-v0.1.Q4_0.gguf",
    n_threads: Annotated[
        int,
        typer.Option("--n-threads", "-t", help="Number of threads to use for chatbot"),
--- a/gpt4all-bindings/csharp/build_linux.sh
+++ b/gpt4all-bindings/csharp/build_linux.sh
@@ -1,3 +1,4 @@
+#!/bin/sh
 mkdir -p runtimes
 rm -rf runtimes/linux-x64
 mkdir -p runtimes/linux-x64/native
@@ -7,4 +8,3 @@ cmake --build runtimes/linux-x64/build --parallel --config Release
 cp runtimes/linux-x64/build/libllmodel.so  runtimes/linux-x64/native/libllmodel.so
 cp runtimes/linux-x64/build/libgptj*.so  runtimes/linux-x64/native/
 cp runtimes/linux-x64/build/libllama*.so  runtimes/linux-x64/native/
-cp runtimes/linux-x64/build/libmpt*.so  runtimes/linux-x64/native/
--- a/gpt4all-bindings/java/README.md
+++ b/gpt4all-bindings/java/README.md
@@ -22,7 +22,7 @@ implementation 'com.hexadevlabs:gpt4all-java-binding:1.1.5'

 To add the library dependency for another build system see [Maven Central Java bindings](https://central.sonatype.com/artifact/com.hexadevlabs/gpt4all-java-binding/).

-To download model binary weights file use a URL such as [`https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin`](https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin).
+To download model binary weights file use a URL such as [`https://gpt4all.io/models/gguf/gpt4all-13b-snoozy-q4_0.gguf`](https://gpt4all.io/models/gguf/gpt4all-13b-snoozy-q4_0.gguf).

 For information about other models available see the [model file list](https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-chat#manual-download-of-models).

@@ -123,4 +123,4 @@ If this is the case you can easily download and install the latest x64 Microsoft
   - Falcon model support included.
 4. Version **1.1.5**:
   - Add a check for model file readability before loading model.
-   
+   
--- a/gpt4all-bindings/python/README.md
+++ b/gpt4all-bindings/python/README.md
@@ -50,7 +50,7 @@ Test it out! In a Python script or console:

 ```python
 from gpt4all import GPT4All
-model = GPT4All("orca-mini-3b.ggmlv3.q4_0.bin")
+model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
 output = model.generate("The capital of France is ", max_tokens=3)
 print(output)
 ```
@@ -59,7 +59,7 @@ print(output)
 GPU Usage
 ```python
 from gpt4all import GPT4All
-model = GPT4All("orca-mini-3b.ggmlv3.q4_0.bin", device='gpu') # device='amd', device='intel'
+model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu') # device='amd', device='intel'
 output = model.generate("The capital of France is ", max_tokens=3)
 print(output)
 ```
--- a/gpt4all-bindings/python/docs/gpt4all_cli.md
+++ b/gpt4all-bindings/python/docs/gpt4all_cli.md
@@ -166,7 +166,7 @@ If you want to use a different model, you can do so with the `-m`/`--model` para
 model file name is provided, it will again check in `.cache/gpt4all/` and might start downloading.
 If instead given a path to an existing model, the command could for example look like this:
 ```shell
-python app.py repl --model /home/user/my-gpt4all-models/GPT4All-13B-snoozy.ggmlv3.q4_0.bin
+python app.py repl --model /home/user/my-gpt4all-models/gpt4all-13b-snoozy-q4_0.gguf
 ```

 When you're done and want to end a session, simply type `/exit`.
--- a/gpt4all-bindings/python/docs/gpt4all_python.md
+++ b/gpt4all-bindings/python/docs/gpt4all_python.md
@@ -11,7 +11,7 @@ pip install gpt4all
 === "GPT4All Example"
    ``` py
    from gpt4all import GPT4All
-    model = GPT4All("orca-mini-3b.ggmlv3.q4_0.bin")
+    model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
    output = model.generate("The capital of France is ", max_tokens=3)
    print(output)
    ```
@@ -35,7 +35,7 @@ Use the GPT4All `chat_session` context manager to hold chat conversations with t

 === "GPT4All Example"
    ``` py
-    model = GPT4All(model_name='orca-mini-3b.ggmlv3.q4_0.bin')
+    model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf')
    with model.chat_session():
        response1 = model.generate(prompt='hello', temp=0)
        response2 = model.generate(prompt='write me a short poem', temp=0)
@@ -89,7 +89,7 @@ To interact with GPT4All responses as the model generates, use the `streaming=Tr
 === "GPT4All Streaming Example"
    ``` py
    from gpt4all import GPT4All
-    model = GPT4All("orca-mini-3b.ggmlv3.q4_0.bin")
+    model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
    tokens = []
    for token in model.generate("The capital of France is", max_tokens=20, streaming=True):
        tokens.append(token)
@@ -135,7 +135,7 @@ is the same as if it weren't provided; that is, `~/.cache/gpt4all/` is the defau
    ``` py
    from pathlib import Path
    from gpt4all import GPT4All
-    model = GPT4All(model_name='orca-mini-3b.ggmlv3.q4_0.bin',
+    model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf',
                    model_path=(Path.home() / '.cache' / 'gpt4all'),
                    allow_download=False)
    response = model.generate('my favorite 3 fruits are:', temp=0)
@@ -152,7 +152,7 @@ If you want to point it at the chat GUI's default folder, it should be:
    from pathlib import Path
    from gpt4all import GPT4All

-    model_name = 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model_name = 'orca-mini-3b-gguf2-q4_0.gguf'
    model_path = Path.home() / 'Library' / 'Application Support' / 'nomic.ai' / 'GPT4All'
    model = GPT4All(model_name, model_path)
    ```
@@ -161,7 +161,7 @@ If you want to point it at the chat GUI's default folder, it should be:
    from pathlib import Path
    from gpt4all import GPT4All
    import os
-    model_name = 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model_name = 'orca-mini-3b-gguf2-q4_0.gguf'
    model_path = Path(os.environ['LOCALAPPDATA']) / 'nomic.ai' / 'GPT4All'
    model = GPT4All(model_name, model_path)
    ```
@@ -170,7 +170,7 @@ If you want to point it at the chat GUI's default folder, it should be:
    from pathlib import Path
    from gpt4all import GPT4All

-    model_name = 'orca-mini-3b.ggmlv3.q4_0.bin'
+    model_name = 'orca-mini-3b-gguf2-q4_0.gguf'
    model_path = Path.home() / '.local' / 'share' / 'nomic.ai' / 'GPT4All'
    model = GPT4All(model_name, model_path)
    ```
@@ -182,7 +182,7 @@ from pathlib import Path
 import gpt4all.gpt4all
 gpt4all.gpt4all.DEFAULT_MODEL_DIRECTORY = Path.home() / 'my' / 'models-directory'
 from gpt4all import GPT4All
-model = GPT4All('orca-mini-3b.ggmlv3.q4_0.bin')
+model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')
 ...
 ```

@@ -193,7 +193,7 @@ Session templates can be customized when starting a `chat_session` context:
 === "GPT4All Custom Session Templates Example"
    ``` py
    from gpt4all import GPT4All
-    model = GPT4All('ggml-Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_1.bin')
+    model = GPT4All('wizardlm-13b-v1.2.Q4_0.gguf')
    system_template = 'A chat between a curious user and an artificial intelligence assistant.'
    # many models use triple hash '###' for keywords, Vicunas are simpler:
    prompt_template = 'USER: {0}\nASSISTANT: '
@@ -222,7 +222,7 @@ To do the same outside a session, the input has to be formatted manually. For ex

 === "GPT4All Templates Outside a Session Example"
    ``` py
-    model = GPT4All('ggml-Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_1.bin')
+    model = GPT4All('wizardlm-13b-v1.2.Q4_0.gguf')
    system_template = 'A chat between a curious user and an artificial intelligence assistant.'
    prompt_template = 'USER: {0}\nASSISTANT: '
    prompts = ['name 3 colors', 'now name 3 fruits', 'what were the 3 colors in your earlier response?']
@@ -285,7 +285,7 @@ customized in a subclass. As an example:
    ```
 === "GPT4All Custom Subclass Example"
    ``` py
-    model = RotatingTemplateGPT4All('ggml-Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_1.bin')
+    model = RotatingTemplateGPT4All('wizardlm-13b-v1.2.Q4_0.gguf')
    with model.chat_session():  # starting a session is optional in this example
        response1 = model.generate("hi, who are you?")
        print(response1)
@@ -345,7 +345,7 @@ logging infrastructure offers [many more customization options][py-logging-cookb
    import logging
    from gpt4all import GPT4All
    logging.basicConfig(level=logging.INFO)
-    model = GPT4All('nous-hermes-13b.ggmlv3.q4_0.bin')
+    model = GPT4All('nous-hermes-llama2-13b.Q4_0.gguf')
    with model.chat_session('You are a geography expert.\nBe terse.',
                            '### Instruction:\n{0}\n### Response:\n'):
        response = model.generate('who are you?', temp=0)
@@ -414,7 +414,7 @@ If you know exactly when a model should stop responding, you can add a custom ca
 === "GPT4All Custom Stop Callback"
    ``` py
    from gpt4all import GPT4All
-    model = GPT4All('orca-mini-3b.ggmlv3.q4_0.bin')
+    model = GPT4All('orca-mini-3b-gguf2-q4_0.gguf')

    def stop_on_token_callback(token_id, token_string):
        # one sentence is enough:
--- a/gpt4all-bindings/python/docs/gpt4all_typescript.md
+++ b/gpt4all-bindings/python/docs/gpt4all_typescript.md
@@ -705,7 +705,7 @@ Type: [boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Glob

 ##### url

-Remote download url. Defaults to `https://gpt4all.io/models/<modelName>`
+Remote download url. Defaults to `https://gpt4all.io/models/gguf/<modelName>`

 Type: [string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)

--- a/gpt4all-bindings/python/docs/index.md
+++ b/gpt4all-bindings/python/docs/index.md
@@ -9,7 +9,7 @@ GPT4All software is optimized to run inference of 3-13 billion parameter large l
 === "GPT4All Example"
    ``` py
    from gpt4all import GPT4All
-    model = GPT4All("orca-mini-3b.ggmlv3.q4_0.bin")
+    model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
    output = model.generate("The capital of France is ", max_tokens=3)
    print(output)
    ```
--- a/gpt4all-bindings/python/gpt4all/gpt4all.py
+++ b/gpt4all-bindings/python/gpt4all/gpt4all.py
@@ -75,7 +75,7 @@ class GPT4All:
        Constructor

        Args:
-            model_name: Name of GPT4All or custom model. Including ".bin" file extension is optional but encouraged.
+            model_name: Name of GPT4All or custom model. Including ".gguf" file extension is optional but encouraged.
            model_path: Path to directory containing model file or, if file does not exist, where to download model.
                Default is None, in which case models will be stored in `~/.cache/gpt4all/`.
            model_type: Model architecture. This argument currently does not have any functionality and is just used as
@@ -141,7 +141,7 @@ class GPT4All:
            Model config.
        """

-        model_filename = append_bin_suffix_if_missing(model_name)
+        model_filename = append_extension_if_missing(model_name)

        # get the config for the model
        config: ConfigType = DEFAULT_MODEL_CONFIG
@@ -201,7 +201,7 @@ class GPT4All:
        Download model from https://gpt4all.io.

        Args:
-            model_filename: Filename of model (with .bin extension).
+            model_filename: Filename of model (with .gguf extension).
            model_path: Path to download model to.
            verbose: If True (default), print debug messages.
            url: the models remote url (e.g. may be hosted on HF)
@@ -213,7 +213,7 @@ class GPT4All:
        def get_download_url(model_filename):
            if url:
                return url
-            return f"https://gpt4all.io/models/{model_filename}"
+            return f"https://gpt4all.io/models/gguf/{model_filename}"

        # Download model
        download_path = os.path.join(model_path, model_filename).replace("\\", "\\\\")
@@ -456,7 +456,7 @@ def empty_chat_session(system_prompt: str = "") -> List[MessageType]:
    return [{"role": "system", "content": system_prompt}]


-def append_bin_suffix_if_missing(model_name):
+def append_extension_if_missing(model_name):
    if not model_name.endswith((".bin", ".gguf")):
-        model_name += ".bin"
+        model_name += ".gguf"
    return model_name
--- a/gpt4all-bindings/python/gpt4all/tests/test_embed_timings.py
+++ b/gpt4all-bindings/python/gpt4all/tests/test_embed_timings.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import sys
 import time
 from io import StringIO
--- a/gpt4all-bindings/python/gpt4all/tests/test_gpt4all.py
+++ b/gpt4all-bindings/python/gpt4all/tests/test_gpt4all.py
@@ -8,7 +8,7 @@ import pytest


 def test_inference():
-    model = GPT4All(model_name='orca-mini-3b.ggmlv3.q4_0.bin')
+    model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf')
    output_1 = model.generate('hello', top_k=1)

    with model.chat_session():
@@ -47,49 +47,44 @@ def do_long_input(model):


 def test_inference_long_orca_3b():
-    model = GPT4All(model_name="orca-mini-3b.ggmlv3.q4_0.bin")
+    model = GPT4All(model_name="orca-mini-3b-gguf2-q4_0.gguf")
    do_long_input(model)


 def test_inference_long_falcon():
-    model = GPT4All(model_name='ggml-model-gpt4all-falcon-q4_0.bin')
+    model = GPT4All(model_name='gpt4all-falcon-q4_0.gguf')
    do_long_input(model)


 def test_inference_long_llama_7b():
-    model = GPT4All(model_name="orca-mini-7b.ggmlv3.q4_0.bin")
+    model = GPT4All(model_name="mistral-7b-openorca.Q4_0.gguf")
    do_long_input(model)


 def test_inference_long_llama_13b():
-    model = GPT4All(model_name='ggml-nous-hermes-13b.ggmlv3.q4_0.bin')
+    model = GPT4All(model_name='nous-hermes-llama2-13b.Q4_0.gguf')
    do_long_input(model)


 def test_inference_long_mpt():
-    model = GPT4All(model_name='ggml-mpt-7b-chat.bin')
+    model = GPT4All(model_name='mpt-7b-chat-q4_0.gguf')
    do_long_input(model)


 def test_inference_long_replit():
-    model = GPT4All(model_name='ggml-replit-code-v1-3b.bin')
-    do_long_input(model)
-
-
-def test_inference_long_groovy():
-    model = GPT4All(model_name='ggml-gpt4all-j-v1.3-groovy.bin')
+    model = GPT4All(model_name='replit-code-v1_5-3b-q4_0.gguf')
    do_long_input(model)


 def test_inference_hparams():
-    model = GPT4All(model_name='orca-mini-3b.ggmlv3.q4_0.bin')
+    model = GPT4All(model_name='orca-mini-3b-gguf2-q4_0.gguf')

    output = model.generate("The capital of france is ", max_tokens=3)
    assert 'Paris' in output


 def test_inference_falcon():
-    model = GPT4All(model_name='ggml-model-gpt4all-falcon-q4_0.bin')
+    model = GPT4All(model_name='gpt4all-falcon-q4_0.gguf')
    prompt = 'hello'
    output = model.generate(prompt)
    assert isinstance(output, str)
@@ -97,7 +92,7 @@ def test_inference_falcon():


 def test_inference_mpt():
-    model = GPT4All(model_name='ggml-mpt-7b-chat.bin')
+    model = GPT4All(model_name='mpt-7b-chat-q4_0.gguf')
    prompt = 'hello'
    output = model.generate(prompt)
    assert isinstance(output, str)
--- a/gpt4all-bindings/python/setup.py
+++ b/gpt4all-bindings/python/setup.py
@@ -61,7 +61,7 @@ copy_prebuilt_C_lib(SRC_CLIB_DIRECtORY,

 setup(
    name=package_name,
-    version="2.0.0rc2",
+    version="2.0.2",
    description="Python bindings for GPT4All",
    author="Nomic and the Open Source Community",
    author_email="support@nomic.ai",
--- a/gpt4all-bindings/typescript/README.md
+++ b/gpt4all-bindings/typescript/README.md
@@ -705,7 +705,7 @@ Type: [boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Glob

 ##### url

-Remote download url. Defaults to `https://gpt4all.io/models/<modelName>`
+Remote download url. Defaults to `https://gpt4all.io/models/gguf/<modelName>`

 Type: [string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)

--- a/gpt4all-bindings/typescript/scripts/build_unix.sh
+++ b/gpt4all-bindings/typescript/scripts/build_unix.sh
@@ -25,9 +25,6 @@ mkdir -p "$NATIVE_DIR" "$BUILD_DIR"
 cmake -S ../../gpt4all-backend -B "$BUILD_DIR" &&
 cmake --build "$BUILD_DIR" -j --config Release && {
  cp "$BUILD_DIR"/libbert*.$LIB_EXT   "$NATIVE_DIR"/
-  cp "$BUILD_DIR"/libfalcon*.$LIB_EXT "$NATIVE_DIR"/
-  cp "$BUILD_DIR"/libreplit*.$LIB_EXT "$NATIVE_DIR"/
  cp "$BUILD_DIR"/libgptj*.$LIB_EXT   "$NATIVE_DIR"/
  cp "$BUILD_DIR"/libllama*.$LIB_EXT  "$NATIVE_DIR"/
-  cp "$BUILD_DIR"/libmpt*.$LIB_EXT    "$NATIVE_DIR"/
 }
--- a/gpt4all-bindings/typescript/spec/chat.mjs
+++ b/gpt4all-bindings/typescript/spec/chat.mjs
@@ -1,7 +1,7 @@
 import { LLModel, createCompletion, DEFAULT_DIRECTORY, DEFAULT_LIBRARIES_DIRECTORY, loadModel } from '../src/gpt4all.js'

 const model = await loadModel(
-    'orca-mini-3b.ggmlv3.q4_0.bin',
+    'orca-mini-3b-gguf2-q4_0.gguf',
    { verbose: true }
 );
 const ll = model.llm;
--- a/gpt4all-bindings/typescript/src/gpt4all.d.ts
+++ b/gpt4all-bindings/typescript/src/gpt4all.d.ts
@@ -444,8 +444,8 @@ interface DownloadModelOptions {
    verbose?: boolean;

    /**
-     * Remote download url. Defaults to `https://gpt4all.io/models/<modelName>`
-     * @default https://gpt4all.io/models/<modelName>
+     * Remote download url. Defaults to `https://gpt4all.io/models/gguf/<modelName>`
+     * @default https://gpt4all.io/models/gguf/<modelName>
     */
    url?: string;
    /**
--- a/gpt4all-bindings/typescript/src/util.js
+++ b/gpt4all-bindings/typescript/src/util.js
@@ -113,7 +113,7 @@ function downloadModel(modelName, options = {}) {
    );
    const finalModelPath = path.join(downloadOptions.modelPath, modelFileName);
    const modelUrl =
-        downloadOptions.url ?? `https://gpt4all.io/models/${modelFileName}`;
+        downloadOptions.url ?? `https://gpt4all.io/models/gguf/${modelFileName}`;

    mkdirp.sync(downloadOptions.modelPath)

--- a/gpt4all-chat/CMakeLists.txt
+++ b/gpt4all-chat/CMakeLists.txt
@@ -18,7 +18,7 @@ endif()

 set(APP_VERSION_MAJOR 2)
 set(APP_VERSION_MINOR 5)
-set(APP_VERSION_PATCH 0)
+set(APP_VERSION_PATCH 3)
 set(APP_VERSION "${APP_VERSION_MAJOR}.${APP_VERSION_MINOR}.${APP_VERSION_PATCH}")

 # Include the binary directory for the generated header file
@@ -189,8 +189,6 @@ install(TARGETS llamamodel-mainline-default DESTINATION lib COMPONENT ${COMPONEN
 if(APPLE)
 install(TARGETS llamamodel-mainline-metal DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN})
 endif()
-install(TARGETS mpt-avxonly DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN})
-install(TARGETS mpt-default DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN})
 install(TARGETS bert-avxonly DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN})
 install(TARGETS bert-default DESTINATION lib COMPONENT ${COMPONENT_NAME_MAIN})

--- a/gpt4all-chat/build_and_run.md
+++ b/gpt4all-chat/build_and_run.md
@@ -47,7 +47,9 @@ Under this release, select the following additional components:
 - Qt Quick 3D
 - Qt 5 Compatibility Module
 - Qt Shader Tools
- Additional Libraries (clicking the checkbox to the left of this item enables all of them)
+- Additional Libraries:
+  - Qt HTTP Server
+  - Qt PDF
 - Qt Debug information Files
 - Qt Quick Timeline

--- a/gpt4all-chat/chat.cpp
+++ b/gpt4all-chat/chat.cpp
@@ -142,17 +142,9 @@ QString Chat::response() const
    return m_response;
 }

-QString Chat::responseState() const
+Chat::ResponseState Chat::responseState() const
 {
-    switch (m_responseState) {
-    case ResponseStopped: return QStringLiteral("response stopped");
-    case LocalDocsRetrieval: return QStringLiteral("retrieving ") + m_collections.join(", ");
-    case LocalDocsProcessing: return QStringLiteral("processing ") + m_collections.join(", ");
-    case PromptProcessing: return QStringLiteral("processing");
-    case ResponseGeneration: return QStringLiteral("generating response");
-    };
-    Q_UNREACHABLE();
-    return QString();
+    return m_responseState;
 }

 void Chat::handleResponseChanged(const QString &response)
@@ -403,6 +395,7 @@ bool Chat::deserialize(QDataStream &stream, int version)
    emit idChanged(m_id);
    stream >> m_name;
    stream >> m_userName;
+    m_generatedName = QLatin1String("nonempty");
    emit nameChanged();

    QString modelId;
--- a/gpt4all-chat/chat.h
+++ b/gpt4all-chat/chat.h
@@ -21,7 +21,7 @@ class Chat : public QObject
    Q_PROPERTY(bool responseInProgress READ responseInProgress NOTIFY responseInProgressChanged)
    Q_PROPERTY(bool isRecalc READ isRecalc NOTIFY recalcChanged)
    Q_PROPERTY(bool isServer READ isServer NOTIFY isServerChanged)
-    Q_PROPERTY(QString responseState READ responseState NOTIFY responseStateChanged)
+    Q_PROPERTY(ResponseState responseState READ responseState NOTIFY responseStateChanged)
    Q_PROPERTY(QList<QString> collectionList READ collectionList NOTIFY collectionListChanged)
    Q_PROPERTY(QString modelLoadingError READ modelLoadingError NOTIFY modelLoadingErrorChanged)
    Q_PROPERTY(QString tokenSpeed READ tokenSpeed NOTIFY tokenSpeedChanged);
@@ -68,7 +68,7 @@ public:

    QString response() const;
    bool responseInProgress() const { return m_responseInProgress; }
-    QString responseState() const;
+    ResponseState responseState() const;
    ModelInfo modelInfo() const;
    void setModelInfo(const ModelInfo &modelInfo);
    bool isRecalc() const;
--- a/gpt4all-chat/chatllm.cpp
+++ b/gpt4all-chat/chatllm.cpp
@@ -9,7 +9,6 @@
 //#define DEBUG
 //#define DEBUG_MODEL_LOADING

-#define MPT_INTERNAL_STATE_VERSION 0
 #define GPTJ_INTERNAL_STATE_VERSION 0
 #define LLAMA_INTERNAL_STATE_VERSION 0
 #define BERT_INTERNAL_STATE_VERSION 0
@@ -325,7 +324,6 @@ bool ChatLLM::loadModel(const ModelInfo &modelInfo)
                    switch (m_llModelInfo.model->implementation().modelType()[0]) {
                    case 'L': m_llModelType = LLModelType::LLAMA_; break;
                    case 'G': m_llModelType = LLModelType::GPTJ_; break;
-                    case 'M': m_llModelType = LLModelType::MPT_; break;
                    case 'B': m_llModelType = LLModelType::BERT_; break;
                    default:
                        {
@@ -380,6 +378,32 @@ bool ChatLLM::isModelLoaded() const
    return m_llModelInfo.model && m_llModelInfo.model->isModelLoaded();
 }

+std::string remove_leading_whitespace(const std::string& input) {
+    auto first_non_whitespace = std::find_if(input.begin(), input.end(), [](unsigned char c) {
+        return !std::isspace(c);
+    });
+
+    if (first_non_whitespace == input.end())
+        return std::string();
+
+    return std::string(first_non_whitespace, input.end());
+}
+
+std::string trim_whitespace(const std::string& input) {
+    auto first_non_whitespace = std::find_if(input.begin(), input.end(), [](unsigned char c) {
+        return !std::isspace(c);
+    });
+
+    if (first_non_whitespace == input.end())
+        return std::string();
+
+    auto last_non_whitespace = std::find_if(input.rbegin(), input.rend(), [](unsigned char c) {
+        return !std::isspace(c);
+    }).base();
+
+    return std::string(first_non_whitespace, last_non_whitespace);
+}
+
 void ChatLLM::regenerateResponse()
 {
    // ChatGPT uses a different semantic meaning for n_past than local models. For ChatGPT, the meaning
@@ -411,29 +435,6 @@ void ChatLLM::resetContext()
    m_ctx = LLModel::PromptContext();
 }

-std::string remove_leading_whitespace(const std::string& input) {
-    auto first_non_whitespace = std::find_if(input.begin(), input.end(), [](unsigned char c) {
-        return !std::isspace(c);
-    });
-
-    return std::string(first_non_whitespace, input.end());
-}
-
-std::string trim_whitespace(const std::string& input) {
-    auto first_non_whitespace = std::find_if(input.begin(), input.end(), [](unsigned char c) {
-        return !std::isspace(c);
-    });
-
-    if (first_non_whitespace == input.end())
-        return std::string();
-
-    auto last_non_whitespace = std::find_if(input.rbegin(), input.rend(), [](unsigned char c) {
-        return !std::isspace(c);
-    }).base();
-
-    return std::string(first_non_whitespace, last_non_whitespace);
-}
-
 QString ChatLLM::response() const
 {
    return QString::fromStdString(remove_leading_whitespace(m_response));
@@ -478,7 +479,7 @@ bool ChatLLM::handleResponse(int32_t token, const std::string &response)
    // check for error
    if (token < 0) {
        m_response.append(response);
-        emit responseChanged(QString::fromStdString(m_response));
+        emit responseChanged(QString::fromStdString(remove_leading_whitespace(m_response)));
        return false;
    }

@@ -488,7 +489,7 @@ bool ChatLLM::handleResponse(int32_t token, const std::string &response)
    m_timer->inc();
    Q_ASSERT(!response.empty());
    m_response.append(response);
-    emit responseChanged(QString::fromStdString(m_response));
+    emit responseChanged(QString::fromStdString(remove_leading_whitespace(m_response)));
    return !m_stopGenerating;
 }

@@ -505,6 +506,11 @@ bool ChatLLM::handleRecalculate(bool isRecalc)
 }
 bool ChatLLM::prompt(const QList<QString> &collectionList, const QString &prompt)
 {
+    if (m_restoreStateFromText) {
+        Q_ASSERT(m_state.isEmpty());
+        processRestoreStateFromText();
+    }
+
    if (!m_processedSystemPrompt)
        processSystemPrompt();
    const QString promptTemplate = MySettings::globalInstance()->modelPromptTemplate(m_modelInfo);
@@ -760,7 +766,6 @@ bool ChatLLM::serialize(QDataStream &stream, int version, bool serializeKV)
    if (version > 1) {
        stream << m_llModelType;
        switch (m_llModelType) {
-        case MPT_: stream << MPT_INTERNAL_STATE_VERSION; break;
        case GPTJ_: stream << GPTJ_INTERNAL_STATE_VERSION; break;
        case LLAMA_: stream << LLAMA_INTERNAL_STATE_VERSION; break;
        case BERT_: stream << BERT_INTERNAL_STATE_VERSION; break;
@@ -909,11 +914,6 @@ void ChatLLM::restoreState()
        return;
    }

-    if (m_restoreStateFromText) {
-        Q_ASSERT(m_state.isEmpty());
-        processRestoreStateFromText();
-    }
-
 #if defined(DEBUG)
    qDebug() << "restoreState" << m_llmThread.objectName() << "size:" << m_state.size();
 #endif
--- a/gpt4all-chat/chatllm.h
+++ b/gpt4all-chat/chatllm.h
@@ -10,7 +10,6 @@
 #include "../gpt4all-backend/llmodel.h"

 enum LLModelType {
-    MPT_,
    GPTJ_,
    LLAMA_,
    CHATGPT_,
--- a/gpt4all-chat/cmake/deploy-qt-mac.cmake.in
+++ b/gpt4all-chat/cmake/deploy-qt-mac.cmake.in
@@ -3,20 +3,15 @@ set(COMPONENT_NAME_MAIN "@COMPONENT_NAME_MAIN@")
 set(CMAKE_CURRENT_SOURCE_DIR "@CMAKE_CURRENT_SOURCE_DIR@")
 execute_process(COMMAND ${MACDEPLOYQT} ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app -qmldir=${CMAKE_CURRENT_SOURCE_DIR} -verbose=2)
 file(GLOB MYGPTJLIBS ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/lib/libgptj*)
-file(GLOB MYMPTLIBS ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/lib/libmpt*)
 file(GLOB MYLLAMALIBS ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/lib/libllama*)
 file(GLOB MYBERTLLIBS ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/lib/libbert*)
 file(GLOB MYLLMODELLIBS ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/lib/libllmodel.*)
 file(COPY ${MYGPTJLIBS}
     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
-file(COPY ${MYMPTLIBS}
-     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
 file(COPY ${MYLLAMALIBS}
     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
 file(COPY ${MYBERTLLIBS}
     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
-file(COPY ${MYLLAMALIBS}
-     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
 file(COPY ${MYLLMODELLIBS}
     DESTINATION ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/packages/${COMPONENT_NAME_MAIN}/data/bin/gpt4all.app/Contents/Frameworks)
 file(COPY "${CMAKE_CURRENT_SOURCE_DIR}/icons/favicon.icns"
--- a/gpt4all-chat/cmake/sign_dmg.py
+++ b/gpt4all-chat/cmake/sign_dmg.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import os
 import subprocess
 import tempfile
--- a/gpt4all-chat/download.cpp
+++ b/gpt4all-chat/download.cpp
@@ -129,7 +129,7 @@ void Download::downloadModel(const QString &modelFile)

    ModelList::globalInstance()->updateDataByFilename(modelFile, ModelList::DownloadingRole, true);
    ModelInfo info = ModelList::globalInstance()->modelInfoByFilename(modelFile);
-    QString url = !info.url.isEmpty() ? info.url : "http://gpt4all.io/models/" + modelFile;
+    QString url = !info.url.isEmpty() ? info.url : "http://gpt4all.io/models/gguf/" + modelFile;
    Network::globalInstance()->sendDownloadStarted(modelFile);
    QNetworkRequest request(url);
    request.setAttribute(QNetworkRequest::User, modelFile);
--- a/gpt4all-chat/main.qml
+++ b/gpt4all-chat/main.qml
@@ -282,8 +282,8 @@ Window {
                    highlighted: comboBox.highlightedIndex === index
                }
                Accessible.role: Accessible.ComboBox
-                Accessible.name: qsTr("ComboBox for displaying/picking the current model")
-                Accessible.description: qsTr("Use this for picking the current model to use; the first item is the current model")
+                Accessible.name: qsTr("List of available models")
+                Accessible.description: qsTr("The top item is the current model")
                onActivated: function (index) {
                    currentChat.stopGenerating()
                    currentChat.reset();
@@ -307,7 +307,7 @@ Window {
                    running: parent.visible
                    Accessible.role: Accessible.Animation
                    Accessible.name: qsTr("Busy indicator")
-                    Accessible.description: qsTr("Displayed when the model is loading")
+                    Accessible.description: qsTr("loading model...")
                }

                Label {
@@ -339,8 +339,8 @@ Window {
        padding: 15

        Accessible.role: Accessible.ButtonMenu
-        Accessible.name: qsTr("Hamburger button")
-        Accessible.description: qsTr("Hamburger button that reveals a drawer on the left of the application")
+        Accessible.name: qsTr("Main menu")
+        Accessible.description: qsTr("Navigation drawer with options")

        background: Item {
            anchors.centerIn: parent
@@ -389,7 +389,7 @@ Window {
        Item {
            Accessible.role: Accessible.Dialog
            Accessible.name: qsTr("Network dialog")
-            Accessible.description: qsTr("Dialog for opt-in to sharing feedback/conversations")
+            Accessible.description: qsTr("opt-in to share feedback/conversations")
        }
    }

@@ -405,7 +405,7 @@ Window {
        padding: 15
        toggled: MySettings.networkIsActive
        source: "qrc:/gpt4all/icons/network.svg"
-        Accessible.name: qsTr("Network button")
+        Accessible.name: qsTr("Network")
        Accessible.description: qsTr("Reveals a dialogue where you can opt-in for sharing data over network")

        onClicked: {
@@ -441,8 +441,8 @@ Window {
        padding: 15
        toggled: currentChat.collectionList.length
        source: "qrc:/gpt4all/icons/db.svg"
-        Accessible.name: qsTr("Add collections of documents to the chat")
-        Accessible.description: qsTr("Provides a button to add collections of documents to the chat")
+        Accessible.name: qsTr("Add documents")
+        Accessible.description: qsTr("add collections of documents to the chat")

        onClicked: {
            collectionsDialog.open()
@@ -460,8 +460,8 @@ Window {
        z: 200
        padding: 15
        source: "qrc:/gpt4all/icons/settings.svg"
-        Accessible.name: qsTr("Settings button")
-        Accessible.description: qsTr("Reveals a dialogue where you can change various settings")
+        Accessible.name: qsTr("Settings")
+        Accessible.description: qsTr("Reveals a dialogue with settings")

        onClicked: {
            settingsDialog.open()
@@ -528,7 +528,7 @@ Window {
        z: 200
        padding: 15
        source: "qrc:/gpt4all/icons/copy.svg"
-        Accessible.name: qsTr("Copy button")
+        Accessible.name: qsTr("Copy")
        Accessible.description: qsTr("Copy the conversation to the clipboard")

        TextEdit{
@@ -595,7 +595,7 @@ Window {
        source: "qrc:/gpt4all/icons/regenerate.svg"

        Accessible.name: text
-        Accessible.description: qsTr("Reset the context which erases current conversation")
+        Accessible.description: qsTr("Reset the context and erase current conversation")

        onClicked: {
            Network.sendResetContext(chatModel.count)
@@ -623,7 +623,7 @@ Window {
            font.pixelSize: theme.fontSizeLarge
            Accessible.role: Accessible.Dialog
            Accessible.name: text
-            Accessible.description: qsTr("Dialog indicating an error")
+            Accessible.description: qsTr("Error dialog")
        }
        background: Rectangle {
            anchors.fill: parent
@@ -641,7 +641,7 @@ Window {
        height: window.height - (window.height * .1)
        Item {
            Accessible.role: Accessible.Dialog
-            Accessible.name: qsTr("Download new models dialog")
+            Accessible.name: qsTr("Download new models")
            Accessible.description: qsTr("Dialog for downloading new models")
        }
    }
@@ -740,8 +740,8 @@ Window {
                    ScrollBar.vertical: ScrollBar { policy: ScrollBar.AlwaysOn }

                    Accessible.role: Accessible.List
-                    Accessible.name: qsTr("List of prompt/response pairs")
-                    Accessible.description: qsTr("This is the list of prompt/response pairs comprising the actual conversation with the model")
+                    Accessible.name: qsTr("Conversation with the model")
+                    Accessible.description: qsTr("prompt / response pairs from the conversation")

                    delegate: TextArea {
                        id: myTextArea
@@ -811,12 +811,21 @@ Window {
                                    running: (currentResponse ? true : false) && value === "" && currentChat.responseInProgress
                                    Accessible.role: Accessible.Animation
                                    Accessible.name: qsTr("Busy indicator")
-                                    Accessible.description: qsTr("Displayed when the model is thinking")
+                                    Accessible.description: qsTr("The model is thinking")
                                }
                                Label {
                                    anchors.verticalCenter: parent.verticalCenter
-                                    text: currentChat.responseState + "..."
                                    color: theme.textAccent
+                                    text: {
+                                        switch (currentChat.responseState) {
+                                        case Chat.ResponseStopped: return "response stopped ...";
+                                        case Chat.LocalDocsRetrieval: return "retrieving " + currentChat.collectionList.join(", ") + " ...";
+                                        case Chat.LocalDocsProcessing: return "processing " + currentChat.collectionList.join(", ") + " ...";
+                                        case Chat.PromptProcessing: return "processing ..."
+                                        case Chat.ResponseGeneration: return "generating response ...";
+                                        default: return ""; // handle unexpected values
+                                        }
+                                    }
                                }
                            }
                        }
@@ -1053,7 +1062,7 @@ Window {
                }
                Accessible.role: Accessible.EditableText
                Accessible.name: placeholderText
-                Accessible.description: qsTr("Textfield for sending messages/prompts to the model")
+                Accessible.description: qsTr("Send messages/prompts to the model")
                Keys.onReturnPressed: (event)=> {
                    if (event.modifiers & Qt.ControlModifier || event.modifiers & Qt.ShiftModifier)
                        event.accepted = false;
@@ -1090,7 +1099,7 @@ Window {
            height: 30
            visible: !currentChat.isServer
            source: "qrc:/gpt4all/icons/send_message.svg"
-            Accessible.name: qsTr("Send the message button")
+            Accessible.name: qsTr("Send message")
            Accessible.description: qsTr("Sends the message/prompt contained in textfield to the model")

            onClicked: {
--- a/gpt4all-chat/metadata/models2.json
+++ b/gpt4all-chat/metadata/models2.json
@@ -94,17 +94,17 @@
  },
  {
    "order": "h",
-    "md5sum": "f5bc6a52f72efd9128efb2eeed802c86",
+    "md5sum": "cf5e8f73747f9d7c6fe72a629808c1de",
    "name": "MPT Chat",
-    "filename": "mpt-7b-chat-q4_0.gguf",
-    "filesize": "3911522272",
+    "filename": "mpt-7b-chat-merges-q4_0.gguf",
+    "filesize": "3796133728",
    "requires": "2.5.0",
    "ramrequired": "8",
    "parameters": "7 billion",
    "quant": "q4_0",
    "type": "MPT",
    "description": "<strong>Good model with novel architecture</strong><br><ul><li>Fast responses<li>Chat based<li>Trained by Mosaic ML<li>Cannot be used commercially</ul>",
-    "url": "https://gpt4all.io/models/gguf/mpt-7b-chat-q4_0.gguf",
+    "url": "https://gpt4all.io/models/gguf/mpt-7b-chat-merges-q4_0.gguf",
    "promptTemplate": "<|im_start|>user\n%1<|im_end|><|im_start|>assistant\n",
    "systemPrompt": "<|im_start|>system\n- You are a helpful assistant chatbot trained by MosaicML.\n- You answer questions.\n- You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.\n- You are more than just an information source, you are also able to write poetry, short stories, and make jokes.<|im_end|>"
  },
@@ -126,20 +126,20 @@
  },
  {
    "order": "j",
-    "md5sum": "51c627fac9062e208f9b386f105cbd48",
+    "md5sum": "e30579a1b109882f10e2a5e75ea388fb",
    "disableGUI": "true",
    "name": "Replit",
-    "filename": "replit-code-v1-3b-q4_0.gguf",
-    "filesize": "1532949760",
+    "filename": "replit-code-v1_5-3b-q4_0.gguf",
+    "filesize": "1870449696",
    "requires": "2.5.0",
    "ramrequired": "4",
    "parameters": "3 billion",
-    "quant": "f16",
+    "quant": "q4_0",
    "type": "Replit",
    "systemPrompt": " ",
    "promptTemplate": "%1",
    "description": "<strong>Trained on subset of the Stack</strong><br><ul><li>Code completion based<li>Licensed for commercial use</ul>",
-    "url": "https://gpt4all.io/models/gguf/replit-code-v1-3b-q4_0.gguf"
+    "url": "https://gpt4all.io/models/gguf/replit-code-v1_5-3b-q4_0.gguf"
  },
  {
    "order": "k",
--- a/gpt4all-chat/metadata/release.json
+++ b/gpt4all-chat/metadata/release.json
@@ -529,6 +529,59 @@
    "contributors":
 "
 * Adam Treat (Nomic AI)
+"
+  },
+  {
+    "version": "2.5.0",
+    "notes":
+"
+* Major new release supports GGUF models only!
+* New models like Mistral Instruct, Replit 1.5, Rift Coder and more
+* All previous version of ggml-based models are no longer supported
+* Extensive changes to vulkan support
+* Better GPU error messages
+* Prompt processing on the GPU
+* Save chats now saves to text (less harddrive space)
+* Many more changes
+",
+    "contributors":
+"
+* Aaron Miller (Nomic AI)
+* Jared Van Bortel (Nomic AI)
+* Adam Treat (Nomic AI)
+* Community (beta testers, bug reporters, bindings authors)
+"
+  },
+  {
+    "version": "2.5.1",
+    "notes":
+"
+* Accessibility fixes
+* Bugfix for crasher on Windows
+",
+    "contributors":
+"
+* Aaron Miller (Nomic AI)
+* Jared Van Bortel (Nomic AI)
+* Victor Tsaran <vtsaran@yahoo.com>
+* Community (beta testers, bug reporters, bindings authors)
+"
+  },
+  {
+    "version": "2.5.2",
+    "notes":
+"
+* Support for GGUF v3 models
+* Important fixes for AMD GPUs
+* Don't start recalculating context immediately for saved chats
+* UI fixes for chat name generation
+* UI fixes for leading whitespaces in chat generation
+",
+    "contributors":
+"
+* Jared Van Bortel (Nomic AI)
+* Adam Treat (Nomic AI)
+* Community (beta testers, bug reporters, bindings authors)
 "
  }
 ]
--- a/gpt4all-chat/qml/ApplicationSettings.qml
+++ b/gpt4all-chat/qml/ApplicationSettings.qml
@@ -35,8 +35,8 @@ MySettingsTab {
            Layout.fillWidth: false
            model: ["Dark", "Light"]
            Accessible.role: Accessible.ComboBox
-            Accessible.name: qsTr("ComboBox for displaying/picking the color theme")
-            Accessible.description: qsTr("Use this for picking the color theme for the chat client to use")
+            Accessible.name: qsTr("Color theme")
+            Accessible.description: qsTr("Color theme for the chat client to use")
            function updateModel() {
                themeBox.currentIndex = themeBox.indexOfValue(MySettings.chatTheme);
            }
@@ -70,8 +70,8 @@ MySettingsTab {
            Layout.fillWidth: false
            model: ["Small", "Medium", "Large"]
            Accessible.role: Accessible.ComboBox
-            Accessible.name: qsTr("ComboBox for displaying/picking the font size")
-            Accessible.description: qsTr("Use this for picking the font size of the chat client")
+            Accessible.name: qsTr("Font size")
+            Accessible.description: qsTr("Font size of the chat client")
            function updateModel() {
                fontBox.currentIndex = fontBox.indexOfValue(MySettings.fontSize);
            }
@@ -105,8 +105,8 @@ MySettingsTab {
            Layout.fillWidth: false
            model: MySettings.deviceList
            Accessible.role: Accessible.ComboBox
-            Accessible.name: qsTr("ComboBox for displaying/picking the device")
-            Accessible.description: qsTr("Use this for picking the device of the chat client")
+            Accessible.name: qsTr("Device")
+            Accessible.description: qsTr("Device of the chat client")
            function updateModel() {
                deviceBox.currentIndex = deviceBox.indexOfValue(MySettings.device);
            }
@@ -143,8 +143,8 @@ MySettingsTab {
            Layout.fillWidth: true
            model: ModelList.userDefaultModelList
            Accessible.role: Accessible.ComboBox
-            Accessible.name: qsTr("ComboBox for displaying/picking the default model")
-            Accessible.description: qsTr("Use this for picking the default model to use; the first item is the current default model")
+            Accessible.name: qsTr("Default model")
+            Accessible.description: qsTr("Default model to use; the first item is the current default model")
            function updateModel() {
                comboBox.currentIndex = comboBox.indexOfValue(MySettings.userDefaultModel);
            }
@@ -194,7 +194,7 @@ MySettingsTab {
            Layout.row: 5
            Layout.column: 2
            text: qsTr("Browse")
-            Accessible.description: qsTr("Opens a folder picker dialog to choose where to save model files")
+            Accessible.description: qsTr("Choose where to save model files")
            onClicked: {
                openFolderDialog("file://" + MySettings.modelPath, function(selectedFolder) {
                    MySettings.modelPath = selectedFolder
--- a/gpt4all-chat/qml/ChatDrawer.qml
+++ b/gpt4all-chat/qml/ChatDrawer.qml
@@ -31,8 +31,8 @@ Drawer {
        anchors.margins: 10

        Accessible.role: Accessible.Pane
-        Accessible.name: qsTr("Drawer on the left of the application")
-        Accessible.description: qsTr("Drawer that is revealed by pressing the hamburger button")
+        Accessible.name: qsTr("Drawer")
+        Accessible.description: qsTr("Main navigation drawer")

        MyButton {
            id: newChat
@@ -42,7 +42,7 @@ Drawer {
            topPadding: 20
            bottomPadding: 20
            text: qsTr("\uFF0B New chat")
-            Accessible.description: qsTr("Use this to create a new chat")
+            Accessible.description: qsTr("Create a new chat")
            background: Rectangle {
                border.color: newChat.down ? theme.backgroundLightest : theme.buttonBorder
                border.width: 2
@@ -135,7 +135,7 @@ Drawer {
                        }
                        Accessible.role: Accessible.Button
                        Accessible.name: qsTr("Select the current chat")
-                        Accessible.description: qsTr("Provides a button to select the current chat or edit the chat when in edit mode")
+                        Accessible.description: qsTr("Select the current chat or edit the chat when in edit mode")
                    }
                    Row {
                        id: buttons
@@ -155,8 +155,7 @@ Drawer {
                                chatName.readOnly = false
                                chatName.selectByMouse = true
                            }
-                            Accessible.name: qsTr("Edit the chat name")
-                            Accessible.description: qsTr("Provides a button to edit the chat name")
+                            Accessible.name: qsTr("Edit chat name")
                        }
                        MyToolButton {
                            id: trashButton
@@ -168,8 +167,7 @@ Drawer {
                                trashQuestionDisplayed = true
                                timer.start()
                            }
-                            Accessible.name: qsTr("Delete of the chat")
-                            Accessible.description: qsTr("Provides a button to delete the chat")
+                            Accessible.name: qsTr("Delete chat")
                        }
                    }
                    Rectangle {
@@ -207,8 +205,7 @@ Drawer {
                                    Network.sendRemoveChat()
                                }
                                Accessible.role: Accessible.Button
-                                Accessible.name: qsTr("Confirm delete of the chat")
-                                Accessible.description: qsTr("Provides a button to confirm delete of the chat")
+                                Accessible.name: qsTr("Confirm chat deletion")
                            }
                            Button {
                                id: cancel
@@ -230,8 +227,7 @@ Drawer {
                                    trashQuestionDisplayed = false
                                }
                                Accessible.role: Accessible.Button
-                                Accessible.name: qsTr("Cancel the delete of the chat")
-                                Accessible.description: qsTr("Provides a button to cancel delete of the chat")
+                                Accessible.name: qsTr("Cancel chat deletion")
                            }
                        }
                    }
@@ -256,7 +252,7 @@ Drawer {
            anchors.bottomMargin: 10
            text: qsTr("Updates")
            font.pixelSize: theme.fontSizeLarge
-            Accessible.description: qsTr("Use this to launch an external application that will check for updates to the installer")
+            Accessible.description: qsTr("Launch an external application that will check for updates to the installer")
            onClicked: {
                if (!LLM.checkForUpdates())
                    checkForUpdatesError.open()
@@ -270,7 +266,7 @@ Drawer {
            anchors.bottom: aboutButton.top
            anchors.bottomMargin: 10
            text: qsTr("Downloads")
-            Accessible.description: qsTr("Use this to launch a dialog to download new models")
+            Accessible.description: qsTr("Launch a dialog to download new models")
            onClicked: {
                downloadClicked()
            }
@@ -282,7 +278,7 @@ Drawer {
            anchors.right: parent.right
            anchors.bottom: parent.bottom
            text: qsTr("About")
-            Accessible.description: qsTr("Use this to launch a dialog to show the about page")
+            Accessible.description: qsTr("Launch a dialog to show the about page")
            onClicked: {
                aboutClicked()
            }
--- a/gpt4all-chat/qml/LocalDocsSettings.qml
+++ b/gpt4all-chat/qml/LocalDocsSettings.qml
@@ -83,7 +83,7 @@ MySettingsTab {
                    text: qsTr("Add")
                    Accessible.role: Accessible.Button
                    Accessible.name: text
-                    Accessible.description: qsTr("Add button")
+                    Accessible.description: qsTr("Add collection")
                    onClicked: {
                        var isError = false;
                        if (root.collection === "") {
--- a/gpt4all-chat/qml/ModelDownloaderDialog.qml
+++ b/gpt4all-chat/qml/ModelDownloaderDialog.qml
@@ -125,7 +125,7 @@ MyDialog {
                                    Layout.fillWidth: true
                                    Layout.alignment: Qt.AlignTop | Qt.AlignHCenter
                                    visible: !isChatGPT && !installed && !calcHash && downloadError === ""
-                                    Accessible.description: qsTr("Cancel/Resume/Download button to stop/restart/start the download")
+                                    Accessible.description: qsTr("Stop/restart/start the download")
                                    background: Rectangle {
                                        border.color: downloadButton.down ? theme.backgroundLightest : theme.buttonBorder
                                        border.width: 2
@@ -151,7 +151,7 @@ MyDialog {
                                    Layout.fillWidth: true
                                    Layout.alignment: Qt.AlignTop | Qt.AlignHCenter
                                    visible: installed || downloadError !== ""
-                                    Accessible.description: qsTr("Remove button to remove model from filesystem")
+                                    Accessible.description: qsTr("Remove model from filesystem")
                                    background: Rectangle {
                                        border.color: removeButton.down ? theme.backgroundLightest : theme.buttonBorder
                                        border.width: 2
@@ -186,8 +186,8 @@ MyDialog {
                                            Download.installModel(filename, openaiKey.text);
                                    }
                                    Accessible.role: Accessible.Button
-                                    Accessible.name: qsTr("Install button")
-                                    Accessible.description: qsTr("Install button to install chatgpt model")
+                                    Accessible.name: qsTr("Install")
+                                    Accessible.description: qsTr("Install chatGPT model")
                                }

                                ColumnLayout {
@@ -385,7 +385,7 @@ MyDialog {
                            linkColor: theme.textColor
                            Accessible.role: Accessible.Paragraph
                            Accessible.name: qsTr("Description")
-                            Accessible.description: qsTr("The description of the file")
+                            Accessible.description: qsTr("File description")
                            onLinkActivated: Qt.openUrlExternally(link)
                        }
                    }
@@ -456,7 +456,7 @@ MyDialog {
            }
            MyButton {
                text: qsTr("Browse")
-                Accessible.description: qsTr("Opens a folder picker dialog to choose where to save model files")
+                Accessible.description: qsTr("Choose where to save model files")
                onClicked: modelPathDialog.open()
            }
        }
--- a/gpt4all-chat/qml/MySettingsTab.qml
+++ b/gpt4all-chat/qml/MySettingsTab.qml
@@ -69,7 +69,7 @@ Item {
                    font.pixelSize: theme.fontSizeLarge
                    Accessible.role: Accessible.Button
                    Accessible.name: text
-                    Accessible.description: qsTr("Restores the settings dialog to a default state")
+                    Accessible.description: qsTr("Restores settings dialog to a default state")
                    onClicked: {
                        root.restoreDefaultsClicked();
                    }
--- a/gpt4all-chat/qml/NetworkDialog.qml
+++ b/gpt4all-chat/qml/NetworkDialog.qml
@@ -89,7 +89,7 @@ NOTE: By turning on this feature, you will be sending your data to the GPT4All O
            }
            Accessible.role: Accessible.EditableText
            Accessible.name: qsTr("Attribution (optional)")
-            Accessible.description: qsTr("Textfield for providing attribution")
+            Accessible.description: qsTr("Provide attribution")
            onEditingFinished: {
                MySettings.networkAttribution = attribution.text;
            }
@@ -103,12 +103,12 @@ NOTE: By turning on this feature, you will be sending your data to the GPT4All O
        spacing: 10
        MyButton {
            text: qsTr("Enable")
-            Accessible.description: qsTr("Enable opt-in button")
+            Accessible.description: qsTr("Enable opt-in")
            DialogButtonBox.buttonRole: DialogButtonBox.AcceptRole
        }
        MyButton {
            text: qsTr("Cancel")
-            Accessible.description: qsTr("Cancel opt-in button")
+            Accessible.description: qsTr("Cancel opt-in")
            DialogButtonBox.buttonRole: DialogButtonBox.RejectRole
        }
        background: Rectangle {
--- a/gpt4all-chat/qml/SettingsDialog.qml
+++ b/gpt4all-chat/qml/SettingsDialog.qml
@@ -21,8 +21,8 @@ MyDialog {

    Item {
        Accessible.role: Accessible.Dialog
-        Accessible.name: qsTr("Settings dialog")
-        Accessible.description: qsTr("Dialog containing various application settings")
+        Accessible.name: qsTr("Settings")
+        Accessible.description: qsTr("Contains various application settings")
    }

    ListModel {
--- a/gpt4all-chat/qml/StartupDialog.qml
+++ b/gpt4all-chat/qml/StartupDialog.qml
@@ -133,7 +133,6 @@ model release that uses your data!")
                font.pixelSize: theme.fontSizeLarge
                Accessible.role: Accessible.Paragraph
                Accessible.name: qsTr("Opt-in for anonymous usage statistics")
-                Accessible.description: qsTr("Label for opt-in")
            }

            ButtonGroup {
@@ -162,7 +161,7 @@ model release that uses your data!")
                    font.pixelSize: theme.fontSizeLarge
                    Accessible.role: Accessible.RadioButton
                    Accessible.name: qsTr("Opt-in for anonymous usage statistics")
-                    Accessible.description: qsTr("Radio button to allow opt-in for anonymous usage statistics")
+                    Accessible.description: qsTr("Allow opt-in for anonymous usage statistics")

                    background: Rectangle {
                        color: "transparent"
@@ -203,7 +202,7 @@ model release that uses your data!")
                    font.pixelSize: theme.fontSizeLarge
                    Accessible.role: Accessible.RadioButton
                    Accessible.name: qsTr("Opt-out for anonymous usage statistics")
-                    Accessible.description: qsTr("Radio button to allow opt-out for anonymous usage statistics")
+                    Accessible.description: qsTr("Allow opt-out for anonymous usage statistics")

                    background: Rectangle {
                        color: "transparent"
@@ -249,7 +248,7 @@ model release that uses your data!")
                font.pixelSize: theme.fontSizeLarge
                Accessible.role: Accessible.Paragraph
                Accessible.name: qsTr("Opt-in for network")
-                Accessible.description: qsTr("Checkbox to allow opt-in for network")
+                Accessible.description: qsTr("Allow opt-in for network")
            }

            ButtonGroup {
@@ -276,7 +275,7 @@ model release that uses your data!")
                    font.pixelSize: theme.fontSizeLarge
                    Accessible.role: Accessible.RadioButton
                    Accessible.name: qsTr("Opt-in for network")
-                    Accessible.description: qsTr("Radio button to allow opt-in anonymous sharing of chats to the GPT4All Datalake")
+                    Accessible.description: qsTr("Allow opt-in anonymous sharing of chats to the GPT4All Datalake")

                    background: Rectangle {
                        color: "transparent"
@@ -317,7 +316,7 @@ model release that uses your data!")
                    font.pixelSize: theme.fontSizeLarge
                    Accessible.role: Accessible.RadioButton
                    Accessible.name: qsTr("Opt-out for network")
-                    Accessible.description: qsTr("Radio button to allow opt-out anonymous sharing of chats to the GPT4All Datalake")
+                    Accessible.description: qsTr("Allow opt-out anonymous sharing of chats to the GPT4All Datalake")

                    background: Rectangle {
                        color: "transparent"
--- a/gpt4all-training/build_map.py
+++ b/gpt4all-training/build_map.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import numpy as np
 from nomic import atlas
 import glob
@@ -51,4 +52,4 @@ atlas.map_embeddings(embeddings,
                     colorable_fields=["source", "loss", "trained_on"],
                     build_topic_model=True,
                     topic_label_field="inputs",
-                     reset_project_if_exists=True,)
+                     reset_project_if_exists=True,)
--- a/gpt4all-training/chat/gpt4all-lora-quantized-OSX-intel
+++ b/gpt4all-training/chat/gpt4all-lora-quantized-OSX-intel
--- a/gpt4all-training/chat/gpt4all-lora-quantized-OSX-m1
+++ b/gpt4all-training/chat/gpt4all-lora-quantized-OSX-m1
--- a/gpt4all-training/chat/gpt4all-lora-quantized-linux-x86
+++ b/gpt4all-training/chat/gpt4all-lora-quantized-linux-x86
--- a/gpt4all-training/chat/gpt4all-lora-quantized-win64.exe
+++ b/gpt4all-training/chat/gpt4all-lora-quantized-win64.exe
--- a/gpt4all-training/clean.py
+++ b/gpt4all-training/clean.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import numpy as np
 import glob
 import os
@@ -71,4 +72,4 @@ for file in glob.glob(os.path.join(prompt_generation_dir, "*.jsonl")):

    clean_name = file.split(".jsonl")[0] + "_clean.jsonl"
    print(f"writing to {curr_len} rows to {clean_name}")
-    df.to_json(clean_name, orient="records", lines=True)
+    df.to_json(clean_name, orient="records", lines=True)
--- a/gpt4all-training/create_hostname.sh
+++ b/gpt4all-training/create_hostname.sh
--- a/gpt4all-training/eval_figures.py
+++ b/gpt4all-training/eval_figures.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import glob
 import pickle
 import numpy as np
--- a/gpt4all-training/eval_self_instruct.py
+++ b/gpt4all-training/eval_self_instruct.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import json
 import torch
 import pickle
--- a/gpt4all-training/generate.py
+++ b/gpt4all-training/generate.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from transformers import AutoModelForCausalLM, AutoTokenizer
 from peft import PeftModelForCausalLM
 from read import read_config
--- a/gpt4all-training/inference.py
+++ b/gpt4all-training/inference.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
 import torch.nn as nn
--- a/gpt4all-training/launcher.sh
+++ b/gpt4all-training/launcher.sh
--- a/gpt4all-training/train.py
+++ b/gpt4all-training/train.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 import os
 from transformers import AutoModelForCausalLM, AutoTokenizer, get_scheduler
 import torch
Author	SHA1	Message	Date
cebtenzzre	3c561bcdf2	python: bump bindings version for AMD fixes	2023-10-30 17:00:05 -04:00
Adam Treat	ffef60912f	Update to llama.cpp	2023-10-30 11:40:16 -04:00
Adam Treat	bc88271520	Bump version to v2.5.3 and release notes.	2023-10-30 11:15:12 -04:00
cebtenzzre	5508e43466	build_and_run: clarify which additional Qt libs are needed Signed-off-by: cebtenzzre <cebtenzzre@gmail.com>	2023-10-30 10:37:32 -04:00
cebtenzzre	79a5522931	fix references to old backend implementations	2023-10-30 10:37:05 -04:00
Adam Treat	f529d55380	Move this logic to QML.	2023-10-30 09:57:21 -04:00
Adam Treat	f5f22fdbd0	Update llama.cpp for latest bugfixes.	2023-10-28 17:47:55 -04:00
Adam Treat	5c0d077f74	Remove leading whitespace in responses.	2023-10-28 16:53:42 -04:00
Adam Treat	131cfcdeae	Don't regenerate the name for deserialized chats.	2023-10-28 16:41:23 -04:00
Adam Treat	dc2e7d6e9b	Don't start recalculating context immediately upon switching to a new chat but rather wait until the first prompt. This allows users to switch between chats fast and to delete chats more easily. Fixes issue #1545	2023-10-28 16:41:23 -04:00
cebtenzzre	7bcd9e8089	update llama.cpp-mainline	2023-10-27 19:29:36 -04:00
cebtenzzre	fd0c501d68	backend: support GGUFv3 (#1582 )	2023-10-27 17:07:23 -04:00
Adam Treat	14b410a12a	Update to latest version of llama.cpp which fixes issue 1507.	2023-10-27 12:08:35 -04:00
Adam Treat	ab96035bec	Update to llama.cpp submodule for some vulkan fixes.	2023-10-26 13:46:38 -04:00
Aaron Miller	9193a9517a	make codespell happy again (#1574 ) * make codespell happy again * no belong Signed-off-by: Aaron Miller <apage43@ninjawhale.com> --------- Signed-off-by: Aaron Miller <apage43@ninjawhale.com>	2023-10-26 10:07:06 -04:00
cebtenzzre	8d7a3f26d3	gpt4all-training: delete old chat executables Signed-off-by: cebtenzzre <cebtenzzre@gmail.com>	2023-10-25 13:27:15 -07:00
Andriy Mulyar	3444a47cad	Update README.md Signed-off-by: Andriy Mulyar <andriy.mulyar@gmail.com>	2023-10-24 22:03:21 -04:00
Adam Treat	89a59e7f99	Bump version and add release notes for 2.5.1	2023-10-24 13:13:04 -04:00
cebtenzzre	f5dd74bcf0	models2.json: add tokenizer merges to mpt-7b-chat model (#1563 )	2023-10-24 12:43:49 -04:00
cebtenzzre	78d930516d	app.py: change default model to Mistral Instruct (#1564 )	2023-10-24 12:43:30 -04:00
cebtenzzre	83b8eea611	README: add clear note about new GGUF format Signed-off-by: cebtenzzre <cebtenzzre@gmail.com>	2023-10-24 12:14:29 -04:00
Andriy Mulyar	1bebe78c56	Update README.md Signed-off-by: Andriy Mulyar <andriy.mulyar@gmail.com>	2023-10-24 12:05:46 -04:00
Andriy Mulyar	b75a209374	Update README.md Signed-off-by: Andriy Mulyar <andriy.mulyar@gmail.com>	2023-10-24 12:04:19 -04:00
cebtenzzre	e90263c23f	make scripts executable (#1555 )	2023-10-24 09:28:21 -04:00
Aaron Miller	f414c28589	llmodel: whitelist library name patterns this fixes some issues that were being seen on installed windows builds of 2.5.0 only load dlls that actually might be model impl dlls, otherwise we pull all sorts of random junk into the process before it might expect to be Signed-off-by: Aaron Miller <apage43@ninjawhale.com>	2023-10-23 21:40:14 -07:00
cebtenzzre	7e5e84fbb7	python: change default extension to .gguf (#1559 )	2023-10-23 22:18:50 -04:00
cebtenzzre	37b007603a	bindings: replace references to GGMLv3 models with GGUF (#1547 )	2023-10-22 11:58:28 -04:00
cebtenzzre	c25dc51935	chat: fix syntax error in main.qml	2023-10-21 21:22:37 -07:00
Thomas	34daf240f9	Update Dockerfile.buildkit (#1542 ) corrected model download directory Signed-off-by: Thomas <tvhdev@vonhaugwitz-softwaresolutions.de>	2023-10-21 14:56:06 -04:00
Victor Tsaran	721d854095	chat: improve accessibility fields (#1532 ) Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>	2023-10-21 10:38:46 -04:00
Andriy Mulyar	d50803ff8e	GGUF Python Release (#1539 )	2023-10-19 19:11:03 -04:00
Adam Treat	9e99cf937a	Add release notes for 2.5.0 and bump the version.	2023-10-19 16:25:55 -04:00
cebtenzzre	245c5ce5ea	update default model URLs (#1538 )	2023-10-19 15:25:37 -04:00
cebtenzzre	4338e72a51	MPT: use upstream llama.cpp implementation (#1515 )	2023-10-19 15:25:17 -04:00