WIP: polymorphic ChatItem replacement

Signed-off-by: Adam Treat <treat.adam@gmail.com>
Co-authored-by: Jared Van Bortel <jared@nomic.ai>

.circleci/config.yml

@@ -0,0 +1,18 @@
+version: 2.1
+setup: true
+orbs:
+  path-filtering: circleci/path-filtering@1.1.0
+
+workflows:
+  version: 2.1
+  generate-config:
+    jobs:
+      - path-filtering/filter:
+          base-revision: main
+          config-path: .circleci/continue_config.yml
+          mapping: |
+            .circleci/.* run-all-workflows true
+            gpt4all-backend/.* run-all-workflows true
+            gpt4all-bindings/python/.* run-python-workflow true
+            gpt4all-bindings/typescript/.* run-ts-workflow true
+            gpt4all-chat/.* run-chat-workflow true

.circleci/grab_notary_id.py

@@ -0,0 +1,17 @@
+import re
+import sys
+
+ID_REG = r"id: (.*)"
+
+def main() -> None:
+    notary_log = sys.argv[1]
+    with open(notary_log, "r") as f:
+        notary_output = f.read()
+        id_m = re.search(ID_REG, notary_output)
+        if id_m:
+            print(id_m.group(1))
+        else:
+            raise RuntimeError("Unable to parse ID from notarization logs")
+
+if __name__ == "__main__":
+    main()

.codespellrc

@@ -0,0 +1,3 @@
+[codespell]
+ignore-words-list = blong, afterall, assistent, crasher, requestor
+skip = ./.git,./gpt4all-chat/translations,*.pdf,*.svg,*.lock

.github/ISSUE_TEMPLATE/bindings-bug.md

@@ -0,0 +1,35 @@
+---
+name: "\U0001F6E0 Bindings Bug Report"
+about: A bug report for the GPT4All Bindings
+labels: ["bindings", "bug-unconfirmed"]
+---
+
+<!-- Before creating a new issue, please make sure to take a few moments to check the issue tracker for existing issues about the bug. -->
+
+### Bug Report
+
+<!-- A clear and concise description of what the bug is. -->
+
+### Example Code
+
+<!-- Please provide a minimal code example that can be used to experience this issue. Delete this section if it does not apply. -->
+
+### Steps to Reproduce
+
+<!-- List the steps that should be taken to experience this issue. -->
+
+1.
+2.
+3.
+
+### Expected Behavior
+
+<!-- In a few words, what did you expect to happen? -->
+
+### Your Environment
+
+- Bindings version (e.g. "Version" from `pip show gpt4all`):
+- Operating System:
+- Chat model used (if applicable):
+
+<!-- You can freely edit this text, please remove all the lines you believe are unnecessary. -->

.github/ISSUE_TEMPLATE/chat-bug.md

@@ -0,0 +1,31 @@
+---
+name: "\U0001F4AC GPT4All Bug Report"
+about: A bug report for GPT4All Chat
+labels: ["chat", "bug-unconfirmed"]
+---
+
+<!-- Before creating a new issue, please make sure to take a few moments to check the issue tracker for existing issues about the bug. -->
+
+### Bug Report
+
+<!-- A clear and concise description of what the bug is. -->
+
+### Steps to Reproduce
+
+<!-- List the steps that should be taken to experience this issue. Provide any relevant information about your configuration, and describe anything that was unexpected. -->
+
+1.
+2.
+3.
+
+### Expected Behavior
+
+<!-- In a few words, what did you expect to happen? -->
+
+### Your Environment
+
+- GPT4All version:
+- Operating System:
+- Chat model used (if applicable):
+
+<!-- You can freely edit this text, please remove all the lines you believe are unnecessary. -->

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+version: 2.1

.github/ISSUE_TEMPLATE/documentation.md

@@ -0,0 +1,9 @@
+---
+name: "\U0001F4C4 Documentation"
+about: An issue related to the GPT4All documentation
+labels: ["documentation"]
+---
+
+### Documentation
+
+<!-- Please describe the issue with the documentation as clearly as possible. -->

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,10 @@
+---
+name: "\U0001F680 Feature Request"
+about: Submit a proposal/request for a new GPT4All feature
+title: "[Feature] Feature request title..."
+labels: ["enhancement"]
+---
+
+### Feature Request
+
+<!-- A clear and concise description of the feature proposal. -->

.github/ISSUE_TEMPLATE/other-bug.md

@@ -0,0 +1,32 @@
+---
+name: "\U0001F41B Other Bug Report"
+about: A bug in another component of GPT4All
+labels: ["bug-unconfirmed"]
+---
+
+<!-- Before creating a new issue, please make sure to take a few moments to check the issue tracker for existing issues about the bug. -->
+
+### Bug Report
+
+<!-- A clear and concise description of what the bug is. -->
+
+### Steps to Reproduce
+
+<!-- List the steps that should be taken to experience this issue. Provide any relevant information about your configuration, and describe anything that was unexpected. If this bug involves original code, please provide a minimal version that can reproduce the issue. -->
+
+1.
+2.
+3.
+
+### Expected Behavior
+
+<!-- In a few words, what did you expect to happen? -->
+
+### Your Environment
+
+- GPT4All version (if applicable):
+- Operating System:
+- Chat model used (if applicable):
+
+<!-- You can freely edit this text, please remove all the lines you believe are unnecessary. -->
+

.github/pull_request_template.md

@@ -0,0 +1,19 @@
+## Describe your changes
+
+## Issue ticket number and link
+
+## Checklist before requesting a review
+- [ ] I have performed a self-review of my code.
+- [ ] If it is a core feature, I have added thorough tests.
+- [ ] I have added thorough documentation for my code.
+- [ ] I have tagged PR with relevant project labels. I acknowledge that a PR without labels may be dismissed.
+- [ ] If this PR addresses a bug, I have provided both a screenshot/video of the original bug and the working solution.
+
+## Demo
+<!-- Screenshots or video of new or updated code changes !-->
+
+### Steps to Reproduce
+<!-- Steps to reproduce demo !-->
+
+## Notes
+<!-- Any other relevant information to include about PR !-->

.github/workflows/close_issues.yml

@@ -0,0 +1,33 @@
+# This workflow will close issues that do not have labels or additional comments.
+# Trigger manually.
+
+name: "Close Issues"
+on:
+  workflow_dispatch:
+
+jobs:
+  close_issues:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Close issues without label or comment
+      uses: actions/github-script@v3
+      with:
+        github-token: ${{secrets.GITHUB_TOKEN}}
+        script: |
+          const repo = context.repo;
+          let page = 1;
+          let issues = [];
+          while (true) {
+            const result = await github.issues.listForRepo({...repo, per_page: 100, page: page});
+            if (result.data.length === 0) break;
+            issues = issues.concat(result.data);
+            page += 1;
+          }
+          for (let { number } of issues) {
+            const issueData = await github.issues.get({...repo, issue_number: number});
+            const comments = await github.issues.listComments({...repo, issue_number: number});
+            if (issueData.data.labels.length === 0 && comments.data.length < 1) {
+              await github.issues.update({...repo, issue_number: number, state: 'closed'});
+              await github.issues.createComment({...repo, issue_number: number, body: 'Issue closed as it does not have any labels or comments.'});
+            }
+          }

.github/workflows/codespell.yml

@@ -0,0 +1,19 @@
+---
+name: Codespell
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  codespell:
+    name: Check for spelling errors
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Codespell
+        uses: codespell-project/actions-codespell@v2

.gitignore

@@ -1,3 +1,6 @@
+*.arrow
+squad_*
+*sbert_embedded*
 *.pkl
 ckpts*
 .deepspeed_env
@@ -169,4 +172,20 @@ cython_debug/
 
 # vs code
 .vscode
-*.bin
+*.bin
+
+.DS_Store
+
+# gpt4all-chat
+CMakeLists.txt.user
+gpt4all-chat/models/*
+build_*
+build-*
+cmake-build-*
+/gpt4all-chat/tests/python/config.py
+
+# IntelliJ
+.idea/
+
+# LLM models
+*.gguf

.gitmodules

@@ -1,3 +1,19 @@
-[submodule "peft"]
-	path = peft
-	url = https://github.com/huggingface/peft.git
+[submodule "llama.cpp-mainline"]
+	path = gpt4all-backend/deps/llama.cpp-mainline
+	url = https://github.com/nomic-ai/llama.cpp.git
+	branch = master
+[submodule "gpt4all-chat/usearch"]
+	path = gpt4all-chat/deps/usearch
+	url = https://github.com/nomic-ai/usearch.git
+[submodule "gpt4all-chat/deps/SingleApplication"]
+	path = gpt4all-chat/deps/SingleApplication
+	url = https://github.com/nomic-ai/SingleApplication.git
+[submodule "gpt4all-chat/deps/fmt"]
+	path = gpt4all-chat/deps/fmt
+	url = https://github.com/fmtlib/fmt.git
+[submodule "gpt4all-chat/deps/DuckX"]
+	path = gpt4all-chat/deps/DuckX
+	url = https://github.com/nomic-ai/DuckX.git
+[submodule "gpt4all-chat/deps/QXlsx"]
+	path = gpt4all-chat/deps/QXlsx
+	url = https://github.com/nomic-ai/QXlsx.git

CONTRIBUTING.md

@@ -0,0 +1,91 @@
+# Contributing
+
+When contributing to this repository, please first discuss the change you wish to make via issue,
+email, or any other method with the owners of this repository before making a change. 
+
+Please note we have a code of conduct, please follow it in all your interactions with the project.
+
+## Pull Request Process
+
+1. Ensure any install or build dependencies are removed before the end of the layer when doing a build.
+2. Make sure Pull Request is tagged with appropriate project identifiers and has a clear description of contribution.
+3. Any new or updated code must have documentation and preferably tests included with Pull Request.
+4. Significant feature or code changes should provide a short video or screenshot demo.
+4. Fill out relevant parts of Pull Request template. 
+4. Pull requests must have sign-off from one other developer. Reach out to a repository owner once your
+   code is ready to be merged into `main`.
+
+## Code of Conduct
+
+### Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+### Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+### Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+### Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+### Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at support@nomic.ai. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+### Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

MAINTAINERS.md

@@ -0,0 +1,82 @@
+# MAINTAINERS
+
+## Rules
+
+* All content inside GPT4All shall have a documented maintainer
+* If a maintainer decides to retire or resign a call for volunteers will go
+  out
+* If no further maintainer can be found in a reasonable time frame, then the
+  content will be marked deprecated and removed in time
+
+## Job
+
+Maintainers will be...
+
+1. Responsible for overseeing content under their stewardship
+2. Responsible for triaging new issues, reviewing PRs, assigning priority
+   to tasks
+3. Responsible for keeping content in sufficient quality in a timely fashion
+
+## List
+
+Adam Treat ([@manyoso](https://github.com/manyoso))<br/>
+E-mail: adam@nomic.ai<br/>
+Discord: `@gonzochess75`
+- Overall project maintainer
+- Chat UI
+
+Jared Van Bortel ([@cebtenzzre](https://github.com/cebtenzzre))<br/>
+E-mail: jared@nomic.ai<br/>
+Discord: `@cebtenzzre`
+- gpt4all-backend
+- Python binding
+- Python CLI app
+
+Jacob Nguyen ([@jacoobes](https://github.com/jacoobes))<br/>
+Discord: `@jacoobes`<br/>
+E-mail: `jacoobes@sern.dev`
+- TypeScript binding
+
+Dominik ([@cosmic-snow](https://github.com/cosmic-snow))<br/>
+E-mail: cosmic-snow@mailfence.com<br/>
+Discord: `@cosmic__snow`
+- Community documentation (GitHub Wiki)
+
+Max Cembalest ([@mcembalest](https://github.com/mcembalest))<br/>
+E-mail: max@nomic.ai<br/>
+Discord: `@maxcembalest.`
+- Official documentation (gpt4all-bindings/python/docs -> https://docs.gpt4all.io/)
+
+Thiago Ramos ([@thiagojramos](https://github.com/thiagojramos))<br/>
+E-mail: thiagojramos@outlook.com<br/>
+- pt\_BR translation
+
+Victor Emanuel ([@SINAPSA-IC](https://github.com/SINAPSA-IC))<br/>
+E-mail: contact@sinapsaro.ro<br/>
+Discord: `@sinapsa_ic_56124_99632`
+- ro\_RO translation
+
+不知火 Shiranui ([@supersonictw](https://github.com/supersonictw))<br/>
+E-mail: supersonic@livemail.tw<br/>
+Discord: `@supersonictw`
+- zh\_TW translation
+
+Jeremy Tayco ([@jstayco](https://github.com/jstayco))<br/>
+E-mail: jstayco@protonmail.ch<br/>
+Discord: `@vertana`
+- es\_MX translation
+
+Riccardo Giovanetti ([@Harvester62](https://github.com/Harvester62))<br/>
+E-mail: riccardo.giovanetti@gmail.com<br/>
+Discord: `@harvester62`
+- it\_IT translation
+
+Tim ([@Tim453](https://github.com/Tim453))<br/>
+E-mail: tim453@mailbox.org<br/>
+Discord: `@Tim453`
+- Flatpak
+
+Jack ([@wuodoo](https://github.com/wuodoo))<br/>
+E-mail: 2296103047@qq.com><br/>
+Discord: `@mikage`
+- zh\_CN translation

README.md

@@ -1,319 +1,114 @@
 <h1 align="center">GPT4All</h1>
-<p align="center">Demo, data, and code to train open-source assistant-style large language model based on GPT-J and LLaMa</p>
+
 <p align="center">
-<a href="https://static.nomic.ai/gpt4all/2023_GPT4All-J_Technical_Report_2.pdf">:green_book: Technical Report 2: GPT4All-J </a>
+  <a href="https://www.nomic.ai/gpt4all">Website</a> &bull; <a href="https://docs.gpt4all.io">Documentation</a> &bull; <a href="https://discord.gg/mGZE39AS3e">Discord</a> &bull; <a href="https://www.youtube.com/watch?v=gQcZDXRVJok">YouTube Tutorial</a>
 </p>
 
 <p align="center">
-<a href="https://s3.amazonaws.com/static.nomic.ai/gpt4all/2023_GPT4All_Technical_Report.pdf">:green_book: Technical Report 1: GPT4All</a>
+  GPT4All runs large language models (LLMs) privately on everyday desktops & laptops.
+</p>
+<p align="center">
+  No API calls or GPUs required - you can just download the application and <a href="https://docs.gpt4all.io/gpt4all_desktop/quickstart.html#quickstart">get started</a>.
 </p>
 
 <p align="center">
-<a href="https://github.com/nomic-ai/pyllamacpp">:snake: Official Python Bindings</a>
+  Read about what's new in <a href="https://www.nomic.ai/blog/tag/gpt4all">our blog</a>.
 </p>
-
 <p align="center">
-<a href="https://github.com/nomic-ai/gpt4all-ts">:computer: Official Typescript Bindings</a>
+  <a href="https://nomic.ai/gpt4all/#newsletter-form">Subscribe to the newsletter</a>
 </p>
 
-<p align="center">
-<a href="https://github.com/nomic-ai/gpt4all-ui">:speech_balloon: Official Web Chat Interface</a>
-</p>
-
-<p align="center">
-<a href="https://github.com/nomic-ai/gpt4all-chat">:speech_balloon: Official Chat Interface</a>
-</p>
-
-<p align="center">
-<a href="https://python.langchain.com/en/latest/modules/models/llms/integrations/gpt4all.html">🦜️🔗 Official Langchain Backend</a> 
-</p>
-
-
-<p align="center">
-<a href="https://discord.gg/mGZE39AS3e">Discord</a>
-</p>
-
-
-
+https://github.com/nomic-ai/gpt4all/assets/70534565/513a0f15-4964-4109-89e4-4f9a9011f311
 
 <p align="center">
 GPT4All is made possible by our compute partner <a href="https://www.paperspace.com/">Paperspace</a>.
 </p>
+<p align="center">
+ <a href="https://www.phorm.ai/query?projectId=755eecd3-24ad-49cc-abf4-0ab84caacf63"><img src="https://img.shields.io/badge/Phorm-Ask_AI-%23F2777A.svg" alt="phorm.ai"></a>
+</p>
 
+## Download Links
 
+<p>
+  &mdash; <a href="https://gpt4all.io/installers/gpt4all-installer-win64.exe">
+    <img src="gpt4all-bindings/python/docs/assets/windows.png" style="height: 1em; width: auto" /> Windows Installer
+  </a> &mdash;
+</p>
+<p>
+  &mdash; <a href="https://gpt4all.io/installers/gpt4all-installer-darwin.dmg">
+    <img src="gpt4all-bindings/python/docs/assets/mac.png" style="height: 1em; width: auto" /> macOS Installer
+  </a> &mdash;
+</p>
+<p>
+  &mdash; <a href="https://gpt4all.io/installers/gpt4all-installer-linux.run">
+    <img src="gpt4all-bindings/python/docs/assets/ubuntu.svg" style="height: 1em; width: auto" /> Ubuntu Installer
+  </a> &mdash;
+</p>
+<p>
+  Windows and Linux require Intel Core i3 2nd Gen / AMD Bulldozer, or better. x86-64 only, no ARM.
+</p>
+<p>
+  macOS requires Monterey 12.6 or newer. Best results with Apple Silicon M-series processors.
+</p>
 
-## GPT4All-J: An Apache-2 Licensed GPT4All Model
-![gpt4all-j-demo](https://user-images.githubusercontent.com/13879686/231876409-e3de1934-93bb-4b4b-9013-b491a969ebbc.gif)
+See the full [System Requirements](gpt4all-chat/system_requirements.md) for more details.
 
-Run on an M1 Mac (not sped up!)
+<br/>
+<br/>
+<p>
+  <a href='https://flathub.org/apps/io.gpt4all.gpt4all'>
+    <img style="height: 2em; width: auto" alt='Get it on Flathub' src='https://flathub.org/api/badge'><br/>
+    Flathub (community maintained)
+  </a>
+</p>
 
+## Install GPT4All Python
 
-### GPT4All-J Chat UI Installers
-Installs a native chat-client with auto-update functionality that runs on your desktop with the GPT4All-J model baked into it.
+`gpt4all` gives you access to LLMs with our Python client around [`llama.cpp`](https://github.com/ggerganov/llama.cpp) implementations. 
 
-[Mac/OSX](https://gpt4all.io/installers/gpt4all-0.1.0-Darwin.dmg)
-
-[Windows](https://gpt4all.io/installers/gpt4all-0.1.0-win64.exe)
-
-[Ubuntu](https://gpt4all.io/installers/gpt4all-0.1.0-Linux.run)
-
-These files are not yet cert signed by Windows/Apple so you will see security warnings on initial installation. We did not want to delay release while waiting for their process to complete.
-
-Find the most up-to-date information on the [GPT4All Website](https://gpt4all.io/)
-
-### Raw Model
-[ggml Model Download Link](https://gpt4all.io/models/ggml-gpt4all-j.bin)
-
-Note this model is only compatible with the C++ bindings found [here](https://github.com/nomic-ai/gpt4all-chat). It will not work with any existing llama.cpp bindings as we had to do a large fork of llama.cpp. GPT4All will support the ecosystem around this new C++ backend going forward.
-
-Python bindings are imminent and will be integrated into this [repository](https://github.com/nomic-ai/pyllamacpp). Stay tuned on the [GPT4All discord](https://discord.gg/mGZE39AS3e) for updates.
-
-## Training GPT4All-J
-
-Please see [GPT4All-J Technical Report](https://static.nomic.ai/gpt4all/2023_GPT4All-J_Technical_Report_2.pdf) for details.
-
-### GPT4All-J Training Data
-
-- We are releasing the curated training data for anyone to replicate GPT4All-J here: [GPT4All-J Training Data](https://huggingface.co/datasets/nomic-ai/gpt4all-j-prompt-generations)
-   - [Atlas Map of Prompts](https://atlas.nomic.ai/map/gpt4all-j-prompts-curated)
-   - [Atlas Map of Responses](https://atlas.nomic.ai/map/gpt4all-j-response-curated)
-
-### GPT4All-J Training Instructions
+Nomic contributes to open source software like [`llama.cpp`](https://github.com/ggerganov/llama.cpp) to make LLMs accessible and efficient **for all**.
 
 ```bash
-accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16  --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config_gptj.json train.py --config configs/train/finetune_gptj.yaml
+pip install gpt4all
+```
+
+```python
+from gpt4all import GPT4All
+model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf") # downloads / loads a 4.66GB LLM
+with model.chat_session():
+    print(model.generate("How can I run LLMs efficiently on my laptop?", max_tokens=1024))
 ```
 
 
-# Original GPT4All Model (based on GPL Licensed LLaMa)
+## Integrations
 
+:parrot::link: [Langchain](https://python.langchain.com/v0.2/docs/integrations/providers/gpt4all/)
+:card_file_box: [Weaviate Vector Database](https://github.com/weaviate/weaviate) - [module docs](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-gpt4all)
+:telescope: [OpenLIT (OTel-native Monitoring)](https://github.com/openlit/openlit) - [Docs](https://docs.openlit.io/latest/integrations/gpt4all)
 
+## Release History
+- **July 2nd, 2024**: V3.0.0 Release
+    - Fresh redesign of the chat application UI
+    - Improved user workflow for LocalDocs
+    - Expanded access to more model architectures
+- **October 19th, 2023**: GGUF Support Launches with Support for:
+    - Mistral 7b base model, an updated model gallery on our website, several new local code models including Rift Coder v1.5
+    - [Nomic Vulkan](https://blog.nomic.ai/posts/gpt4all-gpu-inference-with-vulkan) support for Q4\_0 and Q4\_1 quantizations in GGUF.
+    - Offline build support for running old versions of the GPT4All Local LLM Chat Client.
+- **September 18th, 2023**: [Nomic Vulkan](https://blog.nomic.ai/posts/gpt4all-gpu-inference-with-vulkan) launches supporting local LLM inference on NVIDIA and AMD GPUs.
+- **July 2023**: Stable support for LocalDocs, a feature that allows you to privately and locally chat with your data.
+- **June 28th, 2023**: [Docker-based API server] launches allowing inference of local LLMs from an OpenAI-compatible HTTP endpoint.
 
-![gpt4all-lora-demo](https://user-images.githubusercontent.com/13879686/228352356-de66ca7a-df70-474e-b929-2e3656165051.gif)
+[Docker-based API server]: https://github.com/nomic-ai/gpt4all/tree/cef74c2be20f5b697055d5b8b506861c7b997fab/gpt4all-api
 
-Run on M1 Mac (not sped up!)
+## Contributing
+GPT4All welcomes contributions, involvement, and discussion from the open source community!
+Please see CONTRIBUTING.md and follow the issues, bug reports, and PR markdown templates.
 
-# Try it yourself
+Check project discord, with project owners, or through existing issues/PRs to avoid duplicate work.
+Please make sure to tag all of the above with relevant project identifiers or your contribution could potentially get lost.
+Example tags: `backend`, `bindings`, `python-bindings`, `documentation`, etc.
 
-Here's how to get started with the CPU quantized GPT4All model checkpoint:
-
-1. Download the `gpt4all-lora-quantized.bin` file from [Direct Link](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin) or [[Torrent-Magnet]](https://tinyurl.com/gpt4all-lora-quantized).
-2. Clone this repository, navigate to `chat`, and place the downloaded file there.
-3. Run the appropriate command for your OS:
-   - M1 Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-m1`
-   - Linux: `cd chat;./gpt4all-lora-quantized-linux-x86`
-   - Windows (PowerShell): `cd chat;./gpt4all-lora-quantized-win64.exe`
-   - Intel Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-intel`
-
-For custom hardware compilation, see our [llama.cpp](https://github.com/zanussbaum/gpt4all.cpp) fork.
-
------------
-Find all compatible models in the GPT4All Ecosystem section.
-
-[Secret Unfiltered Checkpoint](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin) - [[Torrent]](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin.torrent)
-
-This model had all refusal to answer responses removed from training. Try it with:
-- M1 Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-m1 -m gpt4all-lora-unfiltered-quantized.bin`
-- Linux: `cd chat;./gpt4all-lora-quantized-linux-x86 -m gpt4all-lora-unfiltered-quantized.bin`
-- Windows (PowerShell): `cd chat;./gpt4all-lora-quantized-win64.exe -m gpt4all-lora-unfiltered-quantized.bin`
-- Intel Mac/OSX: `cd chat;./gpt4all-lora-quantized-OSX-intel -m gpt4all-lora-unfiltered-quantized.bin`
------------
-Note: the full model on GPU (16GB of RAM required) performs much better in our qualitative evaluations.
-
-# Python Client
-## CPU Interface
-To run GPT4All in python, see the new [official Python bindings](https://github.com/nomic-ai/pyllamacpp).
-
-The old bindings are still available but now deprecated. They will not work in a notebook environment.
-To get running using the python client with the CPU interface, first install the [nomic client](https://github.com/nomic-ai/nomic) using `pip install nomic`
-Then, you can use the following script to interact with GPT4All:
-```
-from nomic.gpt4all import GPT4All
-m = GPT4All()
-m.open()
-m.prompt('write me a story about a lonely computer')
-```
-
-## GPU Interface
-There are two ways to get up and running with this model on GPU.
-The setup here is slightly more involved than the CPU model.
-1. clone the nomic client [repo](https://github.com/nomic-ai/nomic) and run `pip install .[GPT4All]` in the home dir.
-2. run `pip install nomic` and install the additional deps from the wheels built [here](https://github.com/nomic-ai/nomic/tree/main/bin)
-
-Once this is done, you can run the model on GPU with a script like the following:
-```
-from nomic.gpt4all import GPT4AllGPU
-m = GPT4AllGPU(LLAMA_PATH)
-config = {'num_beams': 2,
-          'min_new_tokens': 10,
-          'max_length': 100,
-          'repetition_penalty': 2.0}
-out = m.generate('write me a story about a lonely computer', config)
-print(out)
-```
-Where LLAMA_PATH is the path to a Huggingface Automodel compliant LLAMA model.
-Nomic is unable to distribute this file at this time.
-We are working on a GPT4All that does not have this limitation right now.
-
-You can pass any of the [huggingface generation config params](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig) in the config.
-
-# GPT4All Compatibility Ecosystem
-Edge models in the GPT4All Ecosystem. Please PR as the [community grows](https://huggingface.co/models?sort=modified&search=4bit).
-Feel free to convert this to a more structured table.
-
-- [gpt4all](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized.bin.md5)]
-   - [gpt4all-ggml-converted](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-quantized-ggml.bin.md5)]
-- [gpt4all-unfiltered](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin) [[MD5 Signature](https://the-eye.eu/public/AI/models/nomic-ai/gpt4all/gpt4all-lora-unfiltered-quantized.bin.md5)]
-- [ggml-vicuna-7b-4bit](https://huggingface.co/eachadea/ggml-vicuna-7b-4bit)
-- [vicuna-13b-GPTQ-4bit-128g](https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g)
-- [LLaMa-Storytelling-4Bit](https://huggingface.co/GamerUntouch/LLaMa-Storytelling-4Bit)
-- [Alpaca Native 4bit](https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/tree/main)
-
-
-# Roadmap
-## Short Term
- - <span style="color:green">(Done)</span> Train a GPT4All model based on GPTJ to alleviate llama distribution issues.
- - <span style="color:green">(Done)</span> Create improved CPU and GPU interfaces for this model.
- - <span style="color:green">(Done)</span> [Integrate llama.cpp bindings](https://github.com/nomic-ai/pyllamacpp)
- - <span style="color:green">(Done)</span> [Create a good conversational chat interface for the model.](https://github.com/nomic-ai/gpt4all-ui)
- - <span style="color:green">(Done)</span> [Allow users to opt in and submit their chats for subsequent training runs](https://github.com/nomic-ai/gpt4all-ui)
-
-## Medium Term
- - <span style="color:red">(NOT STARTED)</span> Integrate GPT4All with [Atlas](https://atlas.nomic.ai) to allow for document retrieval.
-   - BLOCKED by GPT4All based on GPTJ
- - <span style="color:red">(Done)</span> Integrate GPT4All with Langchain.
- - <span style="color:green">(IN PROGRESS)</span> Build easy custom training scripts to allow users to fine tune models.
-
-## Long Term
- - <span style="color:red">(NOT STARTED)</span> Allow anyone to curate training data for subsequent GPT4All releases using Atlas.
- - <span style="color:green">(IN PROGRESS)</span> Democratize AI. 
-
-# Reproducibility
-
-Trained Model Weights:
-- gpt4all-lora (four full epochs of training):  https://huggingface.co/nomic-ai/gpt4all-lora
-- gpt4all-lora-epoch-2 (three full epochs of training) https://huggingface.co/nomic-ai/gpt4all-lora-epoch-2
-- gpt4all-j (one full epoch of training) (https://huggingface.co/nomic-ai/gpt4all-j)
-- gpt4all-j-lora (one full epoch of training) (https://huggingface.co/nomic-ai/gpt4all-j-lora)
-
-Raw Data:
-- [Training Data Without P3](https://huggingface.co/datasets/nomic-ai/gpt4all_prompt_generations)
-  - Explorer: https://atlas.nomic.ai/map/gpt4all_data_clean_without_p3
-- [Full Dataset with P3](https://huggingface.co/datasets/nomic-ai/gpt4all_prompt_generations_with_p3)
-  - Explorer: https://atlas.nomic.ai/map/gpt4all_data_clean
-- [GPT4All-J Dataset](https://huggingface.co/datasets/nomic-ai/gpt4all-j-prompt-generations)
-   -  Explorer Indexed on Prompts: https://atlas.nomic.ai/map/gpt4all-j-prompts-curated
-   -  Exporer Indexed on Responses: https://atlas.nomic.ai/map/gpt4all-j-response-curated
-
-We are not distributing a LLaMa 7B checkpoint.
-
-You can reproduce our trained model by doing the following:
-
-## Setup
-
-Clone the repo
-
-```
-git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
-git submodule update --init
-```
-
-Setup the environment
-
-```
-python -m pip install -r requirements.txt
-
-cd ../peft
-pip install -e .
-```
-
-## Training
-
-```bash
-accelerate launch --dynamo_backend=inductor --num_processes=8 --num_machines=1 --machine_rank=0 --deepspeed_multinode_launcher standard --mixed_precision=bf16  --use_deepspeed --deepspeed_config_file=configs/deepspeed/ds_config.json train.py --config configs/train/finetune-7b.yaml
-```
-
-## Generate
-
-```bash
-python generate.py --config configs/generate/generate.yaml --prompt "Write a script to reverse a string in Python"
-```
-
-## Need Help?
-
-Join the <a href="https://discord.gg/E7q6fp9bmx"> Discord </a> and ask for help in `#gpt4all-help`
-
-# Sample Generations
-
-### Provide instructions for the given exercise. Leg Raises
->Leg Raises
->1. Stand with your feet shoulder-width apart and your knees slightly bent.
->2. Raise your arms straight out in front of you.
->3. Slowly bend your knees and raise your heels off the ground.
->4. Hold this position for a count of 3, then slowly lower your heels back to the ground.
->5. Repeat the exercise 10 times.
->6. Do not let your knees go beyond your toes.
->7. Keep your back straight and your abdominal muscles engaged.
->8. Do not let your hips sag or your back arch.
->9. Keep your arms straight and your elbows locked.
->10. Do not let your head drop or your shoulders rise.
->11. Breathe normally throughout the exercise.
->12. Rest for 30 seconds before repeating the exercise.
->13. Gradually increase the number of repetitions as you become stronger.
-
-
-### A color description has been provided. Find the CSS code associated with that color. A light red color with a medium light shade of pink
-
->The CSS code associated with the color provided is: #FF6347
-
-
-### Come up with an interesting idea for a new movie plot. Your plot should be described with a title and a summary.
-
-> 1. The title of your movie plot should be "The Last Stand". 
-> 2. The summary of your movie plot should be: A group of mercenaries are hired to protect a small town from a dangerous criminal who is on the run. However, the town's sheriff is not willing to let them take over and decides to take matters into his own hands.
-
-### Reverse a string in python.
-> In Python, you can reverse a list or tuple by using the reversed() function on it. Here's an example of how to use this method with strings: 
->    ```python
->   my_string = "Hello World" # Define your original string here
->   reversed_str = my_string[::-1]   # Use a slice and negative index for reverse order
->   print(reversed_str)     # Output will be reversed version of the inputted string.
-
-### List 10 dogs.
-> 1. Labrador Retriever
-> 2. Golden Retriever
-> 3. Beagle
-> 4. Poodle
-> 5. German Shepherd
-> 6. Bulldog
-> 7. Rottweiler
-> 8. Boxer
-> 9. Shih Tzu
-> 10. Pug
-
-### Write me a poem about the fall of Julius Caesar into a caesar salad in iambic pentameter. 
-
-> The fall of Julius Caesar into a caesar salad<br>
-> In iambic pentameter,<br>
-> The mighty Roman emperor<br>
-> Fell into a salad of lettuce and croutons,<br>
-> His empire crumbling around him,<br>
-> As he was devoured by the hungry diners.<br>
-> The once mighty emperor<br>
-> Was now just a salad topping,<br>
-> His legacy reduced to a mere garnish,<br>
-> As he was consumed by the masses.<br>
-> The fall of Julius Caesar into a caesar salad,<br>
-> In iambic pentameter,<br>
-> Was a tragic end to a great man,<br>
-> Who had once ruled the world with an iron fist.<br>
-> But now, he was just a salad topping,<br>
-> His legacy reduced to a mere garnish,<br>
-> As he was consumed by the masses.
-    
-### What is a three word topic describing the following keywords: baseball, football, soccer: 
-> Sports, athletics, games
-    
 ## Citation
 
 If you utilize this repository, models or data in a downstream project, please consider citing it with:

common/common.cmake

@@ -0,0 +1,41 @@
+function(gpt4all_add_warning_options target)
+    if (MSVC)
+        return()
+    endif()
+    target_compile_options("${target}" PRIVATE
+        # base options
+        -Wall
+        -Wextra
+        # extra options
+        -Wcast-align
+        -Wextra-semi
+        -Wformat=2
+        -Wmissing-include-dirs
+        -Wstrict-overflow=2
+        -Wvla
+        # errors
+        -Werror=format-security
+        -Werror=init-self
+        -Werror=pointer-arith
+        -Werror=undef
+        # disabled warnings
+        -Wno-sign-compare
+        -Wno-unused-parameter
+    )
+    if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
+        target_compile_options("${target}" PRIVATE
+            -Wduplicated-branches
+            -Wduplicated-cond
+            -Wlogical-op
+            -Wno-reorder
+            -Wno-null-dereference
+        )
+    elseif (CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
+        target_compile_options("${target}" PRIVATE
+            -Wunreachable-code-break
+            -Wunreachable-code-return
+            -Werror=pointer-integer-compare
+            -Wno-reorder-ctor
+        )
+    endif()
+endfunction()

eval_data/user_oriented_instructions.jsonl

@@ -1,252 +0,0 @@
-{"id": "user_oriented_task_0", "motivation_app": "Grammarly", "instruction": "The sentence you are given might be too wordy, complicated, or unclear. Rewrite the sentence and make your writing clearer by keeping it concise. Whenever possible, break complex sentences into multiple sentences and eliminate unnecessary words.", "instances": [{"input": "If you have any questions about my rate or if you find it necessary to increase or decrease the scope for this project, please let me know.", "output": "If you have any questions about my rate or find it necessary to increase or decrease this project's scope, please let me know."}]}
-{"id": "user_oriented_task_1", "motivation_app": "Grammarly", "instruction": "Analyze the word choice, phrasing, punctuation, and capitalization in the given email. How may the writer of this email sound to the reader? These tones include Disheartening, Accusatory, Worried, Curious, Surprised, Disapproving, Unassuming, Formal, Assertive, Confident, Appreciative, Concerned, Sad, Informal, Regretful, Encouraging, Egocentric, Joyful, Optimistic, and Excited.", "instances": [{"input": "Hi Jen, \nI hope you're well. Can we catch up today? I'd appreciate your input on my presentation for tomorrow's meeting. I'd especially love it if you could double-check the sales numbers with me. There's a coffee in it for you!", "output": "Confident"}]}
-{"id": "user_oriented_task_2", "motivation_app": "Grammarly", "instruction": "Rewrite the given text and correct grammar, spelling, and punctuation errors.", "instances": [{"input": "If you'd told me year ago that today I would finish a marathon, I would of laughed. Your support had a huge affect on me!", "output": "If you'd told me a year ago that today I would finish a marathon, I would have laughed. Your support had a huge effect on me!"}]}
-{"id": "user_oriented_task_3", "motivation_app": "Google Scholar", "instruction": "You are given a paper citation, convert it to the requested citation style.", "instances": [{"input": "Chicago: Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Lukasz, and Illia Polosukhin. \"Attention Is All You Need.\" arXiv, (2017). https://doi.org/10.48550/arXiv.1706.03762.\nMLA:", "output": "Vaswani, Ashish, et al. \"Attention Is All You Need.\" arXiv, 2017,  https://doi.org/10.48550/arXiv.1706.03762."}]}
-{"id": "user_oriented_task_4", "motivation_app": "Grammarly", "instruction": "Desk jobs require writing a lot of emails, so it isn't surprising we get tired of repeating ourselves. Come up with several synonyms for the given word.", "instances": [{"input": "Sincerely", "output": "Best regards, All the best, Cheers, Best"}]}
-{"id": "user_oriented_task_5", "motivation_app": "Gmail", "instruction": "If you could help me write an email to my friends inviting them to dinner on Friday, it would be greatly appreciated.", "instances": [{"input": "", "output": "Hi there,\n\nI hope you're all doing well. I'm inviting you over for dinner on Friday night. Please let me know if you can make it. I'll be cooking your favorite dishes!\n\nLooking forward to seeing you,"}]}
-{"id": "user_oriented_task_6", "motivation_app": "Gmail", "instruction": "The tone of the email should be changed and rewritten in a more professional manner.", "instances": [{"input": "Subject: Asking for Vacation Time\nHi [Employer], \nI hope all is well. I am writing to request a day off on [date]. I have already taken care of my work responsibilities for that day and will be available to answer any questions or concerns you may have. \nThank you for your time, \n[Your name]", "output": "Subject: Request for Vacation Time\nHello [Employer],\nI am writing to request vacation time from work. I would like to take [Number of Days] off, starting on [Date]. I will be sure to complete all of my work before taking my vacation, and I will be available by email if anything comes up while I'm away. Thank you for your consideration.\n[Your name]"}]}
-{"id": "user_oriented_task_7", "motivation_app": "Twitter", "instruction": "Write a social media post about the call for collaboration on a crowdsourcing project in a persuasive way.", "instances": [{"input": "", "output": "Looking for collaborators on a new crowdsourcing project! This is a great opportunity to get involved in something innovative and impactful. If you're interested, please let me know."}]}
-{"id": "user_oriented_task_8", "motivation_app": "IMDB", "instruction": "My favorite witty review of The Rings of Power series is this:", "instances": [{"input": "", "output": "The Rings of Power series is one of the best things to come out of Middle Earth since, well, the Ring itself! If you're a fan of fantasy, adventure, and intrigue, then this is definitely the series for you. The characters are richly drawn and the plot is twisty and full of surprises. I can't recommend it highly enough!"}]}
-{"id": "user_oriented_task_9", "motivation_app": "you.com (YouWrite)", "instruction": "A helpful writing exercise is writing a story without using adjectives or adverbs. This will help you learn to use other words and phrases to describe what you are trying to say. Here are some examples:", "instances": [{"input": "", "output": "The trees were silhouetted against the sky. The wind was blowing. The leaves were rustling. The grass was waving. The flowers were nodding. The birds were singing. The bees were buzzing. The squirrels were running. The rabbits were hopping. The deer were grazing. The wolves were howling. The moon was rising. The stars were shining."}]}
-{"id": "user_oriented_task_10", "motivation_app": "Google Search", "instruction": "Come up with some search queries on google about coding stuff.", "instances": [{"input": "", "output": "- binary search\n- pop in javascript\n- audio chunking algorithm"}]}
-{"id": "user_oriented_task_11", "motivation_app": "StackOverflow", "instruction": "Discover the most relevant StackOverflow questions for your search term. And output a list of their URLs.", "instances": [{"input": "remove item in array", "output": "1- https://stackoverflow.com/questions/5767325/how-can-i-remove-a-specific-item-from-an-array\n2- https://stackoverflow.com/questions/19544452/remove-last-item-from-array\n3- https://stackoverflow.com/questions/15292278/how-do-i-remove-an-array-item-in-typescript\n4- https://stackoverflow.com/questions/3954438/how-to-remove-item-from-array-by-value\n5- https://stackoverflow.com/questions/40462369/remove-item-from-stored-array-in-angular-2\n6- https://stackoverflow.com/questions/16994212/remove-an-item-from-array-using-underscorejs"}]}
-{"id": "user_oriented_task_12", "motivation_app": "GeeksforGeeks", "instruction": "You are given an algorithm or a search query about coding. Look for a code snippet related to the search query.", "instances": [{"input": "Python loop", "output": "#Print each fruit in a fruit list:\nfruits = [\"apple\", \"banana\", \"cherry\"]\nfor x in fruits:\n   print(x)"}]}
-{"id": "user_oriented_task_13", "motivation_app": "GeeksforGeeks", "instruction": "You are provided with a description of the code, specifications, or prompts. Your help is needed in automating code generation to speed up the development process.", "instances": [{"input": "python function to determine whether a number is prime.", "output": "def is_prime(n):\n      if n == 1:\n           return False\n      for i in range(2, int(n**0.5)+1):\n           if n % i == 0:\n                return False\n      return True"}]}
-{"id": "user_oriented_task_14", "motivation_app": "Google Search", "instruction": "When you're stuck on a coding problem, the best place to look for help is in the documentation. Read the docs carefully and see if there's a section that covers your specific issue. Often, the answer will be right there in front of you. If not, try searching for key terms related to your problem. Come up with the useful links about the given search term.", "instances": [{"input": "sql find duplicate values", "output": "- https://iteration-utilities.readthedocs.io/en/latest/generated/duplicates.html\n- https://orange3-text.readthedocs.io/en/latest/widgets/duplicatedetection.html\n- https://beets.readthedocs.io/en/v1.3.14/plugins/duplicates.html\n- https://doma.readthedocs.io/en/latest/sql/\n- https://koalas.readthedocs.io/en/latest/reference/api/databricks.koalas.DataFrame.drop_duplicates.html\n- https://sqlparse.readthedocs.io/en/latest/analyzing/"}]}
-{"id": "user_oriented_task_15", "motivation_app": "MS Excel", "instruction": "We have entered the home supplies budget in the following table. Calculate the last column with the given function and rewrite the completed table.\nNote that, columns in a row are separated using a comma, and rows are separated using a \"|\".", "instances": [{"input": "Item, Qty, Price, Subtotal | mango, 5, 500, ? | apple, 4, 150, ? | oil, 2, 1500, ?  \nFunction: column2 * column 3", "output": "Item, Qty, Price, Subtotal | mango, 5, 500, 2500 | apple, 4, 150, 600 | oil, 2, 1500, 3000"}]}
-{"id": "user_oriented_task_16", "motivation_app": "MS Excel", "instruction": "Please write the Excel function name associated with each description.", "instances": [{"input": "- Returns the number of days between two dates\n- Returns the starting position of a text string within another text string.\n- Returns the number in the middle of the set of given numbers", "output": "- DAYS\n- FIND\n- MEDIAN"}]}
-{"id": "user_oriented_task_17", "motivation_app": "MS Word", "instruction": "What are the steps to using random (yet coherent) text as filler content within your Word document?", "instances": [{"input": "", "output": "You can quickly add several paragraphs to your document by using =rand(). These steps will guide you through using it:\nPut your Microsoft Word cursor (not your mouse cursor) where you would like random text to appear. It is necessary to place the cursor at the beginning of a line. The cursor just needs to be at the beginning, not on a blank line.\n2- Type =rand()and press Enter. You will now see random text added to your document."}]}
-{"id": "user_oriented_task_18", "motivation_app": "MS Powerpoint", "instruction": "Design a soothing pastel color palette for your slides. Pastel colors generally come across as pretty and delicate, so you\u2019ll want to make sure your presentation calls for a similar mood. Choose up to five colors or color codes.", "instances": [{"input": "", "output": "Color codes: #FDB3AE  #CAE4E2  #FBDF74"}]}
-{"id": "user_oriented_task_19", "motivation_app": "MS Excel", "instruction": "I bought two shirts from the store and each one cost me $10. I also bought a pair of pants and they cost me $20. Then, I bought a dress and it cost me $30. Also,  I bought a pair of shoes and they cost me $40. Lastly, I bought a jacket and it cost me $50.\n\nMake a table chart showing items, the number, and the price of these items. Separate columns with \"|\".", "instances": [{"input": "", "output": "Item | Number | Price\n\nShirts | 2 | $10\nPants | 1 | $20\nDress | 1 | $30\nShoes | 1 | $40\nJacket | 1 | $50"}]}
-{"id": "user_oriented_task_20", "motivation_app": "ludwig.guru", "instruction": "Find sentences from reliable sources such as the Guardian or Forbes that contain the exact match for the given sentence or phrase. Also, mention the source of each sentence.", "instances": [{"input": "There are great options", "output": "- Stella McCartney's faux fur has fashionable, luxury and ethical credentials and there are great options at Dries Van Noten too. (Independent)\n- And in the capital, Funchal, there are great options for self-caterers; I stayed in a beach house in the old town, one of a collection of seafront villas (funchalcottages.co.uk). (The Guardian - Travel)\n- City by city around the world, there are great options for shoppers to bring home not just special goods, but special memories. (Forbes)\n- Yes, there are great options out there that aren't owned by any big company (yet), but Twitter has an opportunity here to act as an app Switzerland, and attract some ad business while it's at it. (TechCrunch)"}]}
-{"id": "user_oriented_task_21", "motivation_app": "ludwig.guru", "instruction": "During writing, we added an asterisk for the word that did not come to mind. You will need to provide several examples to demonstrate all the words that can be used in the sentence instead of the asterisk.", "instances": [{"input": "we * to know", "output": "1- We need to know.\n2- We want to know.\n3- we wanted to know.\n4- We deserve to know why.\n5- We ought to know."}]}
-{"id": "user_oriented_task_22", "motivation_app": "ludwig.guru", "instruction": "Think of alternatives and paraphrases for the underlined word.", "instances": [{"input": "what we have _expected", "output": "+ what we have projected \n+ what we have anticipated\n+ what we have waited\n+ what we have hoped for\n+ what we have accustomed"}]}
-{"id": "user_oriented_task_23", "motivation_app": "LinkedIn", "instruction": "Design a skill assessment\u00a0questioner for R (Programming Language).", "instances": [{"input": "", "output": "Q1. How does a matrix differ from a data frame?\nQ2. What value does this statement return?\nunclass(as.Date(\"1971-01-01\"))\nQ3. What do you use to take an object such as a data frame out of the workspace?\nQ4. What function joins two or more column vectors to form a data frame?\nQ5. What is the value of y in this code?\nx <- NA\ny <- x/1"}]}
-{"id": "user_oriented_task_24", "motivation_app": "LinkedIn", "instruction": "Write a template for First-Person LinkedIn profile summary.", "instances": [{"input": "", "output": "[Degree] in [Area of Study] from [School], I entered the [Industry] world to explore my passion for ____, ____, and ____. Currently, I work as a [Job Title] for [Company], where I'm focused on ____, ____, and ____."}]}
-{"id": "user_oriented_task_25", "motivation_app": "LinkedIn", "instruction": "Write a LinkedIn post to announce that you have accepted a new job offer.", "instances": [{"input": "", "output": "\u201cI\u2019m excited beyond words to share with you my decision to accept the role of Marketing Director at the XYZ Company!\nI couldn\u2019t have had this opportunity if not for the help of my friend, [name], who shared this job opportunity with me, and my former boss, [boss\u2019s name], for her stellar recommendation and guidance.\nI happily look forward to starting this new journey and growing professionally with my new family\u2014the marketing rock stars of XYZ Company.\u201d"}]}
-{"id": "user_oriented_task_26", "motivation_app": "LinkedIn", "instruction": "Make a list of desirable Skills for software engineers to add to LinkedIn.", "instances": [{"input": "", "output": "Test-Driven Development\nObject-Oriented Programming (OOP)\nMVC\nREST API\nTechnical Writing\nDebugging"}]}
-{"id": "user_oriented_task_27", "motivation_app": "LinkedIn", "instruction": "You will be asked to answer the following question as part of the LinkedIn Skill assessments.", "instances": [{"input": "You find that your project has a tag and branch both named push-notifications, which causes confusion when trying to print out given reference. How can you specify which branch you want to look at?", "output": "use git show refs/head/push-notifications"}]}
-{"id": "user_oriented_task_28", "motivation_app": "Amazon", "instruction": "Take the title of the product and extract its attributes. The attributes in this case refer to the characteristics of the products, such as their brand and color. Your answer must a list following the format of \"attribute: value\".", "instances": [{"input": "SAMSUNG 980 PRO SSD 2TB PCIe NVMe Gen 4 Gaming M.2 Internal Solid State Hard Drive Memory Card, Maximum Speed, Thermal Control, MZ-V8P2T0B", "output": "Brand: Samsung, Style: 980 PRO, Capacity: 2TB, Hard Disk Interface: NVMe, Hard Disk Description: Solid State Hard Drive"}]}
-{"id": "user_oriented_task_29", "motivation_app": "Amazon", "instruction": "You should write a Product Description that will persuade the buyer that the product is beneficial.", "instances": [{"input": "Flipslide Game, Electronic Handheld Game | Flip, Slide, and Match the Colors to Beat the Clock - 4 Game Modes - Multiplayer Fun", "output": "Flipside is an addictive, multi-player puzzle game of skill! Flip, slide & match the colors to beat the clock. Four thrilling game modes including, Speed Mode, Multiplayer Mode, Level Mode and Memory mode make challenging yourself or playing with friends a fun distraction for everyone! Play at home, on the road, or wherever you'd like! Recommended for ages 8+."}]}
-{"id": "user_oriented_task_30", "motivation_app": "Amazon", "instruction": "What other Amazon products might interest someone who visited the given product?", "instances": [{"input": "Zeroll Zerolon Hardcoat Anodized Commercial Ice Cream Scoop with Unique Liquid Filled Heat Conductive Handle Easy Release Made in USA, 1.5-Ounce, Black", "output": "- Ice Scoop, Fashion Ice Cream Scoop, Premium Stainless Steel Cookie Scoop, Dog Food Scoop, Sturdy Flour Scoop, Utility Candy Scoop, Dishwasher Safe (Silver/8oz/9 Inch)\n- AmazonCommercial Non-Stick Heat Resistant Silicone Spatula Set, 2 Small & 2 Large Spatulas, Multicolor, Pack of 4\n- Winco Acrylic 4-Hole Ice Cream Cone Stand,Clear,Medium\n- Zeroll 1065FS-ZT Original Zelato Tubmate Aluminum Gelato Spade for Leveling Tubs Packing and Hand-Mixing, Self-Defrosting Fluid-Filled Handle Longer-Lasting Thicker Blade , 9-Inch, Gray"}]}
-{"id": "user_oriented_task_31", "motivation_app": "Amazon", "instruction": "Compare the given item with similar products based on its properties such as price, rating, etc. For product comparison, use a table and separate the columns with \"|\".", "instances": [{"input": "Item: iBayam Journal Planner Pens Colored Pens Fine Point Markers Fine Tip Drawing Pens Porous Fineliner Pen for Journaling Writing Note Taking Calendar Coloring Art Office Back to School Supplies, 18 Color", "output": "Product | this item |\u00a0Fineliner Color Pens Set, 0.38mm Fine Tip Pens, Porous Fine Point Makers Drawing Pen, Perfect for Writing in Bullet Journal and Planner, 24 Assorted Colors |\u00a0iBayam Fineliner Pens, 24 Bright Colors Fine Point Pens Colored Pens for Journaling Note Taking Writing Drawing Coloring Planner Calendar, Office School Teacher Classroom Fine Tip Marker Pens Supplies |\u00a024 colors paint pens for journaling, Colored pens for note taking 0.4mm fineliner cute fine point smooth writing pens, Fast dry felt tip pens agenda office school teacher art markers drawing supplies\nCustomer Rating | 4.7 out of 5 stars (87557) |\u00a0 4.5 out of 5 stars (2099) | 4.7 out of 5 stars (8666) | 4.5 out of 5 stars (396) \nPrice | $6.39 | $7.66 | $8.99 | $10.99\nSold By | Qi US | ai-natebok | Qi US | MingNor\nColor | Black, Grey, Red, Blue, Magenta, Pink, Purple, Violet, Pale Yellow, Yellow, Orange, Raw Sienna, Sap Green, C Green, O Green, Lake Blue, Burnt Sienna, Crimson | 24 Color | 24 Colors | 24 colors\nItem Dimensions | 6.6 x 6.2 x 0.6 inches | 6.3 x 5.8 x 0.6 inches | 6.6 x 6.2 x 0.6 inches | 6.22 x 0.31 x 0.31 inches\nPoint Type | Fine | Extra Fine | Fine | Fine"}]}
-{"id": "user_oriented_task_32", "motivation_app": "Amazon", "instruction": "Write a  review based on the given information.", "instances": [{"input": "- Product: Persil Discs Laundry Detergent Pacs, Original Scent, High Efficiency (HE) Compatible, Laundry Soap, 62 Count\n- Sentiment: Positive", "output": "I love this brand of soap it's always gotten my kids' clothes clean. When I'm done washing with the soap my kids is closed look feel and smell and feel clean and I like that that's what I want is to be able to wash my kids' clothes and not stress that it didn't come out clean or it's still kind of stinks or it just didn't seem like it washed very well. And would this pod of soap it's really easy to use I just put two pods in the wash and if it's a big wash and my kids' clothes are really dirty I just use three and it gets my kids's clothes washed the first time and I really like that. I think it's definitely worth the money. I am glad that I purchased it. It works so well. I love using it. It's a really good buy for me and for getting my family's clothes clean especially my kids' clothes clean. Great buy!"}]}
-{"id": "user_oriented_task_33", "motivation_app": "Netflix", "instruction": "Write a short description about the given movie or series.", "instances": [{"input": "The Witcher (2019)", "output": "Geralt of Rivia, a mutated monster-hunter for hire, journeys toward his destiny in a turbulent world where people often prove more wicked than beasts."}]}
-{"id": "user_oriented_task_34", "motivation_app": "Netflix", "instruction": "Think of some series or movies that would be enjoyable for someone who liked the given series.", "instances": [{"input": "Squid Game", "output": "- 3%\n- Clickbait\n- Circle\n- Hellbound\n- Sweet Home"}]}
-{"id": "user_oriented_task_35", "motivation_app": "Netflix", "instruction": "Give examples of popular shows and movies in the genre.", "instances": [{"input": "Genre: Crime TV Shows", "output": "Breaking Bad, Peaky Blinders, Arrow, Money Heist, Better Call Saul, DAHMER, The Blacklist, Lucifer, NCIS"}]}
-{"id": "user_oriented_task_36", "motivation_app": "Netflix", "instruction": "In what genres does the given movie or series fall?", "instances": [{"input": "Manifest", "output": "Mystery, Sci-Fi, Drama"}]}
-{"id": "user_oriented_task_37", "motivation_app": "Netflix", "instruction": "Give a brief description of the given category of movies and shows.", "instances": [{"input": "Period Dramas", "output": "Want to escape the contemporary world? Explore these historical dramas and shows from the time that have magnificent art and costume design, lots of drama, and a lot of history."}]}
-{"id": "user_oriented_task_38", "motivation_app": "Overleaf", "instruction": "Provide an example of how a table of contents can be generated automatically in a LATEX document. Make sure your example contains the necessary commands.", "instances": [{"input": "", "output": "To create the table of contents is straightforward, the command \\tableofcontents does the job. Sections, subsections and chapters are included in the table of contents. To manually add entries, for example when you want an unnumbered section, use the command \\addcontentsline as shown in the following example:\n\n\\documentclass{article}\n\\usepackage{blindtext}\n\\usepackage{titlesec}\n\\title{Sections and Chapters}\n\\author{Gubert Farnsworth}\n\\date{ }\n\\begin{document}\n\\maketitle\n\\tableofcontents\n\\section{Introduction}\nThis is the first section.\n\\blindtext\n\\addcontentsline{toc}{section}{Unnumbered Section}\n\\section*{Unnumbered Section}\n\\blindtext\n\\section{Second Section}\n\\blindtext\n\\end{document}"}]}
-{"id": "user_oriented_task_39", "motivation_app": "Overleaf", "instruction": "Imagine you have a FEN notation of a chessboard. How can you draw a board to show this state of the chessboard in LATEX? Create a LATEX code that includes all the required commands and packages.", "instances": [{"input": "", "output": "The following example uses the setfen key in a \\chessboard command to demonstrate a FEN example taken from the skak package documentation:\n\n\\documentclass{article}\n\\usepackage{xskak}\n\\begin{document}\n\\newchessgame\n\\chessboard[setfen=r5k1/1b1p1ppp/p7/1p1Q4/2p1r3/PP4Pq/BBP2b1P/R4R1K w - - 0 20]\n\\end{document}"}]}
-{"id": "user_oriented_task_40", "motivation_app": "Overleaf", "instruction": "What is the output of the given LATEX code?", "instances": [{"input": "\\documentclass{article}\n\\begin{document}\nFirst document. This is a simple example, with no \nextra parameters or packages included.\n\\end{document}", "output": "First document. This is a simple example, with no extra parameters or packages included."}]}
-{"id": "user_oriented_task_41", "motivation_app": "Overleaf", "instruction": "You should find a few LATEX templates appropriate for the task at hand and list the links to them.", "instances": [{"input": "Homework Assignment", "output": "- https://www.overleaf.com/latex/templates/gijss-homework-template/xrhhfgqcfbft\n- https://www.overleaf.com/latex/templates/homework-solutions/xykkzgnbzmxp\n- https://www.overleaf.com/latex/templates/ocen-cfd-article-template/mtpnnssgzjzw\n- https://www.overleaf.com/latex/templates/pitt-state-physics-homework-template/wdsxknmntnxk"}]}
-{"id": "user_oriented_task_42", "motivation_app": "Overleaf", "instruction": "Write a simple guide for uploading the bibliography database\n on Overleaf.", "instances": [{"input": "", "output": "If you already have a bibliography file that you need to use in an Overleaf project, you can upload it. Click the Upload icon located on top of the left panel. A dialog box will appear, you can either drag and drop the .bib file(s) or choose select from your computer to upload file(s) from your device. Once the upload process is finished you can include the .bib file in your main .tex document."}]}
-{"id": "user_oriented_task_43", "motivation_app": "Spotify", "instruction": "Choose Spotify playlists that match the given mood. Provide a link to these playlists for easy access.", "instances": [{"input": "Feeling Good", "output": "- https://open.spotify.com/playlist/37i9dQZF1DX9XIFQuFvzM4\n- https://open.spotify.com/playlist/37i9dQZF1DXcEKFjZJYZcc"}]}
-{"id": "user_oriented_task_44", "motivation_app": "Spotify", "instruction": "Curate a Spotify playlist based on the vibe. Publish this playlist as a song list.", "instances": [{"input": "Vibe: coding on weekend", "output": "- Candles by Jon Hopkins\n- Train from Paris by Mindtrap\n- Intro by The xx\n- Take Me Back by WYS\n- Riverline by Tor\n- The Dream by SwuM"}]}
-{"id": "user_oriented_task_45", "motivation_app": "Spotify", "instruction": "How would you describe the meaning of this lyrics? Describe some of the metaphors.", "instances": [{"input": "My lover\u2019s got humor\nShe\u2019s the giggle at a funeral\nKnows everybody\u2019s disapproval\nI should\u2019ve worshipped her sooner", "output": "Instead of using a straight-forward line like, \u201cshe is funny,\u201d the writer opts for the metaphoric version, \u201cshe\u2019s the giggle at a funeral.\u201d This is a powerful metaphor stating that the writer's lover is so full of humor that even in really sad moments, she finds a way to lighten the moment."}]}
-{"id": "user_oriented_task_46", "motivation_app": "Spotify", "instruction": "Suggest some Audiobooks for First-Time Listeners. You can attract them by choosing popular works.", "instances": [{"input": "", "output": "+ Harry Potter and the Sorcerer's Stone By J.K. Rowling\n+ The Vanishing Half by Brit Bennett\n+ The Fellowship of the Ring by J.R.R. Tolkien\n+ And Then There Were None by Agatha Christie.\n+ The Woman in Cabin 10 by Ruth Ware.\n+ The Hitchhiker's Guide to the Galaxy by Douglas Adams.\n+ Spinning Silver by Naomi Novik.\n+ Good Omens by Terry Pratchett and Neil Gaiman.\n+ Charlotte's Web by E. B."}]}
-{"id": "user_oriented_task_47", "motivation_app": "Spotify", "instruction": "Make a list of the most popular podcasts.", "instances": [{"input": "", "output": "+ Top Podcasts\n+ The Joe Rogan Experience\n+ Distractible\n+ Call Her Daddy\n+ Huberman Lab\n+ Anything Goes with Emma Chamberlain\n+ Stuff You Should Know\n+ The Psychology of your 20\u2019s\n+ On Purpose with Jay Shetty\n+ Armchair Expert\n+ Archetypes\n+ Rachel Maddow Presents: Ultra\n+ The Mel Robbins Podcast\n+ Lex Fridman Podcast\n+ The Daily\n+ Think Fast, Talk Smart: Communication Techniques\n+ TED Talks Daily"}]}
-{"id": "user_oriented_task_48", "motivation_app": "Reddit", "instruction": "You should take the time to read the situation and problem carefully and give the best advice you can.", "instances": [{"input": "I'm roadtripping down New Zealand with a friend and he is driving me insane. He hasn't enjoyed all the tours we've been on and doesn't want to chat about ANYTHING. We're a week into our trip and this is the quietest I've ever been when hanging out with another person as he rebuffs all my attempts at conversation. He wears his airpods in the car and around the accommodation - not that I've kept trying to talk to him. Honestly, he is bringing some seriously bad vibes to everything and it's ruining my trip.\nWe don't have anything booked in yet for the South Island, so I'm planning to ditch him in Wellington at the ferry. Probably book my own car in Picton and just relax alone for the rest of the trip. (Just thinking about it is such sharp relief that it makes me feel emotional. I'll pay whatever price I have to just to get away from him.)\nObviously telling him he is ruining my trip with his negative energy isn't the best way to approach this.\nAny advice for the best approach so we can split amicably? Or at least not have an argument that ends with very hurt feelings?", "output": "I would be as honest as you can while being nice. Next time you\u2019re both in the hotel or wtv just say \u201chey man, I was thinking about when we get to X\u2014it seems like we like to travel really differently and I think it\u2019s stressing us both out, so how do you feel about splitting up after that stop?\u201d"}]}
-{"id": "user_oriented_task_49", "motivation_app": "CNN News", "instruction": "You are given a description that provides a set of facts or a scenario. It is up to you to craft a story from these facts and scenarios. The missing pieces must be filled in with imaginative but logical information.", "instances": [{"input": "Ten European football teams \u2013 the Netherlands, England, Belgium, Denmark, France, Germany, Norway, Sweden, Switzerland and Wales \u2013 will participate in a season-long \u201cOneLove\u201d campaign promoting inclusion and opposing discrimination.", "output": "Every country except Sweden and Norway has qualified for the 2022 World Cup in Qatar and each captain of these eight nations will wear a distinctive OneLove armband \u2013 which features a heart containing colors from all backgrounds \u2013 during the tournament.\nThe Netherlands FA, which is spearheading the campaign, chose the colors to represent all heritages, backgrounds, genders and sexual identities; the armband will be worn in Qatar where same-sex relationships are a criminal offense.\nSweden and Norway will participate in the initiative during the upcoming Nations League matches, while England will also wear black armbands during both its UEFA Nations League matches to mark the death of Queen Elizabeth II.\n\u201cThis is an important message which suits the game of football: on the field everybody is equal and this should be the case in every place in society. With the OneLove band we express this message,\u201d said Virgil van Dijk, the Netherlands captain.\n\u201cOn behalf of the Dutch team I have been wearing this band for quite a while now. It is good to see that other countries are joining this initiative.\u201d\nOneLove was founded in the Netherlands in 2020 to emphasize that all football fans have at least one thing in common \u2013 their love of football \u2013 and to speak out against any form of discrimination.\nAs well as focusing on public messaging, the initiative has also developed to offer diversity training to grassroots club.\n\u201cOur love of football unites us all. No matter where you come from, what you look like and who you love. Football is there for everyone and our sport must stand up for the people across the world who face discrimination and exclusion,\u201d said Germany captain Manuel Neuer.\n\u201cI am proud to be sending out this message with my colleagues from the other national teams. Every single voice counts.\u201d\nIn June, England\u2019s captain Harry Kane revealed that he had discussed taking a collective stance regarding human rights in Qatar with Denmark\u2019s Christian Eriksen and France captain Hugo Lloris.\n\u201cI am honored to join my fellow national team captains in supporting the important OneLove campaign,\u201d he said on Tuesday.\n\u201cAs captains we may all be competing against each other on the pitch, but we stand together against all forms of discrimination. This is even more relevant at a time when division is common in society. Wearing the armband together on behalf of our teams will send a clear message when the world is watching.\u201d"}]}
-{"id": "user_oriented_task_50", "motivation_app": "Google Search", "instruction": "Based on the given query, suggest some related search queries.", "instances": [{"input": "learning french", "output": "learning french for beginners\nearning french online\nlearning french free\nfrench language course\nlearning french review\nhow to learn french on your own\nbest way to learn french online"}]}
-{"id": "user_oriented_task_51", "motivation_app": "Google Scholar", "instruction": "Please suggest a few papers to consider based on the search term given. The names of the papers should be listed.", "instances": [{"input": "Topic: scaling law + machine learning", "output": "- Learning to limit data collection via scaling laws: A computational interpretation for the legal principle of data minimization\n- Data and parameter scaling laws for neural machine translation\n- Explaining neural scaling laws\n- Scaling laws for autoregressive generative modeling\n- Scaling laws for neural language models"}]}
-{"id": "user_oriented_task_52", "motivation_app": "Gmail", "instruction": "Prepare an email signature template for the given person or company.", "instances": [{"input": "a graduate student of computer science", "output": "[Your name]\n[Your degree] in Computer Science\n[Your university]\n\nWebsite: [Your website]\nORCID ID: [Your ORCID ID]"}]}
-{"id": "user_oriented_task_53", "motivation_app": "yelp", "instruction": "React properly to reviews from your customers. In your responses, you can highlight your business' policies and customer service.", "instances": [{"input": "Another repeat visit and no disappointment from us. Great place for breakfast or brunch - and you won't need to eat again for awhile. The restaurant is close to the Tenderloin so be aware of your surroundings.\nWe enjoyed the Succotash Omelet (at least my wife enjoyed it), the Pulled Pork Benedict and the Hangtown Fry. Portions are large and pricing is fair. I wish we could have managed a Beignet Flight!\nDefinitely a spot to seek out. It can be crowed at times so be patient. Well worth any wait.", "output": "Hi [customer's name], thank you so much for the wonderful feedback. We\u2019re happy to hear that you enjoyed your experience with us and that you love [restauran't name] so much! We\u2019re beyond thankful for your support and can\u2019t wait to see you back again soon! Thanks!"}]}
-{"id": "user_oriented_task_54", "motivation_app": "Twitter", "instruction": "Write a good Twitter Bio. Try to cover all the provided main points in your generated bio.", "instances": [{"input": "Name: Jane\nMain points:\n- Architect\n- Interested in Tech\nWritten in first person\nUse casual tone", "output": "I like to think of myself as a kind of architect. I'm interested in the tech side of things, and I love coming up with new ideas for how to make things better."}]}
-{"id": "user_oriented_task_55", "motivation_app": "Twitter", "instruction": "We need to write a thematic tweet to share company news and facts. Please take a careful look at the facts and details provided and create a tweet based on them.", "instances": [{"input": "- news: Applications for summer internships are open now\n- Start with a thematic tie-in to Halloween", "output": "Halloween is right around the corner, and we're celebrating with some exciting news: applications are now open for our summer internships!\nIf you want to work with us this summer, we'd love to hear from you. We're looking for people who are passionate about [company name], and committed to helping us grow. If that sounds like you, check out all the details on our website: [website address]."}]}
-{"id": "user_oriented_task_56", "motivation_app": "Indeed", "instruction": "A job description is a document that clearly states the requirements, duties, responsibilities, and skills required to perform a specific role. You need to write a job description based on the given information. You should describe responsibilities and duties in your job description in a logical order. Remember that potential employees often get their first impression of your company from your job description. So, the job description should convey the attractiveness of your company.", "instances": [{"input": "Company: [Company name]\nJob: Software Engineer\nJob Responsibilities:\nReceive and perform code reviews with other engineers.\nWrite unit, integration, and end-to-end tests to verify functionality using automated testing frameworks such as Pytest.\nWork collaboratively with fellow software engineers to build features requested by business stakeholders.\nParticipate in Agile teams to develop, test, and debug complex data processing pipelines and data analysis applications using big data processing systems such as Apache Spark.\nDiagnose, debug, and perform root cause analysis of issues and defects.\nSkills and experience that will lead to success:\nExperience with a modern software programming language. We use Python, and Pyspark extensively.\nExperience with cloud systems like AWS or Azure\nExperience writing SQL queries \nStrong written and verbal communication skills\nEagerness to work in a highly-collaborative environment\nSome visualization experience such as Tableau or PowerBI is helpful", "output": "Software Engineer\n[company name] is looking for a self-motivated software engineer to join our team. We're a team of engineers and data scientists who work with large-scale data processing systems such as Apache Spark and Amazon Redshift to analyze data in the cloud.\nYou'll be responsible for receiving and performing code reviews with other engineers, writing unit, integration, and end-to-end tests to verify functionality using automated testing frameworks such as Pytest, working collaboratively with fellow engineers to build features requested by business stakeholders, participating in Agile teams to develop, test and debug complex data processing pipelines and data analysis applications using big data processing systems such as Apache Spark.\nIn addition to being able to diagnose, debug, and perform root cause analysis of issues and defects when needed, we're looking for someone with some visualization experience (Tableau or PowerBI) who can bring their own ideas about how best use these tools on a project.\nSkills that will lead to success include: Experience with Python programming languages; experience with cloud systems like AWS or Azure; experience writing SQL queries; strong written communication skills; eagerness to work in a highly collaborative environment;"}]}
-{"id": "user_oriented_task_57", "motivation_app": "Gmail", "instruction": "Considering the reason for the rejection, write a polite rejection letter for rejecting a candidate.", "instances": [{"input": "Reason: You came in second", "output": "Dear [Candidate Name],\n\nThank you for applying to [company name]! We're sorry to inform you that we're going to be passing on your application.\nWe were impressed with your initiative and would love to have had the opportunity to work with you. At this time, however, we're not able to offer any positions at our company.\nPlease accept our sincerest apologies for passing on your application, and know that we remain open to working with you in the future!"}]}
-{"id": "user_oriented_task_58", "motivation_app": "Blogger", "instruction": "We've already decided on the title and topic for our blog post. I would appreciate it if you wrote an introduction paragraph to attract the readers' attention.", "instances": [{"input": "Blog title: Healthy Apple Muffins\nBlog Topic: recipe and tips for cooking a good apple muffin", "output": "Let me tell you this, apple muffins are extremely easy to make, and they are totally worth the effort. If you've never been much of a cook or baker before, I can promise you that making these will change your mind. And if they don't get the job done, at least they'll taste good!"}]}
-{"id": "user_oriented_task_59", "motivation_app": "Blogger", "instruction": "Design an outline for a blog post based on the given information and list the sections accordingly.", "instances": [{"input": "This blog post is going to about making a list of last-minute gift ideas. Also, the tone of the post is going to be relaxed and casual.", "output": "Section: Gift card to a favorite restaurant or store\nSection: Movie tickets\nSection: A massage or mani/pedi\nSection: A fun game console\nSection: Books\nSection: Candles, flameless candles\nSection: Portable battery charger for phones/tablets\nSection: Water filter pitcher\nSection: Personalized items with photos/artwork\nSection: Favorite team gear (hat, sweatshirt)\nTakeaway: Don't stress about out running out of time to buy, make a gift."}]}
-{"id": "user_oriented_task_60", "motivation_app": "Blogger", "instruction": "Write a section for a blog post and try to cover all of the provided information about this section in your text.", "instances": [{"input": "Blog Topic: 7 Fall Fashion Trends Worth Taking on Your Trip\nSection Title: Trusty Jeans\nMain point of the section: jeans can be worn all year and look good with everything .", "output": "Jeans are one of the most iconic fashion items for a reason \u2013 they look great with everything and can be worn year-round. While this Fall, the trends lean toward a baggier fit, bringing along your favorite, trusty jeans is a must no matter where you\u2019re headed. If you can\u2019t get on board with the baggy jean trend, at least opt for a straight leg and ditch the skinnies."}]}
-{"id": "user_oriented_task_61", "motivation_app": "Instagram", "instruction": "Create a detailed caption for an Instagram post about a discount. A proper caption should explain the product, offer the discount, and tell the user how to claim it.", "instances": [{"input": "Product: Golang course\nProduct description: teaching the foundational skills for building cloud services, web applications, DevOps tools, and backend development\nDiscount: 50%\nHow to claim: Use the coupon code: GOLANG", "output": "We've just launched the Golang course. In this course you will learn the foundational skills needed for high-paying careers involving building cloud services, web applications, DevOps tools, and backend development. Upon completion of the course, you will be able to build a web server that end-users can use to interact and consume as a REST API.\nLearning outcomes\nAs a graduate you\u2019ll be able to:\n1- Articulate the advantages and disadvantages of Go compared to similar languages.\n2- Design a business case for when to develop a project in Go.\n3- Set up a Go environment.\n4- Program a basic backend using Go.\nCheck out the Golang course syllabus to view the course project.\n\nEnroll this week and save 50% with our new course special. Use the coupon code: GOLANG at checkout to save 50% on your subscription to the Golang course. Start learning soon\u2014this special promotion is available only until [date]."}]}
-{"id": "user_oriented_task_62", "motivation_app": "Instagram", "instruction": "Create a captivating Instagram caption based on the provided details. Try to think of as many captions as you can.", "instances": [{"input": "I am sharing images of the first snow of the year taken from a window in an office", "output": "- It's snowing! Here's the first snow of the year from our office windows.\n- Let the photos do the talking. The first snow of the year is just around the corner.\n- I know it's not snowing outside, but just seeing the first flakes of the year from our office window is enough to make me feel like winter has finally arrived. :D\n- First snow of the year and I'm in a good mood. Because I get to do my favorite thing: Look at pictures of snow on Instagram\n- The first snow of the year and I'm getting cozy in my office chair. \ud83d\ude0d\n- Snowpocalypse: the sweet, delicate joy of first snowfall.\n- Snow day: here's to a great start to the week!\n- Snowing in my office and I am not even mad.\n- It's a snowy day in the office. I'm wearing my long johns, drinking hot chocolate, and binge-watching Game of Thrones. What are you doing?"}]}
-{"id": "user_oriented_task_63", "motivation_app": "Instagram", "instruction": "You are given a topic for an Instagram post. Help the post reach a broader audience by suggesting hashtags related to the post.", "instances": [{"input": "Another episode of women in science is out now", "output": "#womeninSTEM #WomenScience #science #research #stempodcast"}]}
-{"id": "user_oriented_task_64", "motivation_app": "Instagram", "instruction": "Use appropriate emojis to convey the meaning in the given text.", "instances": [{"input": "Cute cat\nplayful\nSoft", "output": "\ud83d\udc31 Cute cat\n\ud83d\ude1b playful\n\ud83d\udc9b Soft"}]}
-{"id": "user_oriented_task_65", "motivation_app": "YouTube", "instruction": "The topic of a YouTube post has been described and based on the information, you need to write a hook for starting the post. A catchy hook will keep your readers interested so they keep reading. It would be great if you could come up with as many hooks as you can.", "instances": [{"input": "A video showing how to make a tasty cup of coffee", "output": "- How do you make a great cup of coffee? We asked a professional.\n- Real life stories of the greatest coffee shops in the world.\n- You'll definitely want a cup of this after watching it so be sure not to miss this.\n- Coffee is something most people drink every day. This video tells you everything about it.\n- Watch this video on 7 habits to make you a better barista."}]}
-{"id": "user_oriented_task_66", "motivation_app": "YouTube", "instruction": "Using the provided topic as a starting point, brainstorm ideas for videos that can be made about it on YouTube.", "instances": [{"input": "In spite of the fact that procrastination feels bad to you, why do you do it?", "output": "+ The reason why you procrastinate\n+ A documentary about procrastination\n+ How to quit procrastination\n+ Recording people talking about their procrastination habits\n+ Interview people while they are procrastinating\n+ People who have failed to accomplish their New Year's resolutions explain why"}]}
-{"id": "user_oriented_task_67", "motivation_app": "YouTube", "instruction": "You should choose a YouTube video title based on the video's content. A video's title tells viewers what to expect from it. It should be direct, honest, and clear. The title of the video needs to capture the attention of viewers, so do not use an unclear or ambiguous one.", "instances": [{"input": "A research study has been conducted to determine if exercise really can \"boost\" your metabolism.", "output": "You Will Not Believe The Results Of A Recent Study On Exercise And The Metabolism!"}]}
-{"id": "user_oriented_task_68", "motivation_app": "YouTube", "instruction": "Choosing a name for your product or business YouTube channel is an important part of the process. Based on the description of the product or business, you should come up with some interesting names. Take some time to brainstorm your ideas.", "instances": [{"input": "Here you will find videos and content that will help students prepare for the application process to graduate schools as well as how to apply to graduate schools", "output": "GradSchoolPrep, GoGradSchool, GradSchoolVidz, Apply2Gradschool"}]}
-{"id": "user_oriented_task_69", "motivation_app": "merriam-webster.com", "instruction": "Find the answer that best describes the underlined SAT word. Select the correct option and explain the meaning of the underlined word.", "instances": [{"input": "Despite the _cacophony, the student tried to study. \nA. Loud sounds\nB. Difficult subject\nC. Late hour\nD. Low lighting", "output": "A. Loud sounds, Although it comes from the Greek for \u201cbad sound,\u201d cacophony can now also mean \u201can incongruous or chaotic mixture\u201d of other things, like colors or smells."}]}
-{"id": "user_oriented_task_70", "motivation_app": "merriam-webster.com", "instruction": "Find synonyms for the given word. You need to write down how the provided synonyms differ from the original word in terms of meaning, usage, etc.", "instances": [{"input": "adversity", "output": "- mischance:The words mischance and adversity are synonyms, but do differ in nuance. Specifically, mischance applies especially to a situation involving no more than slight inconvenience or minor annoyance.\n- misfortune: The meanings of misfortune and adversity largely overlap; however, misfortune may apply to either the incident or conjunction of events that is the cause of an unhappy change of fortune or to the ensuing state of distress.\n- mishap: The synonyms mishap and adversity are sometimes interchangeable, but mishap applies to an often trivial instance of bad luck."}]}
-{"id": "user_oriented_task_71", "motivation_app": "merriam-webster.com", "instruction": "Discuss the origins and history of the word that has been given to you.", "instances": [{"input": "oblique", "output": "Middle English oblique, oblike, borrowed from Anglo-French oblic, oblique, borrowed from Latin obl\u012bquus \"slanting, transverse,\" from ob- \"against, facing\" + -l\u012bquus, of uncertain meaning and origin"}]}
-{"id": "user_oriented_task_72", "motivation_app": "Yelp", "instruction": "Give the provided brand a motto to use as a part of its marketing strategies.", "instances": [{"input": "The Ice cream shop is a small, quaint shop located in the heart of the historic district. They offer a variety of ice cream flavors, as well as Italian ice, gelato, and sorbet. The shop has a few tables and chairs, but most of their business is to-go. They offer a loyalty program where you can get a free scoop of ice cream after 10 purchases.", "output": "- It's a scoop of friendly!\n- Come in and have a treat!\n- For a sweet treat, stop by and see us."}]}
-{"id": "user_oriented_task_73", "motivation_app": "Gmail", "instruction": "A confirmation email should be written appropriately for the situation.", "instances": [{"input": "A meeting has been scheduled, and the sender expects the other to review the slides.", "output": "Hi [name],\n\nI hope you're having a great week so far. I wanted to follow up on our meeting and let you know that we've scheduled a time for you and your team to come in next week. I'll have some slides prepared for you to review, and if there are any questions, I'd be happy to answer them! Just email me back at [email] when you have a moment.\n\nThanks so much for all of your help with this project\u2014we're excited to get started!"}]}
-{"id": "user_oriented_task_74", "motivation_app": "Gmail", "instruction": "You need to write an email to negotiate your salary.", "instances": [{"input": "", "output": "Dear [hiring manager's name],\n\nI wanted to take a moment to thank you for the offer. I'm honored that you would consider me for this position, and I'm excited to be working with you.\nI know that it's not easy to get a sense of how much salary should be offered in a given situation, so I wanted to explain why I believe my skill set is worth more than the offered salary. \nMy experience as a [position title] will prepare me well for this role, which requires extensive project management skills, as well as strong written communication skills and a deep understanding of the company's product offerings. These are all areas where I've already built up substantial experience over the past few years and am confident that they would serve me well in your company. \n\nI hope we can reach an agreement soon!"}]}
-{"id": "user_oriented_task_75", "motivation_app": "Gmail", "instruction": "Write an email to attendees as a reminder that the event is coming up.", "instances": [{"input": "", "output": "Hello, [name]!\n\nWe're excited to announce the [event name] event that you signed up for. The location is [location], and the date and time is [date and time].\n\nIf you have any questions or concerns about the event, please let us know by emailing us at [email]."}]}
-{"id": "user_oriented_task_76", "motivation_app": "Github", "instruction": "It would be helpful if you could suggest an acronym for the repository.", "instances": [{"input": "A python framework for creating, editing, and invoking Noisy Intermediate Scale Quantum (NISQ) circuits.", "output": "Cirq"}]}
-{"id": "user_oriented_task_77", "motivation_app": "Github", "instruction": "Write a step-by-step guide for resolving GitHub merge conflicts.", "instances": [{"input": "", "output": "1- Under your repository name, click Pull requests.\n2- In the \"Pull Requests\" list, click the pull request with a merge conflict that you'd like to resolve.\n3- Near the bottom of your pull request, click Resolve conflicts.\n4- Decide if you want to keep only your branch's changes, keep only the other branch's changes, or make a brand new change, which may incorporate changes from both branches. Delete the conflict markers <<<<<<<, =======, >>>>>>> and make the changes you want in the final merge.\n5- If you have more than one merge conflict in your file, scroll down to the next set of conflict markers and repeat steps four and five to resolve your merge conflict.\n6- Once you've resolved all the conflicts in the file, click Mark as resolved.\n7- If you have more than one file with a conflict, select the next file you want to edit on the left side of the page under \"conflicting files\" and repeat steps four through seven until you've resolved all of your pull request's merge conflicts.\n8- Once you've resolved all your merge conflicts, click Commit merge. This merges the entire base branch into your head branch.\n9- If prompted, review the branch that you are committing to.\nIf the head branch is the default branch of the repository, you can choose either to update this branch with the changes you made to resolve the conflict, or to create a new branch and use this as the head branch of the pull request.\nIf you choose to create a new branch, enter a name for the branch.\nIf the head branch of your pull request is protected you must create a new branch. You won't get the option to update the protected branch.\nClick Create branch and update my pull request or I understand, continue updating BRANCH. The button text corresponds to the action you are performing.\n10- To merge your pull request, click Merge pull request. For more information about other pull request merge options, see \"Merging a pull request.\""}]}
-{"id": "user_oriented_task_78", "motivation_app": "Github", "instruction": "One or more jobs can be run automatically by a workflow. The workflows are defined in the repository as YAML files and will run if an event occurs in the repository, as well as manually, or on a certain schedule.\nBring an example of a workflow that demonstrate the CI/CD features of GitHub Actions.", "instances": [{"input": "", "output": "Using scripts to test your code on a runner is a common example. When this workflow is triggered, it automatically runs a script."}]}
-{"id": "user_oriented_task_79", "motivation_app": "IMDB", "instruction": "The movie tagline is the film's advertising slogan, usually printed on posters prior to the official release. It is a short description or comment on a movie to capture the essence of the movie, and ultimately make you watch the movie. Here is an example:", "instances": [{"input": "Movie: Joker (2019)", "output": "Put on a happy face."}]}
-{"id": "user_oriented_task_80", "motivation_app": "IMDB", "instruction": "You are given some reviews for a movie. Each video might have mentioned some negative or positive points about the movie. Read them carefully and extract the positive and negative points mentioned in all of the reviews. Then, make a list of these points and summarize them as positive points and negative points.", "instances": [{"input": "Review 1: I'm not sure if it's just the fact that Ed Norton really grates on me in this film, but I never really bought into the idea of Fight Club being some kind of cultural phenomenon. Yes it's an innovative story, told via a strong script, engaging characters and intense cinematography. But is it really worthy of such high praise? Personally, I find it starts to fall apart halfway through and actually becomes quite tedious towards the end. Everything just becomes a little bit childish and contrived as the story loses touch with its base elements and the rage against the machine vibe takes over. I have huge respect for the way this film was made but, much like The Matrix, I feel like people attach way more importance to this movie than it deserves.\nReview 2: The film tells the story of men who cannot adapt to the social order in today's world, under the leadership of Tyler Durden, to discover their natural instinct, masculinity, and to escape from their unhappy lives.\nReview 3: Despite a good theme, great acting and important messages that this movie convey in an unorthodox way, I think it fails to connect the audience with the storyline and leaves him in a world of confusion. Although, majority of reviews find this movie entertaining and interesting, yet I would choose to be a minority that believes that this movie is extremely overrated.\nReview 4: Is creating chaos, explosions and membership in a militant sect the way to get rid of material shackles ?! The visual effects are fascinating and the story is compelling until the second half. Unfortunately, it doesn't end well.", "output": "Positive points: \n- Innovative story\n- Strong script\n- Engaging characters\n- Intense cinematography\n- Good theme\n- Great acting\n- Important messages\n- Fascinating visual effects \n\nNegative points: \n- Ed Norton grates on the viewer\n- Starts to fall apart halfway through\n- Childish and contrived towards the end\n- Loses touch with its base elements\n- World of confusion\n- Overrated"}]}
-{"id": "user_oriented_task_81", "motivation_app": "IMDB", "instruction": "Give a brief summary of the movie or series' plot.", "instances": [{"input": "Breaking Bad (TV Series 2008\u20132013)", "output": "Walter H. White is a chemistry genius, but works as a chemistry teacher in an Albequerque, New Mexico high school. His life drastically changes when he's diagnosed with stage III terminal lung cancer, and given a short amount of time left to live: a mere matter of months. To ensure his handicapped son and his pregnant wife have a financial future, Walt uses his chemistry background to create and sell the world's finest crystal methamphetamine. To sell his signature \"blue meth,\" he teams up with Jesse Pinkman, a former student of his. The meth makes them very rich very quickly, but it attracts the attention of his DEA brother in law Hank. As Walt and Jesse's status in the drug world escalates, Walt becomes a dangerous criminal and Jesse becomes a hot-headed salesman. Hank is always hot on his tail, and it forces Walt to come up with new ways to cover his tracks."}]}
-{"id": "user_oriented_task_82", "motivation_app": "IMDB", "instruction": "Decide whether children can watch the given movie. Mention what makes it inappropriate if you think they can't.", "instances": [{"input": "The Dark Knight (2008)", "output": "No, children should not watch The Dark Knight. The movie is rated PG-13 for intense sequences of violence and action, some menace, and brief language."}]}
-{"id": "user_oriented_task_83", "motivation_app": "IMDB", "instruction": "Write a short analysis of the cinematography in the movie.", "instances": [{"input": "Dead Poets Society", "output": "'Dead Poets Society' uses film techniques such as camera angles and camera techniques to help emphasise certain emotions between each character. The Film techniques are used to help the audience understand the theme, character, emotion and setting of a specific scene. All movies use Film Techniques to help the audience join in the experience and share the emotions that occurring in the film. Film techniques used in this film help shape meaning and give a better understanding of the relationship between the characters in the film. Although this film does not only use film techniques to emphasize the relationship of only characters, it uses it to emphasizes the relationship of characters with the themes of the film such as, poetry, carpe diem (seize the day), The school etc."}]}
-{"id": "user_oriented_task_84", "motivation_app": "tripadvisor.com", "instruction": "Please let me know your thoughts on the given place and why you think it deserves to be visited.", "instances": [{"input": "Barcelona, Spain", "output": "There\u2019s nowhere in the world like Barcelona. From days spent strolling the Gothic Quarter to afternoons at the beach and nights partying \u2018til dawn, the city is magnetic. While tapas and bustling mercados are worth every bit of attention they get in guidebooks, Barcelona\u2019s true crown jewel is the work of Antoni Gaud\u00ed. His iconic creations, like the jaw-dropping Basilica of La Sagrada Familia, are instantly recognizable and have fueled millions of travel dreams. You won\u2019t be the only tourist here (especially during the summer), but even amid the crowds you\u2019ll find your own Mediterranean magic any time of year."}]}
-{"id": "user_oriented_task_85", "motivation_app": "tripadvisor.com", "instruction": "Consider the best time of year to visit the given city, and provide your reasons for choosing that time.", "instances": [{"input": "Sydney, Australia", "output": "March to May: The best times to visit Sydney are between March and May and September and November. You\u2019ll find more comfortable temperatures and fewer crowds \u2014 which means you can likely nab better prices on airfare, too.\nBut if you want to warm up with the masses from winter back home, the average daily temperatures in January can hit highs of 30\u00b0C/86\u00b0F and lows around 20\u00b0C/67\u00b0F."}]}
-{"id": "user_oriented_task_86", "motivation_app": "tripadvisor.com", "instruction": "Make a questionnaire to help hotel guests write hotel reviews.", "instances": [{"input": "", "output": "1- Your overall rating of this property:\n2- Title of your review:\n3- Summarize your visit or highlight an interesting detail:\n4- Your Review:\n5- What sort of trip was this? Business/Couples/Family/Friends/Solo\n6- When did you travel?\n7- Could you say a little more about it? (optional)\nWe'd love your opinion! Anything you can share will help other travelers choose their perfect hotel. Thanks.\nHotel Style & Amenities: \nDoes this hotel have laundry service? Yes/No/Not Sure\nDoes this hotel have a game room? Yes/No/Not Sure\nDoes this hotel have a sun terrace? Yes/No/Not Sure\nIs the front desk of the hotel staffed 24-hours a day? Yes/No/Not Sure\n8-  How expensive is this hotel? Budget/Mid-range/Luxury\n9- Add a tip to help travelers choose a good room PE.g., best views, quieter floors, accessibility, etc.)\n10- Do you have photos to share? (optional)"}]}
-{"id": "user_oriented_task_87", "motivation_app": "tripadvisor.com", "instruction": "Make a list of the top places in the U.S. to visit at the given time.", "instances": [{"input": "November", "output": "1- For mild temperatures and local brews: San Diego\n2- For the last gasp of fall foliage: Manchester, VT\n3- For unexpected art and mountain views: Las Vegas\n4- For ice skating and family fun: Chicago\n5- For a waterfront Thanksgiving experience: Key West, FL\n6- For great food and ghost tours: Savannah, GA"}]}
-{"id": "user_oriented_task_88", "motivation_app": "tripadvisor.com", "instruction": "Name the top cities in France that should not be missed. Include the best aspects of each place as well.", "instances": [{"input": "", "output": "1- Paris, Best for: Historic sights and romantic vibes\n2- Nice, Best for: Architecture and dining\n3- Lyon, Best for: Dining and live music\n4- Bordeaux, Best for: Wine and antiques\n5- Marseille, Best for: Museums and nightlife\n6- Strasbourg, Best for: Historic architecture and dining"}]}
-{"id": "user_oriented_task_89", "motivation_app": "StackOverflow", "instruction": "Answer the following question.", "instances": [{"input": "How do I compare strings in Java?\nI've been using the == operator in my program to compare all my strings so far. However, I ran into a bug, changed one of them into .equals() instead, and it fixed the bug.\nIs == bad? When should it and should it not be used? What's the difference?", "output": "== tests for reference equality (whether they are the same object).\n.equals() tests for value equality (whether they are logically \"equal\").\nObjects.equals() checks for null before calling .equals() so you don't have to (available as of JDK7, also available in Guava).\nConsequently, if you want to test whether two strings have the same value you will probably want to use Objects.equals().\nYou almost always want to use Objects.equals(). In the rare situation where you know you're dealing with interned strings, you can use ==."}]}
-{"id": "user_oriented_task_90", "motivation_app": "StackOverflow", "instruction": "You are given a question and two possible answers. The task is to evaluate each answer and determine if it is correct. If incorrect, explain why.", "instances": [{"input": "Is there a way of reading one single character from the user input? For instance, they press one key at the terminal and it is returned (sort of like getch()). I know there's a function in Windows for it, but I'd like something that is cross-platform.\nAnswer 1: Here's a link to the ActiveState Recipes site that says how you can read a single character in Windows, Linux and OSX :\nhttps://code.activestate.com/recipes/134892/\nAnswer 2: if you just want to hold the screen so you can see the result on the terminal just write `input()` at the end of the code and it will hold the screen", "output": "Answer 1 is correct.\nAnswer 2 is incorrect because it does not answer the question. The user is asking how to read a single character."}]}
-{"id": "user_oriented_task_91", "motivation_app": "StackOverflow", "instruction": "You are given two questions. Check if they are duplicates.", "instances": [{"input": "Question1: I am trying to understand the advantages of multiprocessing over threading. I know that multiprocessing gets around the Global Interpreter Lock, but what other advantages are there, and can threading not do the same thing?\nQuestion2: I am learning how to use the threading and the multiprocessing modules in Python to run certain operations in parallel and speed up my code.\nI am finding this hard (maybe because I don't have any theoretical background about it) to understand what the difference is between a threading.Thread() object and a multiprocessing.Process() one.\nAlso, it is not entirely clear to me how to instantiate a queue of jobs and having only 4 (for example) of them running in parallel, while the other wait for resources to free before being executed.\nI find the examples in the documentation clear, but not very exhaustive; as soon as I try to complicate things a bit, I receive a lot of weird errors (like a method that can't be pickled, and so on).\nSo, when should I use the threading and multiprocessing modules?\nCan you link me to some resources that explain the concepts behind these two modules and how to use them properly for complex tasks?", "output": "Yes, they are duplicates."}]}
-{"id": "user_oriented_task_92", "motivation_app": "Leetcode", "instruction": "Design a programming problem related to the subject that has been given to you. Use some examples and constraints to improve your question.", "instances": [{"input": "Dynamic Programming", "output": "Given an integer array nums, return the length of the longest strictly increasing subsequence.\n\n Example 1:\nIn: nums = [10,9,2,5,3,7,101,18]\nOut: 4\nExplanation: The longest increasing subsequence is [2,3,7,101], therefore the length is 4.\n\nExample 2:\nIn: nums = [0,1,0,3,2,3]\nOut: 4\n\nExample 3:\nIn: nums = [7,7,7,7,7,7,7]\nOut: 1\n \nConstraints:\n1 <= nums.length <= 2500\n-104 <= nums[i] <= 104\n \nFollow up: Can you come up with an algorithm that runs in O(n log(n)) time complexity?"}]}
-{"id": "user_oriented_task_93", "motivation_app": "Leetcode", "instruction": "Provide a Java solution to the following problem.", "instances": [{"input": "Given an integer n, return true if it is a power of three. Otherwise, return false.\nAn integer n is a power of three, if there exists an integer x such that n == 3^x.\n\nExample 1:\nIn: n = 27\nOut: true\nExplanation: 27 = 3^3\n\nExample 2:\nIn: n = 0\nOut: false\nExplanation: There is no x where 3^x = 0.\n\nExample 3:\nIn: n = -1\nOut: false\nExplanation: There is no x where 3x = (-1).\n \nConstraints:\n-231 <= n <= 231 - 1\nFollow up: Could you solve it without loops/recursion?", "output": "public class Solution {\n    public boolean isPowerOfThree(int n) {\n        return (Math.log10(n) / Math.log10(3)) % 1 == 0;\n    }\n}"}]}
-{"id": "user_oriented_task_94", "motivation_app": "Leetcode", "instruction": "Think of topics that are most common in classic interview questions for a job in computer science.", "instances": [{"input": "", "output": "+ Array\n+ Strings\n+ Linked List\n+ Trees\n+ Sorting and Searching\n+ Dynamic Programming\n+ Design\n+ Math"}]}
-{"id": "user_oriented_task_95", "motivation_app": "Leetcode", "instruction": "You will need to guide this person through the scenario they have described for the interview.", "instances": [{"input": "Finally! After a while of shooting in the dark and frantically fiddling with sample inputs on the whiteboard, you've came up with an algorithm for solving the coding question your interviewer gave you.\nWhew. Such a relief to have a clear path forward. To not be flailing anymore.\nNow you're cruising, getting ready to code up your solution.\nWhen suddenly, your interviewer throws you a curve ball.\n\"What if we thought of the problem this way?\"\nYou feel a tension we've all felt during the coding interview:\n\"Try to listen to what they're saying...but don't lose your train of thought...ugh, I can't do both!\"", "output": "Your interviewer probably won't fault you for going down the wrong path at first. They've seen really smart engineers do the same thing. They understand it's because you only have a partial view of the maze.\nThey might have let you go down the wrong path for a bit to see if you could keep your thinking organized without help. But now they want to rush you through the part where you discover the dead end and double back. Not because they don't believe you can manage it yourself. But because they want to make sure you have enough time to finish the question.\nBut here's something they will fault you for: failing to listen to them. Nobody wants to work with an engineer who doesn't listen.\nSo when you find yourself in that crucial coding interview moment, when you're torn between holding your train of thought and considering the idea your interviewer is suggesting...remember this:\nListening to your interviewer is the most important thing.\nTake what they're saying and run with it. Think of the next steps that follow from what they're saying.\nEven if it means completely leaving behind the path you were on. Trust the route your interviewer is pointing you down.\nBecause they can see the whole maze."}]}
-{"id": "user_oriented_task_96", "motivation_app": "Leetcode", "instruction": "You are given a programming problem and its implementation. Analyze the problem and implementation and explain the algorithm and approach used to solve the problem.", "instances": [{"input": "Table: Person\n| Column Name | Type     |\n| personId          | int         |\n| lastName         | varchar |\n| firstName         | varchar |\npersonId is the primary key column for this table.\nThis table contains information about the ID of some persons and their first and last names.\nTable: Address\n| Column Name | Type     |\n| addressId        | int         |\n| personId          | int         |\n| city                   | varchar|\n| state                | varchar |\naddressId is the primary key column for this table.\nEach row of this table contains information about the city and state of one person with ID = PersonId.\nWrite an SQL query to report the first name, last name, city, and state of each person in the Person table. If the address of a personId is not present in the Address table, report null instead.\nReturn the result table in any order.\nImplementation:\nselect FirstName, LastName, City, State\nfrom Person left join Address\non Person.PersonId = Address.PersonId\n;", "output": "Since the PersonId in table Address is the foreign key of table Person, we can join these two tables to get the address information of a person.\nConsidering there might be no address information for every person, we should use outer join instead of the default inner join."}]}
-{"id": "user_oriented_task_97", "motivation_app": "Wikipedia", "instruction": "The given article should be checked for any scientific errors and corrected if necessary.", "instances": [{"input": "Zinc is an essential mineral that is required for good health. It is involved in many biochemical processes in the body and is necessary for the proper functioning of the immune system. Excess zink is not harmful as it is excreted in the urine.", "output": "Although zinc is an essential requirement for good health, excess zinc can be harmful. Excessive absorption of zinc suppresses copper and iron absorption. The free zinc ion in solution is highly toxic to plants, invertebrates, and even vertebrate fish. The Free Ion Activity Model is well-established in the literature, and shows that just micromolar amounts of the free ion kills some organisms. A recent example showed 6 micromolar killing 93% of all Daphnia in water.\nThe free zinc ion is a powerful Lewis acid up to the point of being corrosive. Stomach acid contains hydrochloric acid, in which metallic zinc dissolves readily to give corrosive zinc chloride. Swallowing a post-1982 American one cent piece (97.5% zinc) can cause damage to the stomach lining through the high solubility of the zinc ion in the acidic stomach."}]}
-{"id": "user_oriented_task_98", "motivation_app": "Wikipedia", "instruction": "Summarize the article you have been given in a brief manner.", "instances": [{"input": "Mathematics and art are related in a variety of ways. Mathematics has itself been described as an art motivated by beauty. Mathematics can be discerned in arts such as music, dance, painting, architecture, sculpture, and textiles. This article focuses, however, on mathematics in the visual arts.\nMathematics and art have a long historical relationship. Artists have used mathematics since the 4th century BC when the Greek sculptor Polykleitos wrote his Canon, prescribing proportions conjectured to have been based on the ratio 1:\u221a2 for the ideal male nude. Persistent popular claims have been made for the use of the golden ratio in ancient art and architecture, without reliable evidence. In the Italian Renaissance, Luca Pacioli wrote the influential treatise De divina proportione (1509), illustrated with woodcuts by Leonardo da Vinci, on the use of the golden ratio in art. Another Italian painter, Piero della Francesca, developed Euclid's ideas on perspective in treatises such as De Prospectiva Pingendi, and in his paintings. The engraver Albrecht D\u00fcrer made many references to mathematics in his work Melencolia I. In modern times, the graphic artist M. C. Escher made intensive use of tessellation and hyperbolic geometry, with the help of the mathematician H. S. M. Coxeter, while the De Stijl movement led by Theo van Doesburg and Piet Mondrian explicitly embraced geometrical forms. Mathematics has inspired textile arts such as quilting, knitting, cross-stitch, crochet, embroidery, weaving, Turkish and other carpet-making, as well as kilim. In Islamic art, symmetries are evident in forms as varied as Persian girih and Moroccan zellige tilework, Mughal jali pierced stone screens, and widespread muqarnas vaulting.", "output": "This article discusses the relationship between mathematics and art, with a focus on visual arts. It describes how artists have used mathematics since ancient times, and how mathematics has inspired various textile arts."}]}
-{"id": "user_oriented_task_99", "motivation_app": "Wikipedia", "instruction": "Improve the article using your knowledge of the topic in order to make it more informative.", "instances": [{"input": "The \"fair trade\" movement, also known as the \"trade justice\" movement, promotes the use of labour, environmental and social standards for the production of commodities, particularly those exported from the Third and Second Worlds to the First World. Such ideas have also sparked a debate on whether trade itself should be codified as a human right.", "output": "The \"fair trade\" movement, also known as the \"trade justice\" movement, promotes the use of labour, environmental and social standards for the production of commodities, particularly those exported from the Third and Second Worlds to the First World. Such ideas have also sparked a debate on whether trade itself should be codified as a human right.\nImporting firms voluntarily adhere to fair trade standards or governments may enforce them through a combination of employment and commercial law. Proposed and practiced fair trade policies vary widely, ranging from the common prohibition of goods made using slave labour to minimum price support schemes such as those for coffee in the 1980s. Non-governmental organizations also play a role in promoting fair trade standards by serving as independent monitors of compliance with labeling requirements. As such, it is a form of Protectionism."}]}
-{"id": "user_oriented_task_100", "motivation_app": "Wikipedia", "instruction": "Make the article available in a second language by translating it into it.", "instances": [{"input": "Dentistry, also known as dental medicine and oral medicine, is the branch of medicine focused on the teeth, gums, and mouth. It consists of the study, diagnosis, prevention, management, and treatment of diseases, disorders, and conditions of the mouth, most commonly focused on dentition (the development and arrangement of teeth) as well as the oral mucosa. Dentistry may also encompass other aspects of the craniofacial complex including the temporomandibular joint. The practitioner is called a dentist.\nTranslate to French:", "output": "La dentisterie, \u00e9galement connue sous le nom de m\u00e9decine dentaire et de m\u00e9decine buccale, est la branche de la m\u00e9decine ax\u00e9e sur les dents, les gencives et la bouche. Il consiste en l'\u00e9tude, le diagnostic, la pr\u00e9vention, la gestion et le traitement des maladies, troubles et affections de la bouche, le plus souvent ax\u00e9s sur la dentition (le d\u00e9veloppement et la disposition des dents) ainsi que sur la muqueuse buccale.\nLa dentisterie peut \u00e9galement englober d'autres aspects du complexe craniofacial, y compris l'articulation temporo-mandibulaire. Le praticien est appel\u00e9 dentiste."}]}
-{"id": "user_oriented_task_101", "motivation_app": "Wikipedia", "instruction": "An evaluation of the article's quality should be carried out. In order to do so, you should review the quality of the writing and the explanation of the topic.", "instances": [{"input": "The 20th century saw a revolution in music listening as the radio gained popularity worldwide and new media and technologies were developed to record, edit and distribute music. Music performances became increasingly visual with the broadcast and recording of performances.\n20th-century music brought new freedom and wide experimentation with new musical styles and forms that challenged the accepted rules of music of earlier periods. The invention of musical amplification and electronic instruments, especially the synthesizer, in the mid-20th century revolutionized classical and popular music, and accelerated the development of new forms of music.", "output": "The article is well-written and researched, and provides a good overview of the topic."}]}
-{"id": "user_oriented_task_102", "motivation_app": "Wikipedia", "instruction": "Please answer the following question based on the information provided in the article.", "instances": [{"input": "Development studies is an interdisciplinary branch of social science. Development studies is offered as a specialized master's degree in a number of reputed universities around the world. It has grown in popularity as a subject of study since the early 1990s, and has been most widely taught and researched in developing countries and countries with a colonial history, such as the UK, where the discipline originated.  Students of development studies often choose careers in international organisations such as the United Nations, World Bank, non-governmental organisations (NGOs), media and journalism houses, private sector development consultancy firms, corporate social responsibility (CSR) bodies and research centers.\nWhat are some popular careers for students of development studies?", "output": "Some popular careers for students of development studies include working for international organisations such as the United Nations, World Bank, or NGOs, as well as working in media, journalism, private sector development consultancy, or corporate social responsibility."}]}
-{"id": "user_oriented_task_103", "motivation_app": "Coursera", "instruction": "Design a syllabus for the given course. Students should be given a list of the chapters with brief explanations of each chapter's purpose.", "instances": [{"input": "Programming for Everybody (Getting Started with Python)", "output": "- Chapter One - Why we Program?\nThese are the course-wide materials as well as the first part of Chapter One where we explore what it means to write programs. We finish Chapter One and have the quiz and first assignment in the third week of the class. Throughout the course you may want to come back and look at these materials. This section should not take you an entire week.\n- Installing and Using Python\nIn this module you will set things up so you can write Python programs. Not all activities in this module are required for this class so please read the \"Using Python in this Class\" material for details.\n- Chapter One: Why We Program (continued)\nIn the first chapter we try to cover the \"big picture\" of programming so you get a \"table of contents\" of the rest of the book. Don't worry if not everything makes perfect sense the first time you hear it. This chapter is quite broad and you would benefit from reading the chapter in the book in addition to watching the lectures to help it all sink in. You might want to come back and re-watch these lectures after you have funished a few more chapters.\n- Chapter Two: Variables and Expressions\nIn this chapter we cover how a program uses the computer's memory to store, retrieve and calculate information.\n- Chapter Three: Conditional Code\nIn this section we move from sequential code that simply runs one line of code after another to conditional code where some steps are skipped. It is a very simple concept - but it is how computer software makes \"choices\".\n- Chapter Four: Functions\nThis is a relatively short chapter. We will learn about what functions are and how we can use them. The programs in the first chapters of the book are not large enough to require us to develop functions, but as the book moves into more and more complex programs, functions will be an essential way for us to make sense of our code.\n- Chapter Five: Loops and Iteration\nLoops and iteration complete our four basic programming patterns. Loops are the way we tell Python to do something over and over. Loops are the way we build programs that stay with a problem until the problem is solved."}]}
-{"id": "user_oriented_task_104", "motivation_app": "Coursera", "instruction": "List the personality traits that are required to be successful in the given job.", "instances": [{"input": "Social Media Marketer", "output": "- Engaging with people\n- Thinking creatively\n- Following trends"}]}
-{"id": "user_oriented_task_105", "motivation_app": "Coursera", "instruction": "Please list the courses that someone interested in the first course might like.", "instances": [{"input": "Bitcoin and Cryptocurrency Technologies", "output": "- Blockchain 360: A State of the Art for Professionals\n- Blockchain Scalability and its Foundations in Distributed Systems\n- Blockchain Business Models\n- Financial Technology (Fintech) Innovations Specialization"}]}
-{"id": "user_oriented_task_106", "motivation_app": "Coursera", "instruction": "Come up with the courses that one is supposed to take in order to be an expert in a given field.", "instances": [{"input": "Graphic Design", "output": "1- Fundamentals of Graphic Design\n2- Introduction to Typography\n3- Introduction to Imagemaking\n4- Ideas from the History of Graphic Design\n5- Brand New Brand"}]}
-{"id": "user_oriented_task_107", "motivation_app": "National Geographic", "instruction": "Create a daily itinerary based on the given information.", "instances": [{"input": "Our family is looking for a 9-day Morocco trip that has light to moderate activity levels.", "output": "1- DAY 1: MARRAKECH\nArrive in time to meet your fellow travelers for a welcome gathering this evening. Your arrival transfer is included.\n2- DAY 2: MARRAKECH/OUARZAZATE\nWind past red slopes and green terraces as we make our way south through the stunning valleys of Morocco\u2019s High Atlas Mountains, stopping to enjoy a picnic lunch beside a sparkling river. Arrive in Ouarzazate, known as the \u201cgateway to the Sahara,\u201d and step behind the scenes at a film studio to see how the surrounding dunes have starred on the silver screen.\n3- DAY 3: OUARZAZATE/MERZOUGA\nToday, travel between jagged peaks and palm oases toward the desert town of Merzouga. Settle into our hotel near the incredible Erg Chebbi, a sweeping sea of dunes formed by wind-blown sand. As the sun sets, savour a traditional Amazigh dinner, then spread a blanket on the sand for an evening of desert stargazing.\n4- DAY 4: MERZOUGA\nThis morning, hop aboard a 4x4 and ride across the dunes to discover fossils left behind by an ancient sea, and be entranced by Gnaoua musicians in a local village. Return to the hotel for free time to cool off in the pool. Enjoy a tasty Amazigh dinner, then gather around a glowing bonfire for a storytelling session.\n5- DAY 5: MERZOUGA/DADES GORGE\nVenture into the rugged landscape of the Dades Gorge, a dramatic and colourful valley carved by the waters of the Dades River. Visit a local village for lunch and enjoy a special dish\u2014traditional Amazigh pizza! Wind through the valley on a guided walk as you learn about the amazing geology behind the weird and wonderful rock formations of the Gorge. This evening, enjoy free time to relax at the hotel.\n6- DAY 6: DADES GORGE/A\u00cfT BEN HADDOU\nDrive through the Ounila Valley to the ancient fortified village of A\u00eft Ben Haddou, a UNESCO World Heritage site where clay houses crowd together in a traditional ksar behind defensive walls. Explore alleyways twisting between historic structures, and stand guard atop the clay brick ramparts, taking in views of the green oasis around this imposing fort. Entertain your taste buds at a hands-on tagine cooking class, followed by a delicious dinner.\n7- DAY 7: A\u00cfT BEN HADDOU/MARRAKECH\nJourney through the mountains to Marrakech, known as the \u201cred city\u201d for its walls, houses, and many mosques built from red clay. After checking into your hotel and resting up, head to a local henna workshop to have your hands painted in this ancient temporary tattooing art. Then, explore the lively Djemaa el Fna, an ancient marketplace still used as the main square in Marrakech. After, enjoy free time to discover more of Marrakech.\n8- DAY 8: MARRAKECH \nWake up as the call to prayer echoes across the city. After breakfast, set out on a guided walk through the narrow, maze-like streets of the medina, lined with colorful shops brimming with rugs, spices, and much more. Spend the rest of the day exploring Marrakech on your own, or opt to explore the Marjorelle Garden. In the evening, gather for a farewell dinner in a traditional riad, a grand house with an elegant interior courtyard.\n9- DAY 9: MARRAKECH\nDepart at any time."}]}
-{"id": "user_oriented_task_108", "motivation_app": "National Geographic", "instruction": "Give students tips on how to keep their nerves under control during class presentations.", "instances": [{"input": "", "output": "Everyone gets a little nervous when speaking in front of a group. That\u2019s normal. But the more preparation you\u2019ve done\u2014meaning plenty of researching, organizing, and rehearsing\u2014the more confident you\u2019ll be. Preparation is the key. And if you make a mistake or stumble over your words, just regroup and keep going. Nobody\u2019s perfect, and nobody expects you to be."}]}
-{"id": "user_oriented_task_109", "motivation_app": "National Geographic", "instruction": "You can easily make your holidays more eco-friendly by following these simple steps:", "instances": [{"input": "", "output": "- If you prefer to buy a tree, look for local growers with advice on where to recycle. To complete the natural look, make ornaments from collected and dried foliage.\n- Another way to be environmentally friendly during the holidays is to use candles made with soy, ethical beeswax, and coconut wax instead of those made with petroleum-based paraffin wax. LED lights are also an environmentally friendly option among artificial lights.\n-Another way to reduce waste is to use brown paper grocery bags or furoshiki, which is swaddling gifts in reusable squares of cloth, rather than wrapping paper. \n- Instead of throwing away the excess food, find groups that redistribute it at epa.gov by searching for \u201creduce wasted food.\u201d What\u2019s no longer edible might be compostable."}]}
-{"id": "user_oriented_task_110", "motivation_app": "instructables", "instruction": "Think of an idea to help you win a challenge given to you. Write the steps you need to take to develop the idea as well.", "instances": [{"input": "What can you make by recycling and repurposing some old, previously used materials? We want to see it!\nProjects must incorporate some kind of used materials - take that old used thing, and turn it into something new and different!\nE-waste is a huge problem. What can you make using old electronics and other salvageable electronic materials? \nWe have a special judges prize for our favorite Instructable that incorporates or is related to reusing e-waste.\nBe smart and be safe; note that potentially deadly projects (like Lichtenberg/fractal wood burning machines) will not be allowed to be published or submitted to this contest.", "output": "This power booster recycles a 'dead' battery to power a 1.8 volt red LED light. It's mainly made from recycled parts.\nWhen a 1.5 volt battery is new it will probably have more than 1.5 volts in it. As the battery is used the voltage will decrease until the voltage is not enough to run the application ( E.g. a red LED) and will be called 'dead' . But a Dead battery still has some power remaining in it - just not enough voltage. The trick is to use some of that remaining power to increase the voltage in a circuit. The trade off being there will be a reduction in current.\nHere is the required steps:\n1- Connect a switch to a battery holder. A soldiering iron and hot glue gun assisted with this.\n2- Make a small pyramid above the battery holder using 4 bamboo skewer sticks secured with hot glue.\n3- Soldier on a 1.8 volt red LED at the top of the mini Christmas tree.\n4- Test the LED glows with a new battery (1.6 volts)\n5- Test the LED doesn't glow with a dead battery (1.2 volts)\n6- Add in the buzzer circuit, along with a switch.\n7- Add in the windings circuit - to touch the striker of the buzzer a little copper foil was added.\n8- Insert a dead battery (1.2 volt) with the buzzer circuit turned switch off - the LED will not glow but when the buzzer circuit is switched on the Red LED lights up."}]}
-{"id": "user_oriented_task_111", "motivation_app": "instructables", "instruction": "There are many creative projects we can build at home that are related to the given theme, so let's list some of them.", "instances": [{"input": "Theme: Circuits", "output": "1- Build a UV Level Monitoring Budgie - Using IoT and Weather Data APIs\n2- DIY Weather Station With ESP32\n3- Modular Display Clock\n4- Overengineered Fume Extractor Project"}]}
-{"id": "user_oriented_task_112", "motivation_app": "instructables", "instruction": "Make a list of the materials that will be required to build the given tool.", "instances": [{"input": "Coffee Darkness Meter", "output": "- Arduino Nano (or Uno)\n- 0.96\" OLED display\n- TSL2561 lux and IR sensor on a brake-out board (I used Adafruit's)\n- ON/OFF toggle switch\n- Momentary push button\n- 9V battery and battery clip\n- 2x IR LEDs\n- 2x 100ohm resistors\n- 2x 10K resistors\n- 2x 2.2K resistors\n- wire for connecting\n- case for everything (3D printed)"}]}
-{"id": "user_oriented_task_113", "motivation_app": "instructables", "instruction": "Provide a detailed explanation of the steps that are required in order to construct the given project.", "instances": [{"input": "Steam Engine Using an Empty Beer Can", "output": "Step 1: Close Drink Opening\nTo build up steam the beverage can needs to act as a pressure vessel and so needs to be completely sealed, except where the steam will come out of the jet nozzles. It also needs to rotate on itself. Bend up the outer tab into a L shape using some pilers and then bend it forward so a piece of string can hang from it over the center of the can. This will allow the can to rotate on itself. Close the pouring opening by using a knife or other thin object, bending the opening back up so it almost where it was before the beverage can was opened.\nStep 2: Nozzel Jets\nMake a couple of holes in the side of the can, about half way up, opposite each other using a small drill. The diameter of the holes should be about the same as the thin plastic tubing to be used as jet nozzles. Enlarge the holes so they are elliptical allowing the nozzle tubes to lie parallel against the cans side.\nStep 3: Making a Pressure Vessel\nSeal the top of the opened can using super glue and baking soda to make a small pressure vessel. Add some super glue to the closed opening and then sprinkle on some baking soda - they will combine to make a plastic. Keep adding super glue and baking soda until the top is sealed.\nStep 4: Attaching the Nozzle Jets\nFit in the thin plastic tubes to the holes drilled in the sides of the can and seal with super glue and baking soda. Try to have the tubes pointing horizontal in the direction you want to rotate the can when the steam exits the nozzles.\nStep 5: Creating Steam\nAdd a little water into the can though one of the nozzles. Just add maybe a teaspoon or so. \nStep 6: In Action\nThe can was suspended from a tripod over the methylated sprits flame. After a couple of minutes enough steam had built up to rotate the can."}]}
-{"id": "user_oriented_task_114", "motivation_app": "instructables", "instruction": "Estimate the cost of supplies required to create the given project and calculate the total cost.", "instances": [{"input": "DIY Water Chiller\nRequired items are: Ice-maker, Icebox/cooler, Temperature regulator, Aquarium pump x2, 12V power supply", "output": "Ice-maker ($90)\nIcebox/cooler ($17)\nTemperature regulator ($16)\nAquarium pump x2 ($13)\n12V power supply ($8)\nTotal cost = $144"}]}
-{"id": "user_oriented_task_115", "motivation_app": "Socratic by Google", "instruction": "Answer the following literature question as accurately as possible based on your knowledge of literature for high school students.", "instances": [{"input": "What does the green light symbolize at the end of the book?", "output": "The green light symbolizes several different things to different characters throughout the book, but overall, it stands for the inaccessibility of the American Dream. At the start of the novel, the light symbolizes Gatsby's unwavering love for Daisy. He stands at the end of the dock gazing across the bay, reaching for the light, waiting for Daisy to come back to him. His hope, like the eternal green light, is unfaltering. It isn't until Daisy admits later that she had never really noticed the light that the facade began to slip. Nick, the narrator, starts to recognize it as a symbol of Gatsby's unhealthy desperation rather than his love. This is parallel to Gatsby's experience of the American Dream and money. Gatsby is unbelievably rich at the beginning of the novel. This solidifies his belief in the American Dream, though he still reaches for the acceptance of those who are wealthier and more popular. Nick believes money is not an object to Gatsby, though he has no idea how he made all of it. As the story unfolds and the mystery and mystique around Gatsby fade, so does Gatsby and Nick's blind belief and trust in the American Dream. After Gatsby is murdered, the green light is symbolic of the ways that money and affluence ruin the lives of those without. It is also symbolic of the way that the American dream ruins the lives of those eternally reaching for it."}]}
-{"id": "user_oriented_task_116", "motivation_app": "Wysa", "instruction": "Develop a mental exercise that can help people manage their anxiety and explain how it works.", "instances": [{"input": "", "output": "Submodalities\nGiving a physical form to your anxiety can help your mind feel more in control of it. In this exercise, we will visualize what anxiety looks and feels like, and learn to reduce its intensity. Here are the questions that you need to answer:\n1- Everyone feels anxiety differently. Where in your body do you feel it?\n2- Now close your eyes and picture your anxiety. Try to give it color, shape, and temperature. What does it feel like?\n3- Does it move, or is it in one place? If it moves, tell how it is moving.\n4- Now for the good part, close your eyes and try changing this picture. Change how it moves. Slow it done shrink its size. Try to move it away from your body. Imagine it fading. Keep doing this until you feel like you can change how it feels. Changing which submodality helped you curb your anxiety? Was it its shape, size, or something else?\n5- Do you feel better now?"}]}
-{"id": "user_oriented_task_117", "motivation_app": "Wysa", "instruction": "Give me an idea of the mental exercises you do to inspire yourself.", "instances": [{"input": "", "output": "1- Identify your strengths\n2- Understand you motivation style\n3- Build worthiness\n4- Dream ahead"}]}
-{"id": "user_oriented_task_118", "motivation_app": "Weather", "instruction": "Make a list of outdoor activities that are suitable for the given weather condition.", "instances": [{"input": "Temperature: 57 F, Showers", "output": "- Take a walk in the rain\n- Jump in puddles\n- Put on your own musical in the rain\n- Paint with Mud\n- Build a shelter \n- Make Nature Boats \n- Create a Dam"}]}
-{"id": "user_oriented_task_119", "motivation_app": "Weather", "instruction": "Please provide us with a list of the best conservation starter questions related to the weather.", "instances": [{"input": "", "output": "1. Beautiful day, isn't it?\n2. Can you believe all of this rain we've been having?\n3. It looks like it's going to storm.\n4. We couldn't ask for a nicer day, could we?\n5. How about this weather?\n6. It's so nice out today, isn't it?\n7. The weather's nice today, right?\n8. It\u2019s freezing today! Hopefully it doesn\u2019t snow.\n9. Wow, it\u2019s really hot/cold for this time of year.\n10. It\u2019s really been pouring all day, huh?"}]}
-{"id": "user_oriented_task_120", "motivation_app": "Weather", "instruction": "In relation to the given weather scenario, give some tips on how to adjust the travel plans with it.", "instances": [{"input": "a sudden temperature change", "output": "- Make sure that you have a good variety of options, and pack for the weather of your destination.\n- Rapid temperature changes can have major impacts on your body. Make sure you are aware of your particular responses to temperature change and pack accordingly. \n- Make sure to pack ointments, creams and lotions to help keep your skin healthy while you are on the go. \n- With extreme heat, you may experience rash, fainting, headaches, and fatigue as well as nausea, sweating, and loss of coordination. Be particularly aware of heat exhaustion and heat stroke as these are serious medical conditions that can have serious consequences. In cold climates, be aware of frostbite and hypothermia, intense shivering, goosebumps, difficulty using your hands, and false sense of warmth can indicate a developing cold illness."}]}
-{"id": "user_oriented_task_121", "motivation_app": "Notion", "instruction": "Write a to-do list based on the given information.", "instances": [{"input": "Daily tasks in the pharmacy", "output": "- Police sidewalk and entrance for litter \n- Assess window displays to make sure they are not outdated or faded\n- Clean all doors and remove outdated and unnecessary signs \n- Fill bags at each checkout area \n- Clean and straighten checkout counters \n- Tidy up seating/waiting area \n- Face (pull product forward) in all departments \n- Check one department a day for short and outdated product \n- Empty all trash receptacles\n- Vacuum (when the store is closed or only if customer traffic allows for it during store hours)"}]}
-{"id": "user_oriented_task_122", "motivation_app": "Notion", "instruction": "Design a template table for keeping track of all subscriptions.", "instances": [{"input": "", "output": "Service | Category | Status | Billing | Pricing | Renewal Date | Next Renewal | Monthly Costs | Yearly Costs | Nptes"}]}
-{"id": "user_oriented_task_123", "motivation_app": "Notion", "instruction": "Plan an agenda for the daily standup meetings. Assign a time duration to each topic on the agenda.", "instances": [{"input": "", "output": "[10 min] Status check\n- Current and upcoming work\n- Have spare bandwidth\n- Any blockers?\n[5 min] Big project status + general updates\n[5 min] Discuss blockers, otherwise social time\n[2 min] Add follow Ups"}]}
-{"id": "user_oriented_task_124", "motivation_app": "Doulingo", "instruction": "Answer the following question.", "instances": [{"input": "How do you say \"good evening\" in French.", "output": "bonsoir"}]}
-{"id": "user_oriented_task_125", "motivation_app": "Doulingo", "instruction": "What sound does this make?", "instances": [{"input": "\u597d", "output": "h\u01ceo"}]}
-{"id": "user_oriented_task_126", "motivation_app": "Doulingo", "instruction": "Give a grammar tip on how to end a German verb.", "instances": [{"input": "", "output": "Verbs have different endings depending on the subject of the sentence. Lucky for you er and sie use the same ending.\nSubject | Ending\nich        | e\ndu         | st\ner / sie  | t"}]}
-{"id": "user_oriented_task_127", "motivation_app": "Doulingo", "instruction": "This is a test for the reading ability of French. Read the given story carefully, and then answer the question in the end.", "instances": [{"input": "Lucie est \u00e0 la maison avec sa petite-fille, Lin.\nLucie: Oh, non! Nous n'avons pas de pain!\nLin: Ah! Tu vas au supermarch\u00e9?\nLucie: Oui, je veux une baguette pour mon sandwich.\nDo Lin and Lucie have a lot of bread?", "output": "No, that's wrong."}]}
-{"id": "user_oriented_task_128", "motivation_app": "Tasty", "instruction": "Come up with healthy and easy dinner ideas for weeknights.", "instances": [{"input": "", "output": "- Chicken & Veggie Stir-Fry\n- Classic Chicken Noodle Soup\n- Low-Carb Eggplant Lasagna\n- Avocado Lime Salmon\n- Zesty One-Pot Shrimp Pasta\n- Grilled Filet Mignon Street Tacos\n- Bruschetta Pasta"}]}
-{"id": "user_oriented_task_129", "motivation_app": "Tasty", "instruction": "Provide a cooking hack for improving the flavor of the given food.", "instances": [{"input": "popcorn", "output": "Everyone knows butter on popcorn is delicious. But supposedly adding some soy sauce to popcorn butter makes for a next-level popcorn topper. According to several sites, like Food52, the addition of salty soy sauce to creamy butter brings out the savory, cheesy, and nutty flavors in popcorn."}]}
-{"id": "user_oriented_task_130", "motivation_app": "Tasty", "instruction": "Make a list of snacks and foods to serve as party snacks on a game day!", "instances": [{"input": "", "output": "- Classic Chocolate Cake\n- Buffalo Chicken Sliders\n- Smoky Oven-Fried Chicken\n- Classic Hot Crab Dip For A Crowd\n- Potato Crust Breakfast Pizza\n- Muffin Tin Deviled Eggs"}]}
-{"id": "user_oriented_task_131", "motivation_app": "Tasty", "instruction": "Describe how to prepare the given food in your own words. Note down the ingredients you will need and the steps you will take to prepare them.", "instances": [{"input": "Chewy Chocolate Chip Cookies", "output": "Ingredients (for 12 cookies):\n\u00bd cup granulated sugar(100 g)\n\u00be cup brown sugar(165 g), packed\n1 teaspoon salt\n\u00bd cup unsalted butter(115 g), melted\n1 egg\n1 teaspoon vanilla extract\n1 \u00bc cups all-purpose flour(155 g)\n\u00bd teaspoon baking soda\n4 oz milk or semi-sweet chocolate chunks(110 g)\n4 oz dark chocolate chunk(110 g), or your preference\nPreparation:\nIn a large bowl, whisk together the sugars, salt, and butter until a paste forms with no lumps.\nWhisk in the egg and vanilla, beating until light ribbons fall off the whisk and remain for a short while before falling back into the mixture.\nSift in the flour and baking soda, then fold the mixture with a spatula (Be careful not to overmix, which would cause the gluten in the flour to toughen resulting in cakier cookies).\nFold in the chocolate chunks, then chill the dough for at least 30 minutes. For a more intense toffee-like flavor and deeper color, chill the dough overnight. The longer the dough rests, the more complex its flavor will be.\nPreheat oven to 350\u00b0F (180\u00b0C). Line a baking sheet with parchment paper.\nScoop the dough with an ice-cream scoop onto a parchment paper-lined baking sheet, leaving at least 4 inches (10 cm) of space between cookies and 2 inches (5 cm) of space from the edges of the pan so that the cookies can spread evenly.\nBake for 12-15 minutes, or until the edges have started to barely brown.\nCool completely before serving."}]}
-{"id": "user_oriented_task_132", "motivation_app": "Tasty", "instruction": "Suggest a recipe for a vegan dessert.", "instances": [{"input": "", "output": "Honey Lime Fruit Salad\nIngredients: (for 4 servings)\n\u00bd lb fresh strawberry(225 g), quartered\n2 kiwis, peeled and diced\n2 mangoes, diced\n2 bananas, sliced\n\u00bd lb fresh blueberry(225 g)\n2 tablespoons honey\n1 lime, juiced\n\nPreparation:\nPlace sliced fruits in a large bowl.\nIn a small bowl, mix honey and lime juice. Pour syrup over the fruit and mix."}]}
-{"id": "user_oriented_task_133", "motivation_app": "Sudoku", "instruction": "Design a medium-level sudoku puzzle.", "instances": [{"input": "", "output": "|  |  |  |  |  |6|1|4|9|\n|  |8|  |  |1|3|  |  |  |\n|  |  |6|4|2|  |7|  |  |\n|7|  |  |9|  |  |4|3|  |\n|3|6|  |  |  |  |5|9|1|\n|  |9|5|  |8|1|  |  |7|\n|  |  |  |6|  |2|9|  |5|\n|1|  |  |5|4|  |8|6|  |\n|6|5|2|  |  |  |  |7|  |"}]}
-{"id": "user_oriented_task_134", "motivation_app": "Sudoku", "instruction": "Solve the given Sudoku puzzle.", "instances": [{"input": "The Sudoku puzzle is:\n|1|  |5|  |  |  |  |8|  |\n|  |  |  |  |  |3|  |6|4|\n|8|3|4|6|7|  |9|  |  |\n|2|9|1|  |  |8|3|7|6|\n|  |  |6|  |  |9|8|1|  |\n|  |  |  |  |  |2|  |  |  |\n|  |2|  |  |9|7|6|  |  |\n|5|4|  |  |  |  |  |9|8|\n|6|1|  |  |  |5|4|3|7|", "output": "The solved Sudoku puzzle is:\n|1|6|5|9|2|4|7|8|9|\n|9|7|2|8|5|3|1|6|4|\n|8|3|4|6|7|1|9|2|5|\n|2|9|1|5|4|8|3|7|6|\n|4|5|6|7|3|9|8|1|2|\n|7|8|3|1|6|2|5|4|9|\n|3|2|8|4|9|7|6|5|1|\n|5|4|7|3|1|6|2|9|8|\n|6|1|9|2|8|5|4|3|7|"}]}
-{"id": "user_oriented_task_135", "motivation_app": "Goodreads", "instruction": "Think of a motivational quote that you have read in a book. Try to keep it positive and sweet.", "instances": [{"input": "", "output": "\u201cBelieve in yourself. You are braver than you think, more talented than you know, and capable of more than you imagine.\u201d\n\u2015 Roy T. Bennett, The Light in the Heart"}]}
-{"id": "user_oriented_task_136", "motivation_app": "Goodreads", "instruction": "Choose the best books from the given genre.", "instances": [{"input": "Crime & Mystery", "output": "1- The Girl with the Dragon Tattoo\n2- And Then There Were None\n3- Angels & Demons\n4- Rebecca\n5- In Cold Blood\n6- The Godfather\n7- The Lovely Bones\n8- Gone Girl\n9- The Name of the Rose\n10- Shutter Island"}]}
-{"id": "user_oriented_task_137", "motivation_app": "Goodreads", "instruction": "Give a brief description of the book that you are given.", "instances": [{"input": "Matilda", "output": "Matilda is a little girl who is far too good to be true. At age five-and-a-half she's knocking off double-digit multiplication problems and blitz-reading Dickens. Even more remarkably, her classmates love her even though she's a super-nerd and the teacher's pet. But everything is not perfect in Matilda's world...\nFor starters she has two of the most idiotic, self-centered parents who ever lived. Then there's the large, busty nightmare of a school principal, Miss (\"The\") Trunchbull, a former hammer-throwing champion who flings children at will, and is approximately as sympathetic as a bulldozer. Fortunately for Matilda, she has the inner resources to deal with such annoyances: astonishing intelligence, saintly patience, and an innate predilection for revenge."}]}
-{"id": "user_oriented_task_138", "motivation_app": "Goodreads", "instruction": "In the event that a person enjoyed the book that was given, please suggest other books that they might also like.", "instances": [{"input": "The Great Gatsby", "output": "To Kill a Mockingbird, The Catcher in the Rye, 1984, Animal Farm, Pride and Prejudice, The Hunger Games, Of Mice and Men, Lord of the Flies"}]}
-{"id": "user_oriented_task_139", "motivation_app": "Goodreads", "instruction": "Indicate the genre of the book to which it belongs.", "instances": [{"input": "Love in the Time of Cholera", "output": "Fiction"}]}
-{"id": "user_oriented_task_140", "motivation_app": "Wolfram alpha", "instruction": "Solve this equation.", "instances": [{"input": "x^3 - 4x^2 + 6x - 24 = 0", "output": "x = 4, x= -i * sqrt(6), x= i * sqrt(6)"}]}
-{"id": "user_oriented_task_141", "motivation_app": "Wolfram alpha", "instruction": "Prove the given statement using your information.", "instances": [{"input": "\"If x is a positive integer or a solution to x+3>4, then x>0 and x>12.\"", "output": "To determine if it's true, let's look first at the assumptions. We are assuming that either x is a positive integer, or that it solves the inequality x+3>4.\nNext let's consider the conclusion. We are concluding that x must satisfy both inequalities x>0 and x>12. If we look more closely, we see that once we satisfy the second inequality, the first is redundant. (If x>12, then it must already be larger than zero.)\nNow, in order for this statement to be true, we need that if x solves either of the assumptions, then it must solve x>12. Well, the first assumption is that x is a positive integer, which means that x\u22651, so in this case the conclusion holds. The second assumption is that x+3>4, or equivalently, that x>1, which means the conclusion holds as well."}]}
-{"id": "user_oriented_task_142", "motivation_app": "Wolfram alpha", "instruction": "Based on the facts that have been provided, prove the following statement.", "instances": [{"input": "Statement: \"For every integer k, k^2 + 2k + 1 \u2265 0\"\nFact: The square of any real number is non-negative.", "output": "Let k be an integer. Then (k + 1)^2 \u2265 0 because the square\nof any real number is non-negative. But (k+1)^2 = k^2+2k+1. So,\nby combining these two equations, we find that k^2 + 2k + 1 \u2265 0."}]}
-{"id": "user_oriented_task_143", "motivation_app": "Wolfram alpha", "instruction": "Verify the correctness of the given statement.", "instances": [{"input": "\"For all integers j and k, if j and k are odd, then jk is odd.\"", "output": "True"}]}
-{"id": "user_oriented_task_144", "motivation_app": "Wolfram alpha", "instruction": "Provide a formula for computing the nth term in the given sequence", "instances": [{"input": "5, 14, 23, 32, 41, ...", "output": "an = 9*n - 4"}]}
-{"id": "user_oriented_task_145", "motivation_app": "ESPN", "instruction": "Create a table listing all games that meet the specified criteria in the National Football League. Use the season, local time, game, and score as columns of the table.", "instances": [{"input": "Ravens home games in 2011", "output": "season | local time | game | score\n2011-12 | Sun, Jan 15 | Texans (11-7) at Ravens (13-4) | 13 - 20\n2011-12 | Sat, Dec 24 | Browns (4-11) at Ravens (11-4) | 14 - 20\n2011-12 | Sun, Dec 11 | Colts (0-13) at Ravens (10-3) | 10 - 24\n2011-12 | Thu, Nov 24 | 49ers (9-2) at Ravens (8-3) | 6 - 16\n2011-12 | Sun, Nov 20 | Bengals (6-4) at Ravens (7-3) | 24 - 31\n2011-12 | Sun, Oct 30 | Cardinals (1-6) at Ravens (5-2) | 27 - 30\n2011-12 | Sun, Oct 16 | Texans (3-3) at Ravens (4-1) | 14 - 29\n2011-12 | Sun, Oct 2 | Jets (2-2) at Ravens (3-1) | 17 - 34\n2011-12 | Sun, Sep 11 | Steelers (0-1) at Ravens (1-0) | 7 - 35"}]}
-{"id": "user_oriented_task_146", "motivation_app": "ESPN", "instruction": "Using a table, compare the career overviews of the given players in Major League Baseball. Use \"|\" for separating the columns in the table.", "instances": [{"input": "Derek Jeter, Albert Pujols", "output": "| D. Jeter | A. Pujols\ngames played | 2747 | 2746\ngames started | 2734 | 2704\nbatting average | .310 | .296\nhome runs | 260 | 703\nruns batted in | 1311 | 2218"}]}
-{"id": "user_oriented_task_147", "motivation_app": "(Wolfram alpha)?", "instruction": "You will be tested on your knowledge of classic witticisms and aphorisms by completing the given aphorism. Write the original quote as the answer.", "instances": [{"input": "two things are infinite", "output": "The universe and human stupidity; and I'm not sure about the universe. (attributed to Albert Einstein)"}]}
-{"id": "user_oriented_task_148", "motivation_app": "IMDB", "instruction": "A list of all movies that meet the criteria given should be compiled.", "instances": [{"input": "movies directed by Spike Lee by release date", "output": "| | release date\n1 | American Utopia | 10/09/2020\n2 | Da 5 Bloods | 12/06/2020\n3 | BlacKkKlansman | 10/08/2018\n4 | Pass Over | 20/04/2018\n5 | Rodney King | 28/04/2017"}]}
-{"id": "user_oriented_task_149", "motivation_app": "(Wolfram alpha)?", "instruction": "Using a given amount, determine an appropriate tip.", "instances": [{"input": "14% tip on $47.50", "output": "amount of tip = $6.65 \namount with tip = $54.15"}]}
-{"id": "user_oriented_task_150", "motivation_app": "Wordle", "instruction": "Identify all words that match the pattern given.", "instances": [{"input": "H_AR_", "output": "haart, heard, hears, heart, hoard, hoary"}]}
-{"id": "user_oriented_task_151", "motivation_app": "merriam-webster.com", "instruction": "Come up with words that rhyme with the given word.", "instances": [{"input": "instruct", "output": "abduct, bucked, chucked, conduct, construct, deconstruct, deduct, destruct, ducked, duct, induct, obstruct, plucked, reconstruct, shucked, sucked, trucked, tucked"}]}
-{"id": "user_oriented_task_152", "motivation_app": "(Wolfram alpha)?", "instruction": "Look for poems that mention the given object or character. The names of the poems and their poets should be written.", "instances": [{"input": "santa claus", "output": "Santeclaus by William Gilley\nA Visit From St. Nicholas by Clement Clarke Moore"}]}
-{"id": "user_oriented_task_153", "motivation_app": "Messenger", "instruction": "Respond to the last text in the chat using the correct emojis to convey your feelings.", "instances": [{"input": "+ Hey, how did your exams go?\n- They were alright. I'm just glad they're over.\n+ Yeah, I know how you feel. I'm just glad I don't have to worry about them anymore.", "output": "- \ud83d\ude0c\ud83d\ude0a"}]}
-{"id": "user_oriented_task_154", "motivation_app": "Messenger", "instruction": "For answering the chat, think of a funny response you can use.", "instances": [{"input": "+ how are you?", "output": "- I still haven\u2019t figured it out."}]}
-{"id": "user_oriented_task_155", "motivation_app": "Messenger", "instruction": "The last message in the chat is a partial response. You should complete and rewrite it.", "instances": [{"input": "+ What are going to do now?\n- I don't know. I had to give up my dream of being a mathematician.\n+ Why", "output": "+ Why is that? What happened?"}]}
-{"id": "user_oriented_task_156", "motivation_app": "Messenger", "instruction": "Change the response to have a more empathic tone in the chat.", "instances": [{"input": "+ How are you doing?\n- Fine, I had a bunch of problems today.\n+ What kind of problems?", "output": "+ Oh, Is there anything I can help you with?"}]}
-{"id": "user_oriented_task_157", "motivation_app": "Messenger", "instruction": "Currently, there is a chat in which people are arranging a meeting in the future. Retrieve the meeting information from the conversation.", "instances": [{"input": "Person 1: Hey, when are you free? I'd like to meet up.\nPerson 2: I'm free tomorrow afternoon.\nPerson 1: Great, what time?\nPerson 2: How about 3 pm?\nPerson 1: I have another meeting at that time. what about 5 pm?\nPerson 2: That works. Where should we meet?\nPerson 1: How about the coffee shop on Main Street?\nPerson 2: Sounds good. I've heard they have great coffee there. See you tomorrow!", "output": "Date: Tomorrow\nTime: 5 pm\nPlace: Coffee shop on Main Street"}]}
-{"id": "user_oriented_task_158", "motivation_app": "Messenger", "instruction": "The given text must be classified as offensive or not offensive.", "instances": [{"input": "You acted really crazy last night.", "output": "offensive"}]}
-{"id": "user_oriented_task_159", "motivation_app": "https://cohere.ai/", "instruction": "Give a brief summary of the intention of the dialogue that just happened.", "instances": [{"input": "Customer: Hi there, I'm looking for a new phone.\nAI: Hi! What type of phone are you looking for?\nCustomer: I'm not sure. Maybe something with a good camera?\nAI: We have a few phones with great cameras. Would you like to see some options?\nCustomer: Yeah, that would be great.", "output": "A customer wants to buy a new phone."}]}
-{"id": "user_oriented_task_160", "motivation_app": "https://cohere.ai/", "instruction": "Describe the content of the article in a brief manner.", "instances": [{"input": "A study published earlier this year by Zee and her team examined the role of light in sleep for healthy adults in their 20s. Sleeping for only one night with a dim light, such as a TV set with the sound off, raised the blood sugar and heart rate of the young people during the sleep lab experiment. An elevated heart rate at night has been shown in prior studies to be a risk factor for future heart disease and early death, while higher blood sugar levels are a sign of insulin resistance, which can ultimately lead to type 2 diabetes.", "output": "A study finds that sleeping with a light pollution may increase your health risks."}]}
-{"id": "user_oriented_task_161", "motivation_app": "https://cohere.ai/", "instruction": "Correct the transcription of an excerpt containing errors.", "instances": [{"input": "I got got charged interest on ly credit card but I paid my pull balance one day due date. I not missed a pavement year yet. Man you reverse the interest charge?", "output": "I was charged interest on my credit card but I paid my balance on the due date. I haven't missed a payment year yet. Can you reverse the interest charge?"}]}
-{"id": "user_oriented_task_162", "motivation_app": "https://cohere.ai/", "instruction": "Take a look at the contract and extract the parties of the agreement from it.", "instances": [{"input": "As of the 13th day of December 2021, this Music Recording Agreement (\"Agreement\") is made between Good Kid, a Toronto-based musical group (\"Artist\"), and Universal Music Group, a record label with license number 545345 (\"Record Label\"). The Artist and the Recording Label may be referred to in this Agreement individually as \"Parties\" and collectively as \"Parties.\" Work under this Agreement will begin on March 15, 2022, and will last for the duration of the Agreement.", "output": "Artist: Good Kid\nRecording Label: Universal Music Group"}]}
-{"id": "user_oriented_task_163", "motivation_app": "Gmail", "instruction": "On the basis of the subject of the email, determine whether the email should be treated as spam or not.", "instances": [{"input": "Dermatologists don't like her!", "output": "Spam"}]}
-{"id": "user_oriented_task_164", "motivation_app": "Twitter", "instruction": "Classify the tweet into one of the following three categories: event, demo, or news. Event is a happening, often social or public, that people attend or are aware of. Demo is a  public showing or presentation of something, typically to promote or advertise it. And, news is information about recent events or happenings, especially as reported by news media.", "instances": [{"input": "Using this tutorial, you can analyze and classify a user\u2019s review as either positive or negative using sentiment analysis features in an existing Next.js app. We show you how to leverage language AI into your applications. Check it out \u2193", "output": "demo"}]}
-{"id": "user_oriented_task_165", "motivation_app": "CNN News", "instruction": "Give the news title a category. Pick a category from the list of News & Buzz, Travel, Style, Arts & Culture, Politics, Tech, and Science & Health.", "instances": [{"input": "The #Banksy Exhibit in Cambridge, MA is absolutely terrific.", "output": "Arts & Culture"}]}
-{"id": "user_oriented_task_166", "motivation_app": "https://cohere.ai/", "instruction": "Classify the questions in the FAQ into Finding policy details, Change account settings, Filing a claim and viewing status, or Cancelling coverage.", "instances": [{"input": "Could you deposit money into my account rather than mailing me a physical cheque?", "output": "Change account settings"}]}
-{"id": "user_oriented_task_167", "motivation_app": "Quora", "instruction": "Choose an appealing title for your post.", "instances": [{"input": "The typical avocado is over 300 calories from the oil in it. That\u2019s the amount of calories in a large candy bar. If you get enough exercise to eat a large candy bar every day without gaining weight, it wouldn\u2019t be a problem to eat an avocado every day. Other wise you should probably eat them sparingly.", "output": "What will happen if you eat an avocado everyday?"}]}
-{"id": "user_oriented_task_168", "motivation_app": "Quora", "instruction": "Give some examples of what people usually say in the given social situation.", "instances": [{"input": "when someone arrives safely", "output": "- \u201cGlad you made it safe and sound.\u201d\n- \u201cGood to see you got here unscathed.\u201d\n- \u201cSo, bullet proof vests actually work!\u201d"}]}
-{"id": "user_oriented_task_169", "motivation_app": "Indeed", "instruction": "You will be required to write a few questions from the given company's interview section.", "instances": [{"input": "Kohl's", "output": "- Why do you want to work at kohls? \n- If you were helping a customer and another interrupted you, what would you do? \n- What\u2019s your availability? \n- What do you think is the most important aspect of working in a retail store? \n- What\u2019s a reasonable reason for not showing up at work?"}]}
-{"id": "user_oriented_task_170", "motivation_app": "Indeed", "instruction": "Based on the information provided, you need to estimate the average summary for the given job.", "instances": [{"input": "Data entry clerk in United States", "output": "$17.58 Per hour"}]}
-{"id": "user_oriented_task_171", "motivation_app": "Indeed", "instruction": "Describe the responsibilities of the given job.", "instances": [{"input": "Security Officer", "output": "A security officer maintains a safe environment on behalf of their employers. They ensure the staff, customers, visitors and business premises are secure. They patrol perimeters, use surveillance equipment to monitor activity and perform security checks required for visitors to the site."}]}
-{"id": "user_oriented_task_172", "motivation_app": "Indeed", "instruction": "Provide a list of the skills that may help you find the job.", "instances": [{"input": "Crew Members", "output": "Baking\nBasic math\nBilingual\nCash handling\nCashiering\nCommunication skills\nComputer literacy\nComputer skills\nConstruction\nCooking\nCustomer service"}]}
-{"id": "user_oriented_task_173", "motivation_app": "Google Calendar", "instruction": "Schedule the meetings based on the given informqation.", "instances": [{"input": "Person1: available from 9-11 am, meeting duration: 30 min \nPerson2: available from 1-3 pm, meeting duration: 90 min\nPerson3: available from 2-3:30 pm, meeting duration: 60 min", "output": "Person1: 9-9:30 am, Person2: 1-2:30 pm, Person3: 2:30-3:30 pm"}]}
-{"id": "user_oriented_task_174", "motivation_app": "Google Map", "instruction": "Find the directions between the given locations.", "instances": [{"input": "From: Rowan Avenue Elementary School, 600 S Rowan Ave, Los Angeles, CA 90023, United States\nTo: Calvary Cemetery and Mortuary, 4201 Whittier Blvd, Los Angeles, CA 90023, United States", "output": "Head north on S Rowan Ave toward E 6th St. Turn right onto Whittier Blvd. Destination will be on the right."}]}
-{"id": "user_oriented_task_175", "motivation_app": "Explainpaper", "instruction": "Give a brief explanation of the requested part of the paper.", "instances": [{"input": "Paper: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.\nExplain: The dominant sequence transduction models", "output": "The dominant sequence transduction models are the most common models used to translate between two languages. These models are based on complex neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism."}]}
-{"id": "user_oriented_task_176", "motivation_app": "Explainpaper", "instruction": "Answer the question about the paper after reading it carefully.", "instances": [{"input": "To evaluate if the Transformer can generalize to other tasks we performed experiments on English constituency parsing. This task presents specific challenges: the output is subject to strong structural constraints and is significantly longer than the input. Furthermore, RNN sequence-to-sequence models have not been able to attain state-of-the-art results in small-data regimes.\nQuestion: What is English constituency parsing?", "output": "English constituency parsing is a task in natural language processing where the goal is to analyze the syntactic structure of a sentence in English. This can be used to understand the meaning of a sentence, or to generate new sentences with similar meaning."}]}
-{"id": "user_oriented_task_177", "motivation_app": "Meetup", "instruction": "Recommend some helpful activities or conversation starters to use at meetups to break the ice.", "instances": [{"input": "", "output": "- \u201cLet me tell you something very interesting\u2026\u201d (nature, history, ecology, language ,geography etc)\n- Joke (neutral not abrasive)\n- ask for help with something\n- ask open ended questions"}]}
-{"id": "user_oriented_task_178", "motivation_app": "Markdown", "instruction": "Convert the given HTML into Markdown format.", "instances": [{"input": "<code>Use `code` in your Markdown file.</code>", "output": "``Use `code` in your Markdown file.``"}]}
-{"id": "user_oriented_task_179", "motivation_app": "Semantic Scholar", "instruction": "Extract the method that has been used in the research from the abstract of the paper. Give the exact sentences from the abstract that discuss the method.", "instances": [{"input": "We present a state-of-the-art speech recognition system developed using end-to-end deep learning. Our architecture is significantly simpler than traditional speech systems, which rely on laboriously engineered processing pipelines; these traditional systems also tend to perform poorly when used in noisy environments. In contrast, our system does not need hand-designed components to model background noise, reverberation, or speaker variation, but instead directly learns a function that is robust to such effects. We do not need a phoneme dictionary, nor even the concept of a \"phoneme.\" Key to our approach is a well-optimized RNN training system that uses multiple GPUs, as well as a set of novel data synthesis techniques that allow us to efficiently obtain a large amount of varied data for training. Our system, called Deep Speech, outperforms previously published results on the widely studied Switchboard Hub5'00, achieving 16.0% error on the full test set. Deep Speech also handles challenging noisy environments better than widely used, state-of-the-art commercial speech systems.Collapse", "output": "We do not need a phoneme dictionary, nor even the concept of a \"phoneme.\" Key to our approach is a well-optimized RNN training system that uses multiple GPUs, as well as a set of novel data synthesis techniques that allow us to efficiently obtain a large amount of varied data for training."}]}
-{"id": "user_oriented_task_180", "motivation_app": "Semantic Scholar", "instruction": "You need to come up with the names of some of the papers that relate to the paper you are given.", "instances": [{"input": "RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering", "output": "+ Question Answering Infused Pre-training of General-Purpose Contextualized Representations\n+ Zero-Shot Dialogue State Tracking via Cross-Task Transfer\n+ Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections\n+ HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering\n+ FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models\n+ MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering\n+ FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark\n+ Multilingual Transfer Learning for QA Using Translation as Data Augmentation\n+ Towards Zero Shot Conditional Summarization with Adaptive Multi-task Fine-Tuning"}]}
-{"id": "user_oriented_task_181", "motivation_app": "Semantic Scholar", "instruction": "Create a short, concise summary of the paper based on its abstract.", "instances": [{"input": "Few-shot learning (FSL) is one of the key future steps in machine learning and raises a lot of attention. In this paper, we focus on the FSL problem of dialogue understanding, which contains two closely related tasks: intent detection and slot filling. Dialogue understanding has been proven to benefit a lot from jointly learning the two sub-tasks. However, such joint learning becomes challenging in the few-shot scenarios: on the one hand, the sparsity of samples greatly magnifies the difficulty of modeling the connection between the two tasks; on the other hand, how to jointly learn multiple tasks in the few-shot setting is still less investigated. In response to this, we introduce FewJoint, the first FSL benchmark for joint dialogue understanding. FewJoint provides a new corpus with 59 different dialogue domains from real industrial API and a code platform to ease FSL experiment set-up, which are expected to advance the research of this field. Further, we find that insufficient performance of the few-shot setting often leads to noisy sharing between two sub-task and disturbs joint learning. To tackle this, we guide slot with explicit intent information and propose a novel trust gating mechanism that blocks low-confidence intent information to ensure high quality sharing. Besides, we introduce a Reptile-based meta-learning strategy to achieve better generalization in unseen few-shot domains. In the experiments, the proposed method brings significant improvements on two datasets and achieve new state-of-the-art performance.", "output": "This paper introduces FewJoint, the first FSL benchmark for joint dialogue understanding, and guides slot with explicit intent information and proposes a novel trust gating mechanism that blocks low-confidence intent information to ensure high quality sharing."}]}
-{"id": "user_oriented_task_182", "motivation_app": "https://abcnotation.com/", "instruction": "You will need to compose the ABC notation for a given song.", "instances": [{"input": "The South Wind", "output": "X:435\nT:Southwind\nR:WALTZ\nM:3/4\nL:1/4\nQ:1/4=68\nK:G\n|:d/c/|\"G\" B3/2 A/2 G|Bcd|\"D\" (A3 |A2) d/c/|\"G\"B3/2 A/2 G|\"C\" EDE|\"G\"(G3|G2) d/c/|!\n|\"G\" B3/2 A/2 G|Bcd|\"D\" A3/2 (3G/F/G/|\"D\"  A2 d/c/| \"G\" B3/2 A/ G| \"C\" EDE | \"G\" (G3|G) B d|!\n\"G\" g2 g| \"C\" g3/2 f/ e| \"G\" (d3|\"D\"d2) d/c/|\"G\" B3/2 A/ G| B c d| \"D\" (A3|A2) d| \"C\" g2 g|!\n\"C\" g3/2  f/ e| \"G\" d3| \"G\" d2 c| \"G\" B3/2 A/ G| \"D\" A/ c3/2 F| \"G\" (G3|G2):||"}]}
-{"id": "user_oriented_task_183", "motivation_app": "Scribd", "instruction": "Categorize the given magazine based on its content into one of the categories: Politics, Entertainment News, Tech News, Business News, Parenting & Kids, Health & Healing, Cooking, Food & Wine, and Arts & Literature,", "instances": [{"input": "Inc.", "output": "Business News"}]}
-{"id": "user_oriented_task_184", "motivation_app": "Gmail", "instruction": "Decide if the given email belongs to the Promotions or Social category.", "instances": [{"input": "Subject: You have an invitation\nEmail: Hi, I'd like to join your LinkedIn network.", "output": "Social"}]}
-{"id": "user_oriented_task_185", "motivation_app": "Netflix", "instruction": "The story of a movie has been described using many emojis. You need to guess the name of the movie based on the emojis.", "instances": [{"input": "\ud83e\uddd9\u200d\u2642\ufe0f\ud83d\udeaa\ud83d\udcda\ud83e\udd85\u26a1", "output": "Harry Potter"}]}
-{"id": "user_oriented_task_186", "motivation_app": "Netflix", "instruction": "Come up with an interesting idea for a new movie plot. Your plot should be described with a title and a summary.", "instances": [{"input": "", "output": "The Cat burglars. \nA group of criminals specializes in stealing from high-end residences. They use their knowledge of cats to help them break in and avoid detection. However, one member of the group is a police officer working undercover to bring them down."}]}
-{"id": "user_oriented_task_187", "motivation_app": "Grammarly", "instruction": "Change the first person to the third person in the given sentence. The meaning should be kept, but you can paraphrase it or expand it in order to have a better pose.", "instances": [{"input": "We were recently able to increase the amount of stock we hold with the same supplier thereby reducing our risk.", "output": "The company was recently able to increase the amount of stock it holds with the same supplier thereby reducing its risk."}]}
-{"id": "user_oriented_task_188", "motivation_app": "Grammarly", "instruction": "You should capitalize the sentence according to the guide.", "instances": [{"input": "Guide: Every other letter alternates between lower case and upper case.\nSentence: A giant spider blocks your path.", "output": "a gIaNt SpIdEr BlOcKs YoUr PaTh."}]}
-{"id": "user_oriented_task_189", "motivation_app": "merriam-webster.com", "instruction": "Create alliterations by finding synonyms for words in the given sentence.", "instances": [{"input": "David wears a hat everyday.", "output": "David dons a derby daily."}]}
-{"id": "user_oriented_task_190", "motivation_app": "GeeksforGeeks", "instruction": "Convert the given description to a bash command.", "instances": [{"input": "read the first 10 lines from f", "output": "f.readlines(10)"}]}
-{"id": "user_oriented_task_191", "motivation_app": "Notion", "instruction": "Please prepare a table containing a summary of the information of the participants at the conference.", "instances": [{"input": "The conference was called to discuss how to improve the quality of education in schools. The main speaker was Professor Crew Purdie (cpurdie@email.com), who spoke about the importance of setting high standards for students and teachers. He said that schools should be places where students can learn and be challenged. Other speakers included Teacher Oliver Malachi (oliver@email.com) and Parent Kolby Reese (kolbyreese82@email.com). The teacher talked about the need for teachers to be well-prepared and to have high expectations for their students. The parent spoke about the need for parents to be involved in their children's education. There was also a discussion about the importance of funding for schools.", "output": "Name | Email | Occupation\nCrew Purdie | cpurdie@email.com | Professor\nOliver Malachi | oliver@email.com | Teacher\nKolby Reese | kolbyreese82@email.com | Parent"}]}
-{"id": "user_oriented_task_192", "motivation_app": "Coursera", "instruction": "List the concepts that should be learned before approaching the given complex concept.", "instances": [{"input": "Deep Learning", "output": "- Linear Algebra\n- Calculus\n- Statistics\n- Probability\n- Programming\n- Machine Learning"}]}
-{"id": "user_oriented_task_193", "motivation_app": "Grammarly", "instruction": "Given a sentence that may be ambiguous, rewrite the sentence using less ambiguous words.", "instances": [{"input": "The chicken is ready to eat.", "output": "The chicken is going to eat."}]}
-{"id": "user_oriented_task_194", "motivation_app": "Twitter", "instruction": "Assess the sentiment of the given tweet and categorize it as Positive, Negative, or Neutral.", "instances": [{"input": "\"This is the best cheat-sheet paper that I've found in a LONG time.\"", "output": "Positive"}]}
-{"id": "user_oriented_task_195", "motivation_app": "GeeksforGeeks", "instruction": "You are given a Python command and are required to convert it into Ruby.", "instances": [{"input": "print(\"Hello World!\")", "output": "puts \"Hello World!\""}]}
-{"id": "user_oriented_task_196", "motivation_app": "StackOverflow", "instruction": "You are given a sentence in first-order logic. Translate it into natural language.", "instances": [{"input": "\u2203x\u2200y(soldier(x)\u2192general(x,y))", "output": "There is someone (x) such that if if he is a soldier, then he is general of everyone."}]}
-{"id": "user_oriented_task_197", "motivation_app": "Google Search", "instruction": "You are given a search query and a document. Classify whether the document is relevant to the search query or not relevant.", "instances": [{"input": "Search: why sky is blue\nDocument: The Short Answer: Sunlight reaches Earth's atmosphere and is scattered in all directions by all the gases and particles in the air. Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", "output": "relevant"}]}
-{"id": "user_oriented_task_198", "motivation_app": "Quora", "instruction": "Create a list of subtopics for the given topic.", "instances": [{"input": "Music theory", "output": "melody, rhythm, counterpoint, harmony, form, tonal systems, scales, tuning, intervals, consonance, dissonance, durational proportions, the acoustics of pitch systems, composition, performance, orchestration, ornamentation, improvisation, electronic sound production"}]}
-{"id": "user_oriented_task_199", "motivation_app": "Netflix", "instruction": "Summarize the movie in a snarky way. Try to explain the movie in just one sentence.", "instances": [{"input": "The Shining", "output": "A family's first Airbnb experience goes very wrong."}]}
-{"id": "user_oriented_task_200", "motivation_app": "merriam-webster.com", "instruction": "Enter the words that satisfy the given condition.", "instances": [{"input": "5 Countries that Start with S", "output": "Switzerland, Spain, Senegal, Sweden, Sudan"}]}
-{"id": "user_oriented_task_201", "motivation_app": "Tasty", "instruction": "Provide a name for the dish given the ingredients and instructions.", "instances": [{"input": "INGREDIENTS:\n2 (5 oz) cans Bumble Bee\u00ae Solid White Albacore Tuna, drained\n1 avocado\n2 Tbsp Sriracha\n1 Tbsp Dijon mustard\n2 to 3 Tbsp celery, chopped\n2 Tbsp red onion, chopped\n2 green onions, chopped\n1 Tbsp fresh cilantro, chopped\nSalt and pepper, to taste\n2 heaping cups leafy green lettuce\n1 cup matchstick carrots\n4 (10 inch) whole wheat tortillas\nINSTRUCTIONS:\nIn a medium bowl, mash together tuna and avocado until combined. Add in the rest of the ingredients through the salt and pepper, mixing well.\nTo assemble, top each tortilla with a 1/2 cup leafy greens, 1/4 cup matchstick carrots and divide the tuna mixture evenly among the wraps. Tightly roll up the tortilla, slice and enjoy!", "output": "Tuna & Avocado Wrap"}]}
-{"id": "user_oriented_task_202", "motivation_app": "Google Meet", "instruction": "Summarize a meeting from the given list of bullet points. Be sure to convert shorthand into a first-hand account.", "instances": [{"input": "Rose: Analyze data and presents findings\nJohn: propose new idea\nJane: appointed to head project\nTom: need more time to fix software bug", "output": "At the meeting, Rose analyzed data and presented her findings. Then, John proposed a new idea. Moreover, Jane was appointed to head the project. And, Tom said he needs more time to fix a software bug."}]}
-{"id": "user_oriented_task_203", "motivation_app": "Spotify", "instruction": "List a few popular songs from the given album.", "instances": [{"input": "Back To Black", "output": "- Rehab\n- Just Friends\n- Tears Dry on Their Own\n- Wake Up Alone\n- Addicted"}]}
-{"id": "user_oriented_task_204", "motivation_app": "merriam-webster.com", "instruction": "Invent a new word based on a description of the word.", "instances": [{"input": "The feeling of knowing, while you are still in the moment, that something occurring will be remembered forever.", "output": "mnemonty"}]}
-{"id": "user_oriented_task_205", "motivation_app": "w3schools", "instruction": "Given a programming language and the name of a function, write a command to show how to use the function.", "instances": [{"input": "Language: Python\nFunction: input", "output": "username = input(\"Enter username:\")"}]}
-{"id": "user_oriented_task_206", "motivation_app": "Reddit", "instruction": "Provide a pleasant compliment based on the quality you have been given.", "instances": [{"input": "Truthfulness", "output": "Everything would be better if more people were like you."}]}
-{"id": "user_oriented_task_207", "motivation_app": "Overleaf", "instruction": "Create a brief resume introduction using the following information.", "instances": [{"input": "name: Mark\nnationality: Italian\nlanguage: English\nattributes: creative, goal-oriented, motivated\nskills: software engineering, content marketing", "output": "Mark is a creative and goal-oriented software engineer and content marketer. He has a strong background in software engineering and is skilled in content marketing. He is a motivated individual who always looks for new opportunities to learn and grow."}]}
-{"id": "user_oriented_task_208", "motivation_app": "Amazon", "instruction": "Make a list of adjectives that can be used to describe the given brand.", "instances": [{"input": "a creative tech startup", "output": "1. Innovative\n2. Creative\n3. Disruptive\n4. cutting-edge\n5. revolutionary\n6. game-changing\n7. industry-leading\n8. trailblazing\n9. groundbreaking\n10. world-changing"}]}
-{"id": "user_oriented_task_209", "motivation_app": "tripadvisor.com", "instruction": "Someone gives you some information about where they want to go and what they enjoy seeing and doing. Describe a proper road trip for them and find the best stops and sctivities in the way.", "instances": [{"input": "I'm looking for a west coast road trip. My goal is to start near San Diego and travel to Los Angeles.", "output": "Begin near the city of San Diego and make the first stop in coastal La Jolla. This upscale little village is a great a place for an overnight, a day of sunbathing at La Jolla Shores Beach, or a cocktail with a view.\nJust 20 minutes north, stop into beachy Encinitas to window shop the boutiques or sample fish tacos. Walk through the coastal meditation garden at the Self-Realization Fellowship or join surfers in the line-up at one of the fantastic local beaches here\nJust up the coast is cute little San Juan Capistrano, home to one of California\u2019s favorite missions. In spring, more than 6,000 swallows migrate here from Argentina! As you approach LA, don\u2019t miss swank Laguna Hills. Home to the one-of-a-kind Pageant of the Masters art event, Laguna is brimming with artists and charm. It\u2019s a great place for an overnight. Plus, Crystal Cove Beach in nearby Newport Beach is one of California\u2019s finest!\nAfter that, the choice is yours: Disney lovers should head directly to Disneyland in Anaheim to meet Mickey. To join the beautiful people and snap a photo of an iconic California scene, head to Santa Monica Pier and walk the boardwalk or ride the world\u2019s only solar-powered Ferris wheel. A San Diego to LA road trip is the perfect way to enjoy all things Southern California."}]}
-{"id": "user_oriented_task_210", "motivation_app": "StackOverflow", "instruction": "Regex is a set of characters that define a search pattern. This kind of pattern is usually used by string-searching algorithms to find or replace strings or to validate inputs. Convert the natural english text to regex.", "instances": [{"input": "Match a valid ip address", "output": "^(?:(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])$"}]}
-{"id": "user_oriented_task_211", "motivation_app": "sth related to real estate?", "instruction": "Write an engaging and well-written property listing description for selling a house. Address of the house and some of the details are given to you. Fill in the information gap with hallucinations if needed.", "instances": [{"input": "Property Address: 412 Monterey Ave, Capitola, CA 95010\nParking: 3 cars + a finished single car garage\nDetails:\n- open floorplan\n- window with views to park/mountains\n- Kitchen with shaker cabinetry and white Calcatta quartz counters", "output": "Amazing designer updates to this beach chich home that is just a quick 3-minute stroll to Capitola Village! As you enter this light-filled home you will love the open floorplan and huge picture window with views to park/mountains! Kitchen with shaker cabinetry and white Calcatta quartz counters new stainless appliances (gas stove). Master bedroom with own access to yard! Beautiful bathrooms w/designer finishes and custom, subway & picket tile! New composition roof and gutters, new indoor and outdoor paint, new dual pane windows, new wide plank washed oak laminate floors, new doors, new fencing, forced air heat + LOTS more! Home sits high above Monterey Ave with alley access and parking for 3 cars + a finished single car garage! The indoor/outdoor living is ideal from this sunny locale with front, side and backyards that are super sunny with pathways and perfect for gardening or entertaining! Stroll to beach, Gayles bakery, parks, schools & churches from this adorable beach home!"}]}
-{"id": "user_oriented_task_212", "motivation_app": "Redfin", "instruction": "Given a real estate listing description, extract the information and details of the property from the text. The details consist of address, basic house attributes, etc.", "instances": [{"input": "WALK TO THE BEACH FROM CAPITOLA KNOLLS! Single-level upstairs condo with your own large private deck overlooking nature. Adorable home that is move-in ready and waiting for you to put your own personal touches on. Gorgeous and well-maintained landscaped park-like setting with lush green grasses with a community pool and spa to enjoy. Desirable location...walk to Capitola Village, Gayle's Bakery, restaurants, shopping, schools or jump on Hwy 1 right around the corner.", "output": "-Address: \"Capitola\"\n-Type of home: Condo\n-Location: \"walk to Capitola Village, Gayle's Bakery, restaurants, shopping, and schools or jump on Hwy 1 right around the corner.\"\n-House attributes: \"Single-level upstairs, large private deck overlooking nature, move-in ready home\"\n-Outdoor details: \"well-maintained landscaped park-like setting with lush green grasses with a Community pool and spa\""}]}
-{"id": "user_oriented_task_213", "motivation_app": "Redfin", "instruction": "Review the property description and tell us what is good and what is bad about it and list its key takeaways.", "instances": [{"input": "There's real magic in combining inherently compelling Victorian architecture with a modern aesthetic. Add in an extraordinary multi-level garden & the result is the undeniable elegance of 444 29th St. The front of the home is light-filled w/ a seamless connection between living, dining & kitchen. Newly updated kitchen w/ quartz counters & upgraded appliances. A lg dining rm has a bay of windows &, when paired w/ the adjacent living rm, provides an ideal space to entertain. 2 lg bdrms on this level are at the rear. Both w/ lg closets & walk out to the garden. Stylishly remodeled full bath on this level. 3rd bdrm/2nd full bath are located on the ground floor - perfect as a guest rm, home office, fitness area, etc. The enormous garden occupies an extra deep lot & is beautifully landscaped & terraced. Laundry/storage rm, 1-car garage pkg, Tesla solar panels. Prime, flat part of Noe Valley - short walk to all that Noe has to offer. Steps from the J-Church & convenient to shuttles/freeways.", "output": "1- Really engaging opening hook \u2013 very unique wording used here that captures the buyers' attention\n2- Good use of search engine optimization (SEO) keywords, for example using the address within the description, as the address is often what a buyer will search\n3- Use of too many short forms may make it harder for buyers to understand and read. It\u2019s often better to improve the quality of the content vs maximizing the quantity \u2013 in this case it may have made more sense to remove some pieces and reduce the use of acronyms"}]}
-{"id": "user_oriented_task_214", "motivation_app": "Redfin", "instruction": "List some of the top real estate marketing words to add value to the listing and engage more potential buyers.", "instances": [{"input": "", "output": "1. Luxurious\n2. New\n3. Spacious\n4. Captivating\n5. Move-In-Ready\n6. Landscaped\n7. Beautiful\n8. Character\n9. Custom\n10. Pet-Friendly"}]}
-{"id": "user_oriented_task_215", "motivation_app": "Amazon", "instruction": "Suggest some product bundles that are often purchased with a given product.", "instances": [{"input": "MacBook Air", "output": "MacBook Air, MacBook Air sleeve, MacBook Air case, MacBook Air external drive"}]}
-{"id": "user_oriented_task_216", "motivation_app": "Jira", "instruction": "Create a template in markdown to create scope for Jira tickets. Members should use this template as a checklist to ensure they have included all the necessary information when creating a ticket.", "instances": [{"input": "", "output": "# TO SCOPE THIS TICKET:\n[] create a descriptive title\n[] complete TBDs below\n[] assign epic to this ticket\n[] create relevant linking relations\n[] delete this section\n# DELETE ME AND ABOVE!\n\n## \ud83e\uddd1 Story\nWe need to TBDTASK from TBDRESOURCE in order for TBDUSER to TBDACTION\n\n## \ud83d\udd28 Acceptance Criteria\n- AC1\n- AC2\n\n## \ud83d\udcda Resources\n- [PLANNINGDOC1](WWWDOTEXAMPLEDOTCOM)\n- [FIGMADOC1](WWWDOTEXAMPLEDOTCOM)\n- [TECHDOCUMENTATION1](WWWDOTEXAMPLEDOTCOM)\n- [TECHDOCUMENTATION2](WWWDOTEXAMPLEDOTCOM)"}]}
-{"id": "user_oriented_task_217", "motivation_app": "Jira", "instruction": "As part of software engineering, user stories describe what the user wants the system to do, while acceptance criteria describe the conditions a specific user story must meet. Create basic acceptance criteria based on the given user story.", "instances": [{"input": "User story: As a user, I want to use a search field to type a city, name, or street, so that I could find matching hotel options.", "output": "- The search field is placed on the top bar\n- Search starts once the user clicks \u201cSearch\u201d\n- The field contains a placeholder with a grey-colored text: \u201cWhere are you going?\u201d\n- The placeholder disappears once the user starts typing\n- Search is performed if a user types in a city, hotel name, street, or all combined\nSearch is in English, French, German, and Ukrainian\nThe user can\u2019t type more than 200 symbols\nThe search doesn\u2019t support special symbols (characters). If the user has typed a special symbol, show the warning message: \u201cSearch input cannot contain  special symbols.\u201d"}]}
-{"id": "user_oriented_task_218", "motivation_app": "Jira", "instruction": "A user story is an informal, general explanation of a software feature written from the perspective of the end user or customer. Write a user story for a given software.", "instances": [{"input": "Gmail", "output": "As a user, I want to be able to recover the password to my account, so that I will be able to access my account in case I forgot the password."}]}
-{"id": "user_oriented_task_219", "motivation_app": "Jira", "instruction": "Write a Jira ticket for the given task.", "instances": [{"input": "New Employee onboarding", "output": "Title: Please help our new employee get settled.\nPrepare a desk, chair, and computer for them to use.\nCreate their company account, email, and temporary password.\nGrant access to company documents."}]}
-{"id": "user_oriented_task_220", "motivation_app": "Google Docs", "instruction": "Paraphrase the given text in an academic style.", "instances": [{"input": "Lots of papers have been published on this topic.", "output": "A considerable amount of literature has been published on this topic."}]}
-{"id": "user_oriented_task_221", "motivation_app": "Google Sheet", "instruction": "Make a list of the pros and cons of the given decision.", "instances": [{"input": "Implementing a Remote Working Policy", "output": "Pros: \n- Allowing employees to work from home can reduce their stressful commutes and increase their productivity. \n- Remote working provides greater flexibility, particularly for working parents and caregivers who have other responsibilities. \n- It's possible to hire highly creative people who live further away if they don't have to come into the office every day.\n- People who aren't distracted by office noises and regular interruptions will perform better. \n- Team members' communication skills will improve, as they will learn to communicate more effectively using email and IM, for example. \n\nCons:\n- It may be more difficult for team members to build meaningful, productive, creative relationships with one another.\n- The organization would have to budget for equipment that team members can take home.\n- The organization may not be able to keep track of how people spend their time. \n- The company could lose control of its data, because equipment and documents are removed from the office.\n- It will be harder to schedule face-to-face meetings and provide constructive feedback."}]}
-{"id": "user_oriented_task_222", "motivation_app": "Google Docs", "instruction": "Include important study notes and key points that someone should know about the given subject.", "instances": [{"input": "history of the USA", "output": "1. The United States of America was founded in 1776.\n2. The United States Constitution was ratified in 1788.\n3. The United States Declaration of Independence was signed in 1776.\n4. The United States Bill of Rights was ratified in 1791.\n5. The United States of America is a federal republic."}]}
-{"id": "user_oriented_task_223", "motivation_app": "Yelp", "instruction": "Based on the given keywords, write a review of a restaurant.", "instances": [{"input": "Name: Pizzeria\ngood prices, quiet place, tasty, nice staff", "output": "I had a great experience at Pizzeria. The prices were good, it was a quiet place, and the food was tasty. The staff was also very nice."}]}
-{"id": "user_oriented_task_224", "motivation_app": "Google Map", "instruction": "Convert natural language into a list of turn-by-turn directions.", "instances": [{"input": "Go west on I-10 until you hit the 101 then take it north. You will eventually see the Hollywood sign.", "output": "1. Go west on I-10.\n2. Take the 101 north.\n3. You will eventually see the Hollywood sign."}]}
-{"id": "user_oriented_task_225", "motivation_app": "Play Store", "instruction": "Provide an ESRB rating for the following game.", "instances": [{"input": "This is an action-adventure game in which players help Kratos and his son on a dangerous quest. Players explore various realms and engage in frenetic hand-to-hand combat against human-like raiders and fantastical creatures (e.g., centaurs, trolls, dragons). Players use axes and chained blades to battle enemies, often resulting in large blood-splatter effects and dismemberment. Players can perform finishing attacks that depict close-up impalement via bladed and hand-held weapons; repeated axe strikes to a creature's neck results in decapitation. The words \u201cf**k\u201d and \u201csh*t\u201d are heard in the game.", "output": "Mature 17+"}]}
-{"id": "user_oriented_task_226", "motivation_app": "MS Powerpoint", "instruction": "A color description has been provided. Find the CSS code associated with that color.", "instances": [{"input": "A light red color with a medium light shade of pink", "output": "#DB7093"}]}
-{"id": "user_oriented_task_227", "motivation_app": "Leetcode", "instruction": "Identify and fix bugs in the given code and rewrite it", "instances": [{"input": "for i in range(10)\n    print(Answer is:)\n    print(i)", "output": "for i in range(10):\n    print(\"Answer is:\")\n    print(i)"}]}
-{"id": "user_oriented_task_228", "motivation_app": "Google Docs", "instruction": "Make a list of common phrases for the given section of the paper.", "instances": [{"input": "Introduction", "output": "1- ... is an important component of ...\n2- Central to the discipline of ... is ...\n3- Recent developments in ... have shown the need for ...\n4- The study build on ... and contributes to ...\n5- Until now, has only been applied to ..."}]}
-{"id": "user_oriented_task_229", "motivation_app": "Netflix", "instruction": "We have described a scene from a movie that we have seen in the past. Find out what movie it is.", "instances": [{"input": "A choice has to be made between picking the red pill or the blue pill", "output": "Matrix"}]}
-{"id": "user_oriented_task_230", "motivation_app": "Facebook", "instruction": "Take the product description and write a creative ad for it.", "instances": [{"input": "Document Cameras allow teachers to display documents, books, or other materials for their students to see.", "output": "Looking for a way to make your lessons more engaging? Check out our document cameras! With our cameras, you can display documents, books, or other materials for your students to see. This is a great way to add more visual elements to your lessons and make them more exciting for your students."}]}
-{"id": "user_oriented_task_231", "motivation_app": "Google Sheets", "instruction": "Translate the natural language description into a Google Sheets formula.", "instances": [{"input": "if C7 is equal to \"U.S.A.\" or \"Canada\" or \"Europe\" or \"The Americas\"", "output": "IF (OR(C7 = \"U.S.A.\"; C7 = \"Canada\"); \"The Americas\"; \"Europe\")"}]}
-{"id": "user_oriented_task_232", "motivation_app": "Twitter", "instruction": "You are given a tweet and you should decide whether it's offensive or not.", "instances": [{"input": "She has certainly taken some heat for being such an....well idiot.", "output": "Offensive"}]}
-{"id": "user_oriented_task_233", "motivation_app": "Workout", "instruction": "For a quick and efficient office workout, suggest a short routine.", "instances": [{"input": "", "output": "Desk pushups: 20 reps.\nSeated knee-to-chest: 10 reps.\nSquats: 10 reps.\nDesk plank: 30 seconds.\nSeated flutter kicks: 10 reps on each side.\nRomanian deadlifts: 10 reps.\nJumping jacks: 20 reps."}]}
-{"id": "user_oriented_task_234", "motivation_app": "Yelp", "instruction": "Predict how many stars the author will give to the restaurant from a Yelp review.", "instances": [{"input": "The reviews were great, but honestly i felt it was just ok. Seemed like a typical tourist spot were the food was mediocre. The service was ok, not great, but not the worst.\nThe bread was very good. I ordered the muscle fries. They had a sweet Vidalia onion flavor. No need for fries bc they were soggy wo much flavor. My boyfriend ordered a pesto chicken pasta and it was again mediocre.", "output": "3 stars"}]}
-{"id": "user_oriented_task_235", "motivation_app": "Yelp", "instruction": "Categorize the Business into one of the Restaurants, Home Services, Auto Services and Miscellaneous based on its Specialties.", "instances": [{"input": "Call us at 650-636-4884 or visit our website to receive a quote. This shop specializes in New Tires and General Auto Repair. We carry all tires in-house and have a wide range to fit any budget or car specifics. If you are unsure what tires you need there are experts on hand and available to help you pick the best tire for your needs. We also carry commercial vehicle tires and can serve a wide array of fleets.", "output": "Auto Services"}]}
-{"id": "user_oriented_task_236", "motivation_app": "Grammarly", "instruction": "Notify me of any suggestions you may have for making the text more grammatically correct.", "instances": [{"input": "This is the second time I've been here, and the vibes, and the food were exquisite.", "output": "It appears that you have an unnecessary comma in a compound subject. Consider removing it."}]}
-{"id": "user_oriented_task_237", "motivation_app": "Tasty", "instruction": "Using the ingredients that are available, create a dinner recipe.", "instances": [{"input": "Kale + almond butter + apple cider vinegar + agave nectar + tamari", "output": "Gently pat the kale leaves dry.\nDe-stem the kale and tear into pieces of desired size. Add the kale to a large bowl.\nAdd the almond butter, apple cider vinegar, liquid aminos, and agave nectar to a small bowl and whisk together for 30 seconds.\nPour the dressing over the kale leaves, roll up your sleeves, and massage the kale with your hands for 2-3 minutes. The more you massage the kale, the more tender and flavorful the leaves will become.*\nTransfer the kale to one large or two small salad bowls and top with desired garnishes."}]}
-{"id": "user_oriented_task_238", "motivation_app": "Amazon", "instruction": "Categorize the given product into one of Electronics, Computers, Smart Home, or Arts & Crafts departments.", "instances": [{"input": "Google Nest Learning Thermostat", "output": "Smart Home"}]}
-{"id": "user_oriented_task_239", "motivation_app": "Workout", "instruction": "Provide instructions for the given exercise.", "instances": [{"input": "Leg Raises", "output": "To start toning your abs by hitting up the lower abdominal muscles, Riggins suggests 30 seconds of leg raises. Here's how: Turn on your back with legs straight and your feet and ankles together. Raise your legs up and down in a vertical position from your body, while keeping your belly button. Slowly bring legs back down, but if that is too difficult, tuck your knees. (And be careful not to strain your lower back!)"}]}
-{"id": "user_oriented_task_240", "motivation_app": "Grammarly", "instruction": "Rewrite the text and correct the spelling errors.", "instances": [{"input": "It solves problems comon and uniqe to every team.", "output": "It solves problems common and unique to every team."}]}
-{"id": "user_oriented_task_241", "motivation_app": "merriam-webster.com", "instruction": "Write down antonyms for the given word.", "instances": [{"input": "laureating", "output": "discrediting, humiliating, dishonoring, shaming"}]}
-{"id": "user_oriented_task_242", "motivation_app": "w3schools", "instruction": "Write the HTML syntax for the required action.", "instances": [{"input": "adding an image to the page", "output": "<img src=\"url\" alt=\"alternatetext\">"}]}
-{"id": "user_oriented_task_243", "motivation_app": "w3schools", "instruction": "Identify the programming language used to write the given code.", "instances": [{"input": "if (20 > 18) {\n  printf(\"20 is greater than 18\");\n}", "output": "C"}]}
-{"id": "user_oriented_task_244", "motivation_app": "merriam-webster.com", "instruction": "Decide which part of speech the underlined word belongs to.", "instances": [{"input": "school will _keep through the winter", "output": "verb"}]}
-{"id": "user_oriented_task_245", "motivation_app": "merriam-webster.com", "instruction": "Define what the underlined word means for kids.", "instances": [{"input": "_keep a promise", "output": "to be faithful to : Fulfill"}]}
-{"id": "user_oriented_task_246", "motivation_app": "Grammarly", "instruction": "Expand the given sentence and give it more details and depth.", "instances": [{"input": "It changed my life.", "output": "The experience changed the course of my life forever."}]}
-{"id": "user_oriented_task_247", "motivation_app": "Airbnb", "instruction": "Extract the address of the property from the \"About this space\" section of Airbnb.", "instances": [{"input": "Tucked in the foothills of the quaint historic mining town of Grass Valley, CA this funky and spacious chalet invites you to experience the great outdoors with family, friends and pets. Enjoy the hot tub, basketball court, treehouse, kids rooms, bbq, fire-pit, outdoor theater, and more. One hour from Tahoe and 10 minutes to town, you\u2019re never short of things to explore.", "output": "Grass Valley, California, United States"}]}
-{"id": "user_oriented_task_248", "motivation_app": "Strava", "instruction": "Design a simple training challenge on Strava that everyone could attend.", "instances": [{"input": "", "output": "This month, push yourself to log 30 hours of whatever activities get you stoked to move. Carve out time for yourself and your health and get out there for less than one hour a day \u2013 or go long on the weekends to mix it up. It doesn\u2019t always have to be your most all-out effort, yoga or walks count for just as much. Commit to consistency to earn this time-intensive trophy."}]}
-{"id": "user_oriented_task_249", "motivation_app": "Telegram", "instruction": "Suggest some names for a friendly group in telegram.", "instances": [{"input": "", "output": "1. The Conversation Club \n2. The Chatterbox \n3. F.R.I.E.N.D.S \n4. The Chatty Collective \n5. The Tea Talkers \n6. The Support Squad \n7. The Cheerful Chums"}]}
-{"id": "user_oriented_task_250", "motivation_app": "Google Search", "instruction": "Write what the pronunciation of the given word sounds like. Follow the \"Google pronunciation dictionary\" scheme for phonetic spelling.", "instances": [{"input": "interpretations", "output": "in\u00b7tr\u00b7pruh\u00b7tay\u00b7shnz"}]}
-{"id": "user_oriented_task_251", "motivation_app": "Reddit", "instruction": "Explain the meaning of the given phrase in simple terms. Use an example if possible. It would be helpful if you could give an example.", "instances": [{"input": "\"With a little give in them\"", "output": "It means with some space to move. It's a metaphor. \"Give\" is a noun here. It's meaning is similar to\"flexibility\". If something has no \"give\" it's tight and rigid. It's the opposite of flexible and loose. For example, you might use it to talk about clothes: \"These jeans are so tight. There's no give in them. I can't move.\""}]}

gpt4all-backend/CMakeLists.txt

@@ -0,0 +1,189 @@
+cmake_minimum_required(VERSION 3.23)  # for FILE_SET
+
+include(../common/common.cmake)
+
+set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
+
+if (APPLE)
+    option(BUILD_UNIVERSAL "Build a Universal binary on macOS" ON)
+else()
+    option(LLMODEL_KOMPUTE "llmodel: use Kompute"              ON)
+    option(LLMODEL_VULKAN  "llmodel: use Vulkan"               OFF)
+    option(LLMODEL_CUDA    "llmodel: use CUDA"                 ON)
+    option(LLMODEL_ROCM    "llmodel: use ROCm"                 OFF)
+endif()
+
+if (APPLE)
+  if (BUILD_UNIVERSAL)
+    # Build a Universal binary on macOS
+    # This requires that the found Qt library is compiled as Universal binaries.
+    set(CMAKE_OSX_ARCHITECTURES "arm64;x86_64" CACHE STRING "" FORCE)
+  else()
+    # Build for the host architecture on macOS
+    if (NOT CMAKE_OSX_ARCHITECTURES)
+      set(CMAKE_OSX_ARCHITECTURES "${CMAKE_HOST_SYSTEM_PROCESSOR}" CACHE STRING "" FORCE)
+    endif()
+  endif()
+endif()
+
+# Include the binary directory for the generated header file
+include_directories("${CMAKE_CURRENT_BINARY_DIR}")
+
+set(LLMODEL_VERSION_MAJOR 0)
+set(LLMODEL_VERSION_MINOR 5)
+set(LLMODEL_VERSION_PATCH 0)
+set(LLMODEL_VERSION "${LLMODEL_VERSION_MAJOR}.${LLMODEL_VERSION_MINOR}.${LLMODEL_VERSION_PATCH}")
+project(llmodel VERSION ${LLMODEL_VERSION} LANGUAGES CXX C)
+
+set(CMAKE_CXX_STANDARD 23)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
+set(BUILD_SHARED_LIBS ON)
+
+# Check for IPO support
+include(CheckIPOSupported)
+check_ipo_supported(RESULT IPO_SUPPORTED OUTPUT IPO_ERROR)
+if (NOT IPO_SUPPORTED)
+    message(WARNING "Interprocedural optimization is not supported by your toolchain! This will lead to bigger file sizes and worse performance: ${IPO_ERROR}")
+else()
+    message(STATUS "Interprocedural optimization support detected")
+endif()
+
+set(DIRECTORY deps/llama.cpp-mainline)
+include(llama.cpp.cmake)
+
+set(BUILD_VARIANTS)
+if (APPLE)
+    list(APPEND BUILD_VARIANTS metal)
+endif()
+if (LLMODEL_KOMPUTE)
+    list(APPEND BUILD_VARIANTS kompute kompute-avxonly)
+else()
+    list(PREPEND BUILD_VARIANTS cpu cpu-avxonly)
+endif()
+if (LLMODEL_VULKAN)
+    list(APPEND BUILD_VARIANTS vulkan vulkan-avxonly)
+endif()
+if (LLMODEL_CUDA)
+    cmake_minimum_required(VERSION 3.18)  # for CMAKE_CUDA_ARCHITECTURES
+
+    # Defaults must be set before enable_language(CUDA).
+    # Keep this in sync with the arch list in ggml/src/CMakeLists.txt.
+    if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
+        # 52 == lowest CUDA 12 standard
+        # 60 == f16 CUDA intrinsics
+        # 61 == integer CUDA intrinsics
+        # 70 == compute capability at which unrolling a loop in mul_mat_q kernels is faster
+        if (GGML_CUDA_F16 OR GGML_CUDA_DMMV_F16)
+            set(CMAKE_CUDA_ARCHITECTURES "60;61;70;75") # needed for f16 CUDA intrinsics
+        else()
+            set(CMAKE_CUDA_ARCHITECTURES "52;61;70;75") # lowest CUDA 12 standard + lowest for integer intrinsics
+            #set(CMAKE_CUDA_ARCHITECTURES "OFF") # use this to compile much faster, but only F16 models work
+        endif()
+    endif()
+    message(STATUS "Using CUDA architectures: ${CMAKE_CUDA_ARCHITECTURES}")
+
+    include(CheckLanguage)
+    check_language(CUDA)
+    if (NOT CMAKE_CUDA_COMPILER)
+        message(WARNING "CUDA Toolkit not found. To build without CUDA, use -DLLMODEL_CUDA=OFF.")
+    endif()
+    enable_language(CUDA)
+    list(APPEND BUILD_VARIANTS cuda cuda-avxonly)
+endif()
+if (LLMODEL_ROCM)
+    enable_language(HIP)
+    list(APPEND BUILD_VARIANTS rocm rocm-avxonly)
+endif()
+
+# Go through each build variant
+foreach(BUILD_VARIANT IN LISTS BUILD_VARIANTS)
+    # Determine flags
+    if (BUILD_VARIANT MATCHES avxonly)
+        set(GPT4ALL_ALLOW_NON_AVX OFF)
+    else()
+        set(GPT4ALL_ALLOW_NON_AVX ON)
+    endif()
+    set(GGML_AVX2 ${GPT4ALL_ALLOW_NON_AVX})
+    set(GGML_F16C ${GPT4ALL_ALLOW_NON_AVX})
+    set(GGML_FMA  ${GPT4ALL_ALLOW_NON_AVX})
+
+    set(GGML_METAL   OFF)
+    set(GGML_KOMPUTE OFF)
+    set(GGML_VULKAN  OFF)
+    set(GGML_CUDA    OFF)
+    set(GGML_ROCM    OFF)
+    if (BUILD_VARIANT MATCHES metal)
+        set(GGML_METAL   ON)
+    elseif (BUILD_VARIANT MATCHES kompute)
+        set(GGML_KOMPUTE ON)
+    elseif (BUILD_VARIANT MATCHES vulkan)
+        set(GGML_VULKAN  ON)
+    elseif (BUILD_VARIANT MATCHES cuda)
+        set(GGML_CUDA    ON)
+    elseif (BUILD_VARIANT MATCHES rocm)
+        set(GGML_HIPBLAS ON)
+    endif()
+
+    # Include GGML
+    include_ggml(-mainline-${BUILD_VARIANT})
+
+    if (BUILD_VARIANT MATCHES metal)
+        set(GGML_METALLIB "${GGML_METALLIB}" PARENT_SCOPE)
+    endif()
+
+    # Function for preparing individual implementations
+    function(prepare_target TARGET_NAME BASE_LIB)
+        set(TARGET_NAME ${TARGET_NAME}-${BUILD_VARIANT})
+        message(STATUS "Configuring model implementation target ${TARGET_NAME}")
+        # Link to ggml/llama
+        target_link_libraries(${TARGET_NAME}
+            PRIVATE ${BASE_LIB}-${BUILD_VARIANT})
+        # Let it know about its build variant
+        target_compile_definitions(${TARGET_NAME}
+            PRIVATE GGML_BUILD_VARIANT="${BUILD_VARIANT}")
+        # Enable IPO if possible
+# FIXME: Doesn't work with msvc reliably. See https://github.com/nomic-ai/gpt4all/issues/841
+#        set_property(TARGET ${TARGET_NAME}
+#                     PROPERTY INTERPROCEDURAL_OPTIMIZATION ${IPO_SUPPORTED})
+    endfunction()
+
+    # Add each individual implementations
+    add_library(llamamodel-mainline-${BUILD_VARIANT} SHARED
+        src/llamamodel.cpp src/llmodel_shared.cpp)
+    gpt4all_add_warning_options(llamamodel-mainline-${BUILD_VARIANT})
+    target_compile_definitions(llamamodel-mainline-${BUILD_VARIANT} PRIVATE
+        LLAMA_VERSIONS=>=3 LLAMA_DATE=999999)
+    target_include_directories(llamamodel-mainline-${BUILD_VARIANT} PRIVATE
+        src include/gpt4all-backend
+    )
+    prepare_target(llamamodel-mainline llama-mainline)
+
+    if (NOT PROJECT_IS_TOP_LEVEL AND BUILD_VARIANT STREQUAL cuda)
+        set(CUDAToolkit_BIN_DIR ${CUDAToolkit_BIN_DIR} PARENT_SCOPE)
+    endif()
+endforeach()
+
+add_library(llmodel
+    src/dlhandle.cpp
+    src/llmodel.cpp
+    src/llmodel_c.cpp
+    src/llmodel_shared.cpp
+)
+gpt4all_add_warning_options(llmodel)
+target_sources(llmodel PUBLIC
+    FILE_SET public_headers TYPE HEADERS BASE_DIRS include
+    FILES include/gpt4all-backend/llmodel.h
+          include/gpt4all-backend/llmodel_c.h
+          include/gpt4all-backend/sysinfo.h
+)
+target_compile_definitions(llmodel PRIVATE LIB_FILE_EXT="${CMAKE_SHARED_LIBRARY_SUFFIX}")
+target_include_directories(llmodel PRIVATE src include/gpt4all-backend)
+
+set_target_properties(llmodel PROPERTIES
+                              VERSION ${PROJECT_VERSION}
+                              SOVERSION ${PROJECT_VERSION_MAJOR})
+
+set(COMPONENT_NAME_MAIN ${PROJECT_NAME})
+set(CMAKE_INSTALL_PREFIX ${CMAKE_BINARY_DIR}/install)

gpt4all-backend/README.md

@@ -0,0 +1,42 @@
+# GPT4ALL Backend
+This directory contains the C/C++ model backend used by GPT4All for inference on the CPU. This backend acts as a universal library/wrapper for all models that the GPT4All ecosystem supports. Language bindings are built on top of this universal library. The native GPT4all Chat application directly uses this library for all inference.
+
+# What models are supported by the GPT4All ecosystem?
+
+Currently, there are three different model architectures that are supported:
+
+1. GPTJ - Based off of the GPT-J architecture with examples found [here](https://huggingface.co/EleutherAI/gpt-j-6b)
+2. LLAMA - Based off of the LLAMA architecture with examples found [here](https://huggingface.co/models?sort=downloads&search=llama)
+3. MPT - Based off of Mosaic ML's MPT architecture with examples found [here](https://huggingface.co/mosaicml/mpt-7b)
+
+# Why so many different architectures? What differentiates them?
+
+One of the major differences is license. Currently, the LLAMA based models are subject to a non-commercial license, whereas the GPTJ and MPT base models allow commercial usage. In the early advent of the recent explosion of activity in open source local models, the llama models have generally been seen as performing better, but that is changing quickly. Every week - even every day! - new models are released with some of the GPTJ and MPT models competitive in performance/quality with LLAMA. What's more, there are some very nice architectural innovations with the MPT models that could lead to new performance/quality gains.
+
+# How does GPT4All make these models available for CPU inference?
+
+By leveraging the ggml library written by Georgi Gerganov and a growing community of developers. There are currently multiple different versions of this library. The original github repo can be found [here](https://github.com/ggerganov/ggml), but the developer of the library has also created a LLAMA based version [here](https://github.com/ggerganov/llama.cpp). Currently, this backend is using the latter as a submodule.
+
+# Does that mean GPT4All is compatible with all llama.cpp models and vice versa?
+
+Unfortunately, no for three reasons:
+
+1. The upstream [llama.cpp](https://github.com/ggerganov/llama.cpp) project has introduced [a compatibility breaking](https://github.com/ggerganov/llama.cpp/commit/b9fd7eee57df101d4a3e3eabc9fd6c2cb13c9ca1) re-quantization method recently. This is a breaking change that renders all previous models (including the ones that GPT4All uses) inoperative with newer versions of llama.cpp since that change.
+2. The GPT4All backend has the llama.cpp submodule specifically pinned to a version prior to this breaking change.
+3. The GPT4All backend currently supports MPT based models as an added feature. Neither llama.cpp nor the original ggml repo support this architecture as of this writing, however efforts are underway to make MPT available in the ggml repo which you can follow [here.](https://github.com/ggerganov/ggml/pull/145)
+
+# What is being done to make them more compatible?
+
+A few things. Number one, we are maintaining compatibility with our current model zoo by way of the submodule pinning. However, we are also exploring how we can update to newer versions of llama.cpp without breaking our current models. This might involve an additional magic header check or it could possibly involve keeping the currently pinned submodule and also adding a new submodule with later changes and differentiating them with namespaces or some other manner. Investigations continue.
+
+# What about GPU inference?
+
+In newer versions of llama.cpp, there has been some added support for NVIDIA GPU's for inference. We're investigating how to incorporate this into our downloadable installers.
+
+# Ok, so bottom line... how do I make my model on Hugging Face compatible with GPT4All ecosystem right now?
+
+1. Check to make sure the Hugging Face model is available in one of our three supported architectures
+2. If it is, then you can use the conversion script inside of our pinned llama.cpp submodule for GPTJ and LLAMA based models
+3. Or if your model is an MPT model you can use the conversion script located directly in this backend directory under the scripts subdirectory 
+
+# Check back for updates as we'll try to keep this updated as things change!

gpt4all-backend/include/gpt4all-backend/llmodel.h

@@ -0,0 +1,265 @@
+#ifndef LLMODEL_H
+#define LLMODEL_H
+
+#include <algorithm>
+#include <cassert>
+#include <cstddef>
+#include <cstdint>
+#include <functional>
+#include <optional>
+#include <span>
+#include <stdexcept>
+#include <string>
+#include <string_view>
+#include <unordered_map>
+#include <utility>
+#include <vector>
+
+class Dlhandle;
+
+using namespace std::string_literals;
+
+#define LLMODEL_MAX_PROMPT_BATCH 128
+
+class LLModel {
+public:
+    using Token = int32_t;
+
+    class BadArchError: public std::runtime_error {
+    public:
+        BadArchError(std::string arch)
+            : runtime_error("Unsupported model architecture: " + arch)
+            , m_arch(std::move(arch))
+            {}
+
+        const std::string &arch() const noexcept { return m_arch; }
+
+    private:
+        std::string m_arch;
+    };
+
+    class MissingImplementationError: public std::runtime_error {
+    public:
+        using std::runtime_error::runtime_error;
+    };
+
+    class UnsupportedModelError: public std::runtime_error {
+    public:
+        using std::runtime_error::runtime_error;
+    };
+
+    struct GPUDevice {
+        const char *backend;
+        int index;
+        int type;
+        size_t heapSize;
+        std::string name;
+        std::string vendor;
+
+        GPUDevice(const char *backend, int index, int type, size_t heapSize, std::string name, std::string vendor):
+            backend(backend), index(index), type(type), heapSize(heapSize), name(std::move(name)),
+            vendor(std::move(vendor)) {}
+
+        std::string selectionName() const
+        {
+            assert(backend == "cuda"s || backend == "kompute"s);
+            return backendName() + ": " + name;
+        }
+
+        std::string backendName() const { return backendIdToName(backend); }
+
+        static std::string backendIdToName(const std::string &backend) { return s_backendNames.at(backend); }
+
+        static std::string updateSelectionName(const std::string &name) {
+            if (name == "Auto" || name == "CPU" || name == "Metal")
+                return name;
+            auto it = std::find_if(s_backendNames.begin(), s_backendNames.end(), [&name](const auto &entry) {
+                return name.starts_with(entry.second + ": ");
+            });
+            if (it != s_backendNames.end())
+                return name;
+            return "Vulkan: " + name; // previously, there were only Vulkan devices
+        }
+
+    private:
+        static inline const std::unordered_map<std::string, std::string> s_backendNames {
+            {"cpu", "CPU"}, {"metal", "Metal"}, {"cuda", "CUDA"}, {"kompute", "Vulkan"},
+        };
+    };
+
+    class Implementation {
+    public:
+        Implementation(const Implementation &) = delete;
+        Implementation(Implementation &&);
+        ~Implementation();
+
+        std::string_view modelType() const { return m_modelType; }
+        std::string_view buildVariant() const { return m_buildVariant; }
+
+        static LLModel *construct(const std::string &modelPath, const std::string &backend = "auto", int n_ctx = 2048);
+        static std::vector<GPUDevice> availableGPUDevices(size_t memoryRequired = 0);
+        static int32_t maxContextLength(const std::string &modelPath);
+        static int32_t layerCount(const std::string &modelPath);
+        static bool isEmbeddingModel(const std::string &modelPath);
+        static void setImplementationsSearchPath(const std::string &path);
+        static const std::string &implementationsSearchPath();
+        static bool hasSupportedCPU();
+        // 0 for no, 1 for yes, -1 for non-x86_64
+        static int cpuSupportsAVX2();
+
+    private:
+        Implementation(Dlhandle &&);
+
+        static const std::vector<Implementation> &implementationList();
+        static const Implementation *implementation(const char *fname, const std::string &buildVariant);
+        static LLModel *constructGlobalLlama(const std::optional<std::string> &backend = std::nullopt);
+
+        char *(*m_getFileArch)(const char *fname);
+        bool (*m_isArchSupported)(const char *arch);
+        LLModel *(*m_construct)();
+
+        std::string_view m_modelType;
+        std::string_view m_buildVariant;
+        Dlhandle *m_dlhandle;
+    };
+
+    struct PromptContext {
+        std::vector<int32_t> tokens;    // current tokens in the context window
+        int32_t n_past = 0;             // number of tokens in past conversation
+        int32_t n_ctx = 0;              // number of tokens possible in context window
+        int32_t n_predict = 200;
+        int32_t top_k = 40;
+        float   top_p = 0.9f;
+        float   min_p = 0.0f;
+        float   temp = 0.9f;
+        int32_t n_batch = 9;
+        float   repeat_penalty = 1.10f;
+        int32_t repeat_last_n = 64;     // last n tokens to penalize
+        float   contextErase = 0.5f;    // percent of context to erase if we exceed the context window
+    };
+
+    using ProgressCallback = std::function<bool(float progress)>;
+
+    explicit LLModel() {}
+    virtual ~LLModel() {}
+
+    virtual bool supportsEmbedding() const = 0;
+    virtual bool supportsCompletion() const = 0;
+    virtual bool loadModel(const std::string &modelPath, int n_ctx, int ngl) = 0;
+    virtual bool isModelBlacklisted(const std::string &modelPath) const { (void)modelPath; return false; }
+    virtual bool isEmbeddingModel(const std::string &modelPath) const { (void)modelPath; return false; }
+    virtual bool isModelLoaded() const = 0;
+    virtual size_t requiredMem(const std::string &modelPath, int n_ctx, int ngl) = 0;
+    virtual size_t stateSize() const = 0;
+    virtual size_t saveState(std::span<uint8_t> dest) const = 0;
+    virtual size_t restoreState(std::span<const uint8_t> src) = 0;
+
+    // This method requires the model to return true from supportsCompletion otherwise it will throw
+    // an error
+    virtual void prompt(const std::string &prompt,
+                        const std::string &promptTemplate,
+                        std::function<bool(int32_t)> promptCallback,
+                        std::function<bool(int32_t, const std::string&)> responseCallback,
+                        bool allowContextShift,
+                        PromptContext &ctx,
+                        bool special = false,
+                        std::optional<std::string_view> fakeReply = {});
+
+    using EmbedCancelCallback = bool(unsigned *batchSizes, unsigned nBatch, const char *backend);
+
+    virtual size_t embeddingSize() const {
+        throw std::logic_error(std::string(implementation().modelType()) + " does not support embeddings");
+    }
+    // user-specified prefix
+    virtual void embed(const std::vector<std::string> &texts, float *embeddings, std::optional<std::string> prefix,
+                       int dimensionality = -1, size_t *tokenCount = nullptr, bool doMean = true, bool atlas = false,
+                       EmbedCancelCallback *cancelCb = nullptr);
+    // automatic prefix
+    virtual void embed(const std::vector<std::string> &texts, float *embeddings, bool isRetrieval,
+                       int dimensionality = -1, size_t *tokenCount = nullptr, bool doMean = true, bool atlas = false);
+
+    virtual void setThreadCount(int32_t n_threads) { (void)n_threads; }
+    virtual int32_t threadCount() const { return 1; }
+
+    const Implementation &implementation() const {
+        return *m_implementation;
+    }
+
+    virtual std::vector<GPUDevice> availableGPUDevices(size_t memoryRequired) const {
+        (void)memoryRequired;
+        return {};
+    }
+
+    virtual bool initializeGPUDevice(size_t memoryRequired, const std::string &name) const {
+        (void)memoryRequired;
+        (void)name;
+        return false;
+    }
+
+    virtual bool initializeGPUDevice(int device, std::string *unavail_reason = nullptr) const {
+        (void)device;
+        if (unavail_reason) {
+            *unavail_reason = "model has no GPU support";
+        }
+        return false;
+    }
+
+    virtual bool usingGPUDevice() const { return false; }
+    virtual const char *backendName() const { return "cpu"; }
+    virtual const char *gpuDeviceName() const { return nullptr; }
+
+    void setProgressCallback(ProgressCallback callback) { m_progressCallback = callback; }
+
+protected:
+    // These are pure virtual because subclasses need to implement as the default implementation of
+    // 'prompt' above calls these functions
+    virtual std::vector<Token> tokenize(std::string_view str, bool special = false) = 0;
+    virtual bool isSpecialToken(Token id) const = 0;
+    virtual std::string tokenToString(Token id) const = 0;
+    virtual void initSampler(PromptContext &ctx) = 0;
+    virtual Token sampleToken() const = 0;
+    virtual bool evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const = 0;
+    virtual void shiftContext(PromptContext &promptCtx) = 0;
+    virtual int32_t contextLength() const = 0;
+    virtual const std::vector<Token> &endTokens() const = 0;
+    virtual bool shouldAddBOS() const = 0;
+
+    virtual int32_t maxContextLength(std::string const &modelPath) const
+    {
+        (void)modelPath;
+        return -1;
+    }
+
+    virtual int32_t layerCount(std::string const &modelPath) const
+    {
+        (void)modelPath;
+        return -1;
+    }
+
+    const Implementation *m_implementation = nullptr;
+
+    ProgressCallback m_progressCallback;
+    static bool staticProgressCallback(float progress, void* ctx)
+    {
+        LLModel* model = static_cast<LLModel*>(ctx);
+        if (model && model->m_progressCallback)
+            return model->m_progressCallback(progress);
+        return true;
+    }
+
+    bool decodePrompt(std::function<bool(int32_t)> promptCallback,
+                      std::function<bool(int32_t, const std::string&)> responseCallback,
+                      bool allowContextShift,
+                      PromptContext &promptCtx,
+                      std::vector<Token> embd_inp,
+                      bool isResponse = false);
+    void generateResponse(std::function<bool(int32_t, const std::string&)> responseCallback,
+                          bool allowContextShift,
+                          PromptContext &promptCtx);
+
+    Token m_tokenize_last_token = -1; // not serialized
+
+    friend class LLMImplementation;
+};
+
+#endif // LLMODEL_H

gpt4all-backend/include/gpt4all-backend/llmodel_c.h

@@ -0,0 +1,301 @@
+#ifndef LLMODEL_C_H
+#define LLMODEL_C_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+
+#ifdef __GNUC__
+#define DEPRECATED __attribute__ ((deprecated))
+#elif defined(_MSC_VER)
+#define DEPRECATED __declspec(deprecated)
+#else
+#pragma message("WARNING: You need to implement DEPRECATED for this compiler")
+#define DEPRECATED
+#endif
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Opaque pointer to the underlying model.
+ */
+typedef void *llmodel_model;
+
+/**
+ * llmodel_prompt_context structure for holding the prompt context.
+ * NOTE: The implementation takes care of all the memory handling of the raw logits pointer and the
+ * raw tokens pointer. Attempting to resize them or modify them in any way can lead to undefined
+ * behavior.
+ */
+struct llmodel_prompt_context {
+    int32_t *tokens;        // current tokens in the context window
+    size_t tokens_size;     // the size of the raw tokens vector
+    int32_t n_past;         // number of tokens in past conversation
+    int32_t n_ctx;          // number of tokens possible in context window
+    int32_t n_predict;      // number of tokens to predict
+    int32_t top_k;          // top k logits to sample from
+    float top_p;            // nucleus sampling probability threshold
+    float min_p;            // Min P sampling
+    float temp;             // temperature to adjust model's output distribution
+    int32_t n_batch;        // number of predictions to generate in parallel
+    float repeat_penalty;   // penalty factor for repeated tokens
+    int32_t repeat_last_n;  // last n tokens to penalize
+    float context_erase;    // percent of context to erase if we exceed the context window
+};
+
+struct llmodel_gpu_device {
+    const char * backend;
+    int index;
+    int type; // same as VkPhysicalDeviceType
+    size_t heapSize;
+    const char * name;
+    const char * vendor;
+};
+
+#ifndef __cplusplus
+typedef struct llmodel_prompt_context llmodel_prompt_context;
+typedef struct llmodel_gpu_device llmodel_gpu_device;
+#endif
+
+/**
+ * Callback type for prompt processing.
+ * @param token_id The token id of the prompt.
+ * @return a bool indicating whether the model should keep processing.
+ */
+typedef bool (*llmodel_prompt_callback)(int32_t token_id);
+
+/**
+ * Callback type for response.
+ * @param token_id The token id of the response.
+ * @param response The response string. NOTE: a token_id of -1 indicates the string is an error string.
+ * @return a bool indicating whether the model should keep generating.
+ */
+typedef bool (*llmodel_response_callback)(int32_t token_id, const char *response);
+
+/**
+ * Embedding cancellation callback for use with llmodel_embed.
+ * @param batch_sizes The number of tokens in each batch that will be embedded.
+ * @param n_batch The number of batches that will be embedded.
+ * @param backend The backend that will be used for embedding. One of "cpu", "kompute", "cuda", or "metal".
+ * @return True to cancel llmodel_embed, false to continue.
+ */
+typedef bool (*llmodel_emb_cancel_callback)(unsigned *batch_sizes, unsigned n_batch, const char *backend);
+
+/**
+ * Create a llmodel instance.
+ * Recognises correct model type from file at model_path
+ * @param model_path A string representing the path to the model file.
+ * @return A pointer to the llmodel_model instance; NULL on error.
+ */
+DEPRECATED llmodel_model llmodel_model_create(const char *model_path);
+
+/**
+ * Create a llmodel instance.
+ * Recognises correct model type from file at model_path
+ * @param model_path A string representing the path to the model file; will only be used to detect model type.
+ * @param backend A string representing the implementation to use. One of 'auto', 'cpu', 'metal', 'kompute', or 'cuda'.
+ * @param error A pointer to a string; will only be set on error.
+ * @return A pointer to the llmodel_model instance; NULL on error.
+ */
+llmodel_model llmodel_model_create2(const char *model_path, const char *backend, const char **error);
+
+/**
+ * Destroy a llmodel instance.
+ * Recognises correct model type using type info
+ * @param model a pointer to a llmodel_model instance.
+ */
+void llmodel_model_destroy(llmodel_model model);
+
+/**
+ * Estimate RAM requirement for a model file
+ * @param model A pointer to the llmodel_model instance.
+ * @param model_path A string representing the path to the model file.
+ * @param n_ctx Maximum size of context window
+ * @param ngl Number of GPU layers to use (Vulkan)
+ * @return size greater than 0 if the model was parsed successfully, 0 if file could not be parsed.
+ */
+size_t llmodel_required_mem(llmodel_model model, const char *model_path, int n_ctx, int ngl);
+
+/**
+ * Load a model from a file.
+ * @param model A pointer to the llmodel_model instance.
+ * @param model_path A string representing the path to the model file.
+ * @param n_ctx Maximum size of context window
+ * @param ngl Number of GPU layers to use (Vulkan)
+ * @return true if the model was loaded successfully, false otherwise.
+ */
+bool llmodel_loadModel(llmodel_model model, const char *model_path, int n_ctx, int ngl);
+
+/**
+ * Check if a model is loaded.
+ * @param model A pointer to the llmodel_model instance.
+ * @return true if the model is loaded, false otherwise.
+ */
+bool llmodel_isModelLoaded(llmodel_model model);
+
+/**
+ * Get the size of the internal state of the model.
+ * NOTE: This state data is specific to the type of model you have created.
+ * @param model A pointer to the llmodel_model instance.
+ * @return the size in bytes of the internal state of the model
+ */
+uint64_t llmodel_get_state_size(llmodel_model model);
+
+/**
+ * Saves the internal state of the model to the specified destination address.
+ * NOTE: This state data is specific to the type of model you have created.
+ * @param model A pointer to the llmodel_model instance.
+ * @param dest A pointer to the destination.
+ * @param size The size of the destination buffer.
+ * @return the number of bytes copied, or zero on error.
+ */
+uint64_t llmodel_save_state_data(llmodel_model model, uint8_t *dest, uint64_t size);
+
+/**
+ * Restores the internal state of the model using data from the specified address.
+ * NOTE: This state data is specific to the type of model you have created.
+ * @param model A pointer to the llmodel_model instance.
+ * @param src A pointer to the state data.
+ * @param size The size of the source data.
+ * @return The number of bytes read, or zero on error.
+ */
+uint64_t llmodel_restore_state_data(llmodel_model model, const uint8_t *src, size_t size);
+
+/**
+ * Generate a response using the model.
+ * @param model A pointer to the llmodel_model instance.
+ * @param prompt A string representing the input prompt.
+ * @param prompt_template A string representing the input prompt template.
+ * @param prompt_callback A callback function for handling the processing of prompt.
+ * @param response_callback A callback function for handling the generated response.
+ * @param allow_context_shift Whether to allow shifting of context to make room for more input.
+ * @param special True if special tokens in the prompt should be processed, false otherwise.
+ * @param fake_reply A string to insert into context as the model's reply, or NULL to generate one.
+ * @param ctx A pointer to the llmodel_prompt_context structure.
+ */
+void llmodel_prompt(llmodel_model model, const char *prompt,
+                    const char *prompt_template,
+                    llmodel_prompt_callback prompt_callback,
+                    llmodel_response_callback response_callback,
+                    bool allow_context_shift,
+                    llmodel_prompt_context *ctx,
+                    bool special,
+                    const char *fake_reply);
+
+/**
+ * Generate an embedding using the model.
+ * NOTE: If given NULL pointers for the model or text, or an empty text, a NULL pointer will be
+ * returned. Bindings should signal an error when NULL is the return value.
+ * @param model A pointer to the llmodel_model instance.
+ * @param texts A pointer to a NULL-terminated array of strings representing the texts to generate an
+ * embedding for.
+ * @param embedding_size A pointer to a size_t type that will be set by the call indicating the length
+ * of the returned floating point array.
+ * @param prefix The model-specific prefix representing the embedding task, without the trailing colon. NULL for no
+ * prefix.
+ * @param dimensionality The embedding dimension, for use with Matryoshka-capable models. Set to -1 to for full-size.
+ * @param token_count Return location for the number of prompt tokens processed, or NULL.
+ * @param do_mean True to average multiple embeddings if the text is longer than the model can accept, False to
+ * truncate.
+ * @param atlas Try to be fully compatible with the Atlas API. Currently, this means texts longer than 8192 tokens with
+ * long_text_mode="mean" will raise an error. Disabled by default.
+ * @param cancel_cb Cancellation callback, or NULL. See the documentation of llmodel_emb_cancel_callback.
+ * @param error Return location for a malloc()ed string that will be set on error, or NULL.
+ * @return A pointer to an array of floating point values passed to the calling method which then will
+ * be responsible for lifetime of this memory. NULL if an error occurred.
+ */
+float *llmodel_embed(llmodel_model model, const char **texts, size_t *embedding_size, const char *prefix,
+                     int dimensionality, size_t *token_count, bool do_mean, bool atlas,
+                     llmodel_emb_cancel_callback cancel_cb, const char **error);
+
+/**
+ * Frees the memory allocated by the llmodel_embedding function.
+ * @param ptr A pointer to the embedding as returned from llmodel_embedding.
+ */
+void llmodel_free_embedding(float *ptr);
+
+/**
+ * Set the number of threads to be used by the model.
+ * @param model A pointer to the llmodel_model instance.
+ * @param n_threads The number of threads to be used.
+ */
+void llmodel_setThreadCount(llmodel_model model, int32_t n_threads);
+
+/**
+ * Get the number of threads currently being used by the model.
+ * @param model A pointer to the llmodel_model instance.
+ * @return The number of threads currently being used.
+ */
+int32_t llmodel_threadCount(llmodel_model model);
+
+/**
+ * Set llmodel implementation search path.
+ * Default is "."
+ * @param path The path to the llmodel implementation shared objects. This can be a single path or
+ * a list of paths separated by ';' delimiter.
+ */
+void llmodel_set_implementation_search_path(const char *path);
+
+/**
+ * Get llmodel implementation search path.
+ * @return The current search path; lifetime ends on next set llmodel_set_implementation_search_path() call.
+ */
+const char *llmodel_get_implementation_search_path();
+
+/**
+ * Get a list of available GPU devices given the memory required.
+ * @param memoryRequired The minimum amount of VRAM, in bytes
+ * @return A pointer to an array of llmodel_gpu_device's whose number is given by num_devices.
+ */
+struct llmodel_gpu_device* llmodel_available_gpu_devices(size_t memoryRequired, int* num_devices);
+
+/**
+ * Initializes a GPU device based on a specified string criterion.
+ *
+ * This function initializes a GPU device based on a string identifier provided. The function
+ * allows initialization based on general device type ("gpu"), vendor name ("amd", "nvidia", "intel"),
+ * or any specific device name.
+ *
+ * @param memoryRequired The amount of memory (in bytes) required by the application or task
+ *                       that will utilize the GPU device.
+ * @param device A string specifying the desired criterion for GPU device selection. It can be:
+ *               - "gpu": To initialize the best available GPU.
+ *               - "amd", "nvidia", or "intel": To initialize the best available GPU from that vendor.
+ *               - A specific GPU device name: To initialize a GPU with that exact name.
+ *
+ * @return True if the GPU device is successfully initialized based on the provided string
+ *         criterion. Returns false if the desired GPU device could not be initialized.
+ */
+bool llmodel_gpu_init_gpu_device_by_string(llmodel_model model, size_t memoryRequired, const char *device);
+
+/**
+ * Initializes a GPU device by specifying a valid gpu device pointer.
+ * @param device A gpu device pointer.
+ * @return True if the GPU device is successfully initialized, false otherwise.
+ */
+bool llmodel_gpu_init_gpu_device_by_struct(llmodel_model model, const llmodel_gpu_device *device);
+
+/**
+ * Initializes a GPU device by its index.
+ * @param device An integer representing the index of the GPU device to be initialized.
+ * @return True if the GPU device is successfully initialized, false otherwise.
+ */
+bool llmodel_gpu_init_gpu_device_by_int(llmodel_model model, int device);
+
+/**
+ * @return The name of the llama.cpp backend currently in use. One of "cpu", "kompute", or "metal".
+ */
+const char *llmodel_model_backend_name(llmodel_model model);
+
+/**
+ * @return The name of the GPU device currently in use, or NULL for backends other than Kompute.
+ */
+const char *llmodel_model_gpu_device_name(llmodel_model model);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif // LLMODEL_C_H

gpt4all-backend/include/gpt4all-backend/sysinfo.h

@@ -0,0 +1,65 @@
+#ifndef SYSINFO_H
+#define SYSINFO_H
+
+#include <fstream>
+#include <iomanip>
+#include <sstream>
+#include <string>
+
+#if defined(__linux__)
+#   include <unistd.h>
+#elif defined(__APPLE__)
+#   include <sys/types.h>
+#   include <sys/sysctl.h>
+#elif defined(_WIN32)
+#   define WIN32_LEAN_AND_MEAN
+#   ifndef NOMINMAX
+#       define NOMINMAX
+#   endif
+#   include <windows.h>
+#endif
+
+static long long getSystemTotalRAMInBytes()
+{
+    long long totalRAM = 0;
+
+#if defined(__linux__)
+    std::ifstream file("/proc/meminfo");
+    std::string line;
+    while (std::getline(file, line)) {
+        if (line.find("MemTotal") != std::string::npos) {
+            std::string memTotalStr = line.substr(line.find(":") + 1);
+            memTotalStr.erase(0, memTotalStr.find_first_not_of(" "));
+            memTotalStr = memTotalStr.substr(0, memTotalStr.find(" "));
+            totalRAM = std::stoll(memTotalStr) * 1024;  // Convert from KB to bytes
+            break;
+        }
+    }
+    file.close();
+#elif defined(__APPLE__)
+    int mib[2] = {CTL_HW, HW_MEMSIZE};
+    size_t length = sizeof(totalRAM);
+    sysctl(mib, 2, &totalRAM, &length, NULL, 0);
+#elif defined(_WIN32)
+    MEMORYSTATUSEX memoryStatus;
+    memoryStatus.dwLength = sizeof(memoryStatus);
+    GlobalMemoryStatusEx(&memoryStatus);
+    totalRAM = memoryStatus.ullTotalPhys;
+#endif
+
+    return totalRAM;
+}
+
+static double getSystemTotalRAMInGB()
+{
+    return static_cast<double>(getSystemTotalRAMInBytes()) / (1024 * 1024 * 1024);
+}
+
+static std::string getSystemTotalRAMInGBString()
+{
+    std::stringstream ss;
+    ss << std::fixed << std::setprecision(2) << getSystemTotalRAMInGB() << " GB";
+    return ss.str();
+}
+
+#endif // SYSINFO_H

gpt4all-backend/src/dlhandle.cpp

@@ -0,0 +1,73 @@
+#include "dlhandle.h"
+
+#include <string>
+
+#ifndef _WIN32
+#   include <dlfcn.h>
+#else
+#   include <cassert>
+#   include <sstream>
+#   define WIN32_LEAN_AND_MEAN
+#   ifndef NOMINMAX
+#       define NOMINMAX
+#   endif
+#   include <windows.h>
+#endif
+
+using namespace std::string_literals;
+namespace fs = std::filesystem;
+
+
+#ifndef _WIN32
+
+Dlhandle::Dlhandle(const fs::path &fpath)
+{
+    chandle = dlopen(fpath.c_str(), RTLD_LAZY | RTLD_LOCAL);
+    if (!chandle) {
+        throw Exception("dlopen: "s + dlerror());
+    }
+}
+
+Dlhandle::~Dlhandle()
+{
+    if (chandle) dlclose(chandle);
+}
+
+void *Dlhandle::get_internal(const char *symbol) const
+{
+    return dlsym(chandle, symbol);
+}
+
+#else // defined(_WIN32)
+
+Dlhandle::Dlhandle(const fs::path &fpath)
+{
+    fs::path afpath = fs::absolute(fpath);
+
+    // Suppress the "Entry Point Not Found" dialog, caused by outdated nvcuda.dll from the GPU driver
+    UINT lastErrorMode = GetErrorMode();
+    SetErrorMode(lastErrorMode | SEM_FAILCRITICALERRORS);
+
+    chandle = LoadLibraryExW(afpath.c_str(), NULL, LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR);
+
+    SetErrorMode(lastErrorMode);
+
+    if (!chandle) {
+        DWORD err = GetLastError();
+        std::ostringstream ss;
+        ss << "LoadLibraryExW failed with error 0x" << std::hex << err;
+        throw Exception(ss.str());
+    }
+}
+
+Dlhandle::~Dlhandle()
+{
+    if (chandle) FreeLibrary(HMODULE(chandle));
+}
+
+void *Dlhandle::get_internal(const char *symbol) const
+{
+    return GetProcAddress(HMODULE(chandle), symbol);
+}
+
+#endif // defined(_WIN32)

gpt4all-backend/src/dlhandle.h

@@ -0,0 +1,47 @@
+#pragma once
+
+#include <filesystem>
+#include <stdexcept>
+#include <string>
+#include <utility>
+
+namespace fs = std::filesystem;
+
+
+class Dlhandle {
+    void *chandle = nullptr;
+
+public:
+    class Exception : public std::runtime_error {
+    public:
+        using std::runtime_error::runtime_error;
+    };
+
+    Dlhandle() = default;
+    Dlhandle(const fs::path &fpath);
+    Dlhandle(const Dlhandle &o) = delete;
+    Dlhandle(Dlhandle &&o)
+        : chandle(o.chandle)
+    {
+        o.chandle = nullptr;
+    }
+
+    ~Dlhandle();
+
+    Dlhandle &operator=(Dlhandle &&o) {
+        chandle = std::exchange(o.chandle, nullptr);
+        return *this;
+    }
+
+    template <typename T>
+    T *get(const std::string &symbol) const {
+        return reinterpret_cast<T *>(get_internal(symbol.c_str()));
+    }
+
+    auto get_fnc(const std::string &symbol) const {
+        return get<void*(...)>(symbol);
+    }
+
+private:
+    void *get_internal(const char *symbol) const;
+};

gpt4all-backend/src/llamamodel_impl.h

@@ -0,0 +1,75 @@
+#ifndef LLAMAMODEL_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
+#error This file is NOT meant to be included outside of llamamodel.cpp. Doing so is DANGEROUS. Be sure to know what you are doing before proceeding to #define LLAMAMODEL_H_I_KNOW_WHAT_I_AM_DOING_WHEN_INCLUDING_THIS_FILE
+#endif
+#ifndef LLAMAMODEL_H
+#define LLAMAMODEL_H
+
+#include "llmodel.h"
+
+#include <memory>
+#include <span>
+#include <string>
+#include <string_view>
+#include <vector>
+
+struct LLamaPrivate;
+struct EmbModelSpec;
+
+class LLamaModel : public LLModel {
+public:
+    LLamaModel();
+    ~LLamaModel();
+
+    bool supportsEmbedding() const override { return m_supportsEmbedding; }
+    bool supportsCompletion() const override { return m_supportsCompletion; }
+    bool loadModel(const std::string &modelPath, int n_ctx, int ngl) override;
+    bool isModelBlacklisted(const std::string &modelPath) const override;
+    bool isEmbeddingModel(const std::string &modelPath) const override;
+    bool isModelLoaded() const override;
+    size_t requiredMem(const std::string &modelPath, int n_ctx, int ngl) override;
+    size_t stateSize() const override;
+    size_t saveState(std::span<uint8_t> dest) const override;
+    size_t restoreState(std::span<const uint8_t> src) override;
+    void setThreadCount(int32_t n_threads) override;
+    int32_t threadCount() const override;
+    std::vector<GPUDevice> availableGPUDevices(size_t memoryRequired = 0) const override;
+    bool initializeGPUDevice(size_t memoryRequired, const std::string &name) const override;
+    bool initializeGPUDevice(int device, std::string *unavail_reason = nullptr) const override;
+    bool usingGPUDevice() const override;
+    const char *backendName() const override;
+    const char *gpuDeviceName() const override;
+
+    size_t embeddingSize() const override;
+    // user-specified prefix
+    void embed(const std::vector<std::string> &texts, float *embeddings, std::optional<std::string> prefix,
+               int dimensionality = -1, size_t *tokenCount = nullptr, bool doMean = true, bool atlas = false,
+               EmbedCancelCallback *cancelCb = nullptr) override;
+    // automatic prefix
+    void embed(const std::vector<std::string> &texts, float *embeddings, bool isRetrieval, int dimensionality = -1,
+               size_t *tokenCount = nullptr, bool doMean = true, bool atlas = false) override;
+
+private:
+    std::unique_ptr<LLamaPrivate> d_ptr;
+    bool m_supportsEmbedding = false;
+    bool m_supportsCompletion = false;
+
+protected:
+    std::vector<Token> tokenize(std::string_view str, bool special) override;
+    bool isSpecialToken(Token id) const override;
+    std::string tokenToString(Token id) const override;
+    void initSampler(PromptContext &ctx) override;
+    Token sampleToken() const override;
+    bool evalTokens(PromptContext &ctx, const std::vector<int32_t> &tokens) const override;
+    void shiftContext(PromptContext &promptCtx) override;
+    int32_t contextLength() const override;
+    const std::vector<Token> &endTokens() const override;
+    bool shouldAddBOS() const override;
+    int32_t maxContextLength(std::string const &modelPath) const override;
+    int32_t layerCount(std::string const &modelPath) const override;
+
+    void embedInternal(const std::vector<std::string> &texts, float *embeddings, std::string prefix, int dimensionality,
+                       size_t *tokenCount, bool doMean, bool atlas, EmbedCancelCallback *cancelCb,
+                       const EmbModelSpec *spec);
+};
+
+#endif // LLAMAMODEL_H

gpt4all-backend/src/llmodel.cpp

@@ -0,0 +1,347 @@
+#include "llmodel.h"
+
+#include "dlhandle.h"
+
+#include <cassert>
+#include <cstdlib>
+#include <filesystem>
+#include <fstream>
+#include <iostream>
+#include <iterator>
+#include <memory>
+#include <optional>
+#include <regex>
+#include <sstream>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+#ifdef _WIN32
+#   define WIN32_LEAN_AND_MEAN
+#   ifndef NOMINMAX
+#       define NOMINMAX
+#   endif
+#   include <windows.h>
+#endif
+
+#ifdef _MSC_VER
+#   include <intrin.h>
+#endif
+
+#if defined(__APPLE__) && defined(__aarch64__)
+#   include "sysinfo.h" // for getSystemTotalRAMInBytes
+#endif
+
+namespace fs = std::filesystem;
+
+#ifndef __APPLE__
+static const std::string DEFAULT_BACKENDS[] = {"kompute", "cpu"};
+#elif defined(__aarch64__)
+static const std::string DEFAULT_BACKENDS[] = {"metal", "cpu"};
+#else
+static const std::string DEFAULT_BACKENDS[] = {"cpu"};
+#endif
+
+std::string s_implementations_search_path = ".";
+
+#if !(defined(__x86_64__) || defined(_M_X64))
+    // irrelevant on non-x86_64
+    #define cpu_supports_avx()  -1
+    #define cpu_supports_avx2() -1
+#elif defined(_MSC_VER)
+    // MSVC
+    static int get_cpu_info(int func_id, int reg_id) {
+        int info[4];
+        __cpuid(info, func_id);
+        return info[reg_id];
+    }
+
+    // AVX via EAX=1: Processor Info and Feature Bits, bit 28 of ECX
+    #define cpu_supports_avx()  !!(get_cpu_info(1, 2) & (1 << 28))
+    // AVX2 via EAX=7, ECX=0: Extended Features, bit 5 of EBX
+    #define cpu_supports_avx2() !!(get_cpu_info(7, 1) & (1 <<  5))
+#else
+    // gcc/clang
+    #define cpu_supports_avx()  !!__builtin_cpu_supports("avx")
+    #define cpu_supports_avx2() !!__builtin_cpu_supports("avx2")
+#endif
+
+LLModel::Implementation::Implementation(Dlhandle &&dlhandle_)
+    : m_dlhandle(new Dlhandle(std::move(dlhandle_))) {
+    auto get_model_type = m_dlhandle->get<const char *()>("get_model_type");
+    assert(get_model_type);
+    m_modelType = get_model_type();
+    auto get_build_variant = m_dlhandle->get<const char *()>("get_build_variant");
+    assert(get_build_variant);
+    m_buildVariant = get_build_variant();
+    m_getFileArch = m_dlhandle->get<char *(const char *)>("get_file_arch");
+    assert(m_getFileArch);
+    m_isArchSupported = m_dlhandle->get<bool(const char *)>("is_arch_supported");
+    assert(m_isArchSupported);
+    m_construct = m_dlhandle->get<LLModel *()>("construct");
+    assert(m_construct);
+}
+
+LLModel::Implementation::Implementation(Implementation &&o)
+    : m_getFileArch(o.m_getFileArch)
+    , m_isArchSupported(o.m_isArchSupported)
+    , m_construct(o.m_construct)
+    , m_modelType(o.m_modelType)
+    , m_buildVariant(o.m_buildVariant)
+    , m_dlhandle(o.m_dlhandle) {
+    o.m_dlhandle = nullptr;
+}
+
+LLModel::Implementation::~Implementation()
+{
+    delete m_dlhandle;
+}
+
+static bool isImplementation(const Dlhandle &dl)
+{
+    return dl.get<bool(uint32_t)>("is_g4a_backend_model_implementation");
+}
+
+// Add the CUDA Toolkit to the DLL search path on Windows.
+// This is necessary for chat.exe to find CUDA when started from Qt Creator.
+static void addCudaSearchPath()
+{
+#ifdef _WIN32
+    if (const auto *cudaPath = _wgetenv(L"CUDA_PATH")) {
+        auto libDir = std::wstring(cudaPath) + L"\\bin";
+        if (!AddDllDirectory(libDir.c_str())) {
+            auto err = GetLastError();
+            std::wcerr << L"AddDllDirectory(\"" << libDir << L"\") failed with error 0x" << std::hex << err << L"\n";
+        }
+    }
+#endif
+}
+
+const std::vector<LLModel::Implementation> &LLModel::Implementation::implementationList()
+{
+    if (cpu_supports_avx() == 0) {
+        throw std::runtime_error("CPU does not support AVX");
+    }
+
+    // NOTE: allocated on heap so we leak intentionally on exit so we have a chance to clean up the
+    // individual models without the cleanup of the static list interfering
+    static auto* libs = new std::vector<Implementation>([] () {
+        std::vector<Implementation> fres;
+
+        addCudaSearchPath();
+
+        std::string impl_name_re = "llamamodel-mainline-(cpu|metal|kompute|vulkan|cuda)";
+        if (cpu_supports_avx2() == 0) {
+            impl_name_re += "-avxonly";
+        }
+        std::regex re(impl_name_re);
+        auto search_in_directory = [&](const std::string& paths) {
+            std::stringstream ss(paths);
+            std::string path;
+            // Split the paths string by the delimiter and process each path.
+            while (std::getline(ss, path, ';')) {
+                std::u8string u8_path(path.begin(), path.end());
+                // Iterate over all libraries
+                for (const auto &f : fs::directory_iterator(u8_path)) {
+                    const fs::path &p = f.path();
+
+                    if (p.extension() != LIB_FILE_EXT) continue;
+                    if (!std::regex_search(p.stem().string(), re)) continue;
+
+                    // Add to list if model implementation
+                    Dlhandle dl;
+                    try {
+                        dl = Dlhandle(p);
+                    } catch (const Dlhandle::Exception &e) {
+                        std::cerr << "Failed to load " << p.filename().string() << ": " << e.what() << "\n";
+                        continue;
+                    }
+                    if (!isImplementation(dl)) {
+                        std::cerr << "Not an implementation: " << p.filename().string() << "\n";
+                        continue;
+                    }
+                    fres.emplace_back(Implementation(std::move(dl)));
+                }
+            }
+        };
+
+        search_in_directory(s_implementations_search_path);
+
+        return fres;
+    }());
+    // Return static result
+    return *libs;
+}
+
+static std::string applyCPUVariant(const std::string &buildVariant)
+{
+    if (buildVariant != "metal" && cpu_supports_avx2() == 0) {
+        return buildVariant + "-avxonly";
+    }
+    return buildVariant;
+}
+
+const LLModel::Implementation* LLModel::Implementation::implementation(const char *fname, const std::string& buildVariant)
+{
+    bool buildVariantMatched = false;
+    std::optional<std::string> archName;
+    for (const auto& i : implementationList()) {
+        if (buildVariant != i.m_buildVariant) continue;
+        buildVariantMatched = true;
+
+        char *arch = i.m_getFileArch(fname);
+        if (!arch) continue;
+        archName = arch;
+
+        bool archSupported = i.m_isArchSupported(arch);
+        free(arch);
+        if (archSupported) return &i;
+    }
+
+    if (!buildVariantMatched)
+        return nullptr;
+    if (!archName)
+        throw UnsupportedModelError("Unsupported file format");
+
+    throw BadArchError(std::move(*archName));
+}
+
+LLModel *LLModel::Implementation::construct(const std::string &modelPath, const std::string &backend, int n_ctx)
+{
+    std::vector<std::string> desiredBackends;
+    if (backend != "auto") {
+        desiredBackends.push_back(backend);
+    } else {
+        desiredBackends.insert(desiredBackends.end(), DEFAULT_BACKENDS, std::end(DEFAULT_BACKENDS));
+    }
+
+    for (const auto &desiredBackend: desiredBackends) {
+        const auto *impl = implementation(modelPath.c_str(), applyCPUVariant(desiredBackend));
+
+        if (impl) {
+            // Construct llmodel implementation
+            auto *fres = impl->m_construct();
+            fres->m_implementation = impl;
+
+#if defined(__APPLE__) && defined(__aarch64__) // FIXME: See if metal works for intel macs
+            /* TODO(cebtenzzre): after we fix requiredMem, we should change this to happen at
+             * load time, not construct time. right now n_ctx is incorrectly hardcoded 2048 in
+             * most (all?) places where this is called, causing underestimation of required
+             * memory. */
+            if (backend == "auto" && desiredBackend == "metal") {
+                // on a 16GB M2 Mac a 13B q4_0 (0.52) works for me but a 13B q4_K_M (0.55) does not
+                size_t req_mem = fres->requiredMem(modelPath, n_ctx, 100);
+                if (req_mem >= size_t(0.53f * getSystemTotalRAMInBytes())) {
+                    delete fres;
+                    continue;
+                }
+            }
+#else
+            (void)n_ctx;
+#endif
+
+            return fres;
+        }
+    }
+
+    throw MissingImplementationError("Could not find any implementations for backend: " + backend);
+}
+
+LLModel *LLModel::Implementation::constructGlobalLlama(const std::optional<std::string> &backend)
+{
+    static std::unordered_map<std::string, std::unique_ptr<LLModel>> implCache;
+
+    const std::vector<Implementation> *impls;
+    try {
+        impls = &implementationList();
+    } catch (const std::runtime_error &e) {
+        std::cerr << __func__ << ": implementationList failed: " << e.what() << "\n";
+        return nullptr;
+    }
+
+    std::vector<std::string> desiredBackends;
+    if (backend) {
+        desiredBackends.push_back(backend.value());
+    } else {
+        desiredBackends.insert(desiredBackends.end(), DEFAULT_BACKENDS, std::end(DEFAULT_BACKENDS));
+    }
+
+    const Implementation *impl = nullptr;
+
+    for (const auto &desiredBackend: desiredBackends) {
+        auto cacheIt = implCache.find(desiredBackend);
+        if (cacheIt != implCache.end())
+            return cacheIt->second.get(); // cached
+
+        for (const auto &i: *impls) {
+            if (i.m_modelType == "LLaMA" && i.m_buildVariant == applyCPUVariant(desiredBackend)) {
+                impl = &i;
+                break;
+            }
+        }
+
+        if (impl) {
+            auto *fres = impl->m_construct();
+            fres->m_implementation = impl;
+            implCache[desiredBackend] = std::unique_ptr<LLModel>(fres);
+            return fres;
+        }
+    }
+
+    std::cerr << __func__ << ": could not find Llama implementation for backend: " << backend.value_or("default") << "\n";
+    return nullptr;
+}
+
+std::vector<LLModel::GPUDevice> LLModel::Implementation::availableGPUDevices(size_t memoryRequired)
+{
+    std::vector<LLModel::GPUDevice> devices;
+#ifndef __APPLE__
+    static const std::string backends[] = {"kompute", "cuda"};
+    for (const auto &backend: backends) {
+        auto *llama = constructGlobalLlama(backend);
+        if (llama) {
+            auto backendDevs = llama->availableGPUDevices(memoryRequired);
+            devices.insert(devices.end(), backendDevs.begin(), backendDevs.end());
+        }
+    }
+#endif
+    return devices;
+}
+
+int32_t LLModel::Implementation::maxContextLength(const std::string &modelPath)
+{
+    auto *llama = constructGlobalLlama();
+    return llama ? llama->maxContextLength(modelPath) : -1;
+}
+
+int32_t LLModel::Implementation::layerCount(const std::string &modelPath)
+{
+    auto *llama = constructGlobalLlama();
+    return llama ? llama->layerCount(modelPath) : -1;
+}
+
+bool LLModel::Implementation::isEmbeddingModel(const std::string &modelPath)
+{
+    auto *llama = constructGlobalLlama();
+    return llama && llama->isEmbeddingModel(modelPath);
+}
+
+void LLModel::Implementation::setImplementationsSearchPath(const std::string& path)
+{
+    s_implementations_search_path = path;
+}
+
+const std::string& LLModel::Implementation::implementationsSearchPath()
+{
+    return s_implementations_search_path;
+}
+
+bool LLModel::Implementation::hasSupportedCPU()
+{
+    return cpu_supports_avx() != 0;
+}
+
+int LLModel::Implementation::cpuSupportsAVX2()
+{
+    return cpu_supports_avx2();
+}

gpt4all-backend/src/llmodel_c.cpp

@@ -0,0 +1,293 @@
+#include "llmodel_c.h"
+
+#include "llmodel.h"
+
+#include <algorithm>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <exception>
+#include <functional>
+#include <iostream>
+#include <memory>
+#include <optional>
+#include <string>
+#include <string_view>
+#include <vector>
+
+struct LLModelWrapper {
+    LLModel *llModel = nullptr;
+    LLModel::PromptContext promptContext;
+    ~LLModelWrapper() { delete llModel; }
+};
+
+llmodel_model llmodel_model_create(const char *model_path)
+{
+    const char *error;
+    auto fres = llmodel_model_create2(model_path, "auto", &error);
+    if (!fres) {
+        fprintf(stderr, "Unable to instantiate model: %s\n", error);
+    }
+    return fres;
+}
+
+static void llmodel_set_error(const char **errptr, const char *message)
+{
+    thread_local static std::string last_error_message;
+    if (errptr) {
+        last_error_message = message;
+        *errptr = last_error_message.c_str();
+    }
+}
+
+llmodel_model llmodel_model_create2(const char *model_path, const char *backend, const char **error)
+{
+    LLModel *llModel;
+    try {
+        llModel = LLModel::Implementation::construct(model_path, backend);
+    } catch (const std::exception& e) {
+        llmodel_set_error(error, e.what());
+        return nullptr;
+    }
+
+    auto wrapper = new LLModelWrapper;
+    wrapper->llModel = llModel;
+    return wrapper;
+}
+
+void llmodel_model_destroy(llmodel_model model)
+{
+    delete static_cast<LLModelWrapper *>(model);
+}
+
+size_t llmodel_required_mem(llmodel_model model, const char *model_path, int n_ctx, int ngl)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->requiredMem(model_path, n_ctx, ngl);
+}
+
+bool llmodel_loadModel(llmodel_model model, const char *model_path, int n_ctx, int ngl)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+
+    std::string modelPath(model_path);
+    if (wrapper->llModel->isModelBlacklisted(modelPath)) {
+        size_t slash = modelPath.find_last_of("/\\");
+        auto basename = slash == std::string::npos ? modelPath : modelPath.substr(slash + 1);
+        std::cerr << "warning: model '" << basename << "' is out-of-date, please check for an updated version\n";
+    }
+    return wrapper->llModel->loadModel(modelPath, n_ctx, ngl);
+}
+
+bool llmodel_isModelLoaded(llmodel_model model)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->isModelLoaded();
+}
+
+uint64_t llmodel_get_state_size(llmodel_model model)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->stateSize();
+}
+
+uint64_t llmodel_save_state_data(llmodel_model model, uint8_t *dest, uint64_t size)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->saveState({dest, size_t(size)});
+}
+
+uint64_t llmodel_restore_state_data(llmodel_model model, const uint8_t *src, uint64_t size)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->restoreState({src, size_t(size)});
+}
+
+void llmodel_prompt(llmodel_model model, const char *prompt,
+                    const char *prompt_template,
+                    llmodel_prompt_callback prompt_callback,
+                    llmodel_response_callback response_callback,
+                    bool allow_context_shift,
+                    llmodel_prompt_context *ctx,
+                    bool special,
+                    const char *fake_reply)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+
+    auto response_func = [response_callback](int32_t token_id, const std::string &response) {
+        return response_callback(token_id, response.c_str());
+    };
+
+    // Copy the C prompt context
+    wrapper->promptContext.n_past = ctx->n_past;
+    wrapper->promptContext.n_ctx = ctx->n_ctx;
+    wrapper->promptContext.n_predict = ctx->n_predict;
+    wrapper->promptContext.top_k = ctx->top_k;
+    wrapper->promptContext.top_p = ctx->top_p;
+    wrapper->promptContext.min_p = ctx->min_p;
+    wrapper->promptContext.temp = ctx->temp;
+    wrapper->promptContext.n_batch = ctx->n_batch;
+    wrapper->promptContext.repeat_penalty = ctx->repeat_penalty;
+    wrapper->promptContext.repeat_last_n = ctx->repeat_last_n;
+    wrapper->promptContext.contextErase = ctx->context_erase;
+
+    // Call the C++ prompt method
+    wrapper->llModel->prompt(prompt, prompt_template, prompt_callback, response_func, allow_context_shift,
+                             wrapper->promptContext, special,
+                             fake_reply ? std::make_optional<std::string_view>(fake_reply) : std::nullopt);
+
+    // Update the C context by giving access to the wrappers raw pointers to std::vector data
+    // which involves no copies
+    ctx->tokens = wrapper->promptContext.tokens.data();
+    ctx->tokens_size = wrapper->promptContext.tokens.size();
+
+    // Update the rest of the C prompt context
+    ctx->n_past = wrapper->promptContext.n_past;
+    ctx->n_ctx = wrapper->promptContext.n_ctx;
+    ctx->n_predict = wrapper->promptContext.n_predict;
+    ctx->top_k = wrapper->promptContext.top_k;
+    ctx->top_p = wrapper->promptContext.top_p;
+    ctx->min_p = wrapper->promptContext.min_p;
+    ctx->temp = wrapper->promptContext.temp;
+    ctx->n_batch = wrapper->promptContext.n_batch;
+    ctx->repeat_penalty = wrapper->promptContext.repeat_penalty;
+    ctx->repeat_last_n = wrapper->promptContext.repeat_last_n;
+    ctx->context_erase = wrapper->promptContext.contextErase;
+}
+
+float *llmodel_embed(
+    llmodel_model model, const char **texts, size_t *embedding_size, const char *prefix, int dimensionality,
+    size_t *token_count, bool do_mean, bool atlas, llmodel_emb_cancel_callback cancel_cb, const char **error
+) {
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+
+    if (!texts || !*texts) {
+        llmodel_set_error(error, "'texts' is NULL or empty");
+        return nullptr;
+    }
+
+    std::vector<std::string> textsVec;
+    while (*texts) { textsVec.emplace_back(*texts++); }
+
+    size_t embd_size;
+    float *embedding;
+
+    try {
+        embd_size = wrapper->llModel->embeddingSize();
+        if (dimensionality > 0 && dimensionality < int(embd_size))
+            embd_size = dimensionality;
+
+        embd_size *= textsVec.size();
+
+        std::optional<std::string> prefixStr;
+        if (prefix) { prefixStr = prefix; }
+
+        embedding = new float[embd_size];
+        wrapper->llModel->embed(textsVec, embedding, prefixStr, dimensionality, token_count, do_mean, atlas, cancel_cb);
+    } catch (std::exception const &e) {
+        llmodel_set_error(error, e.what());
+        return nullptr;
+    }
+
+    *embedding_size = embd_size;
+    return embedding;
+}
+
+void llmodel_free_embedding(float *ptr)
+{
+    delete[] ptr;
+}
+
+void llmodel_setThreadCount(llmodel_model model, int32_t n_threads)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    wrapper->llModel->setThreadCount(n_threads);
+}
+
+int32_t llmodel_threadCount(llmodel_model model)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->threadCount();
+}
+
+void llmodel_set_implementation_search_path(const char *path)
+{
+    LLModel::Implementation::setImplementationsSearchPath(path);
+}
+
+const char *llmodel_get_implementation_search_path()
+{
+    return LLModel::Implementation::implementationsSearchPath().c_str();
+}
+
+// RAII wrapper around a C-style struct
+struct llmodel_gpu_device_cpp: llmodel_gpu_device {
+    llmodel_gpu_device_cpp() = default;
+
+    llmodel_gpu_device_cpp(const llmodel_gpu_device_cpp  &) = delete;
+    llmodel_gpu_device_cpp(      llmodel_gpu_device_cpp &&) = delete;
+
+    const llmodel_gpu_device_cpp &operator=(const llmodel_gpu_device_cpp  &) = delete;
+          llmodel_gpu_device_cpp &operator=(      llmodel_gpu_device_cpp &&) = delete;
+
+    ~llmodel_gpu_device_cpp() {
+        free(const_cast<char *>(name));
+        free(const_cast<char *>(vendor));
+    }
+};
+
+static_assert(sizeof(llmodel_gpu_device_cpp) == sizeof(llmodel_gpu_device));
+
+struct llmodel_gpu_device *llmodel_available_gpu_devices(size_t memoryRequired, int *num_devices)
+{
+    static thread_local std::unique_ptr<llmodel_gpu_device_cpp[]> c_devices;
+
+    auto devices = LLModel::Implementation::availableGPUDevices(memoryRequired);
+    *num_devices = devices.size();
+
+    if (devices.empty()) { return nullptr; /* no devices */ }
+
+    c_devices = std::make_unique<llmodel_gpu_device_cpp[]>(devices.size());
+    for (unsigned i = 0; i < devices.size(); i++) {
+        const auto &dev  =   devices[i];
+              auto &cdev = c_devices[i];
+        cdev.backend  = dev.backend;
+        cdev.index    = dev.index;
+        cdev.type     = dev.type;
+        cdev.heapSize = dev.heapSize;
+        cdev.name     = strdup(dev.name.c_str());
+        cdev.vendor   = strdup(dev.vendor.c_str());
+    }
+
+    return c_devices.get();
+}
+
+bool llmodel_gpu_init_gpu_device_by_string(llmodel_model model, size_t memoryRequired, const char *device)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->initializeGPUDevice(memoryRequired, std::string(device));
+}
+
+bool llmodel_gpu_init_gpu_device_by_struct(llmodel_model model, const llmodel_gpu_device *device)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->initializeGPUDevice(device->index);
+}
+
+bool llmodel_gpu_init_gpu_device_by_int(llmodel_model model, int device)
+{
+    auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->initializeGPUDevice(device);
+}
+
+const char *llmodel_model_backend_name(llmodel_model model)
+{
+    const auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->backendName();
+}
+
+const char *llmodel_model_gpu_device_name(llmodel_model model)
+{
+    const auto *wrapper = static_cast<LLModelWrapper *>(model);
+    return wrapper->llModel->gpuDeviceName();
+}

gpt4all-backend/src/llmodel_shared.cpp

@@ -0,0 +1,401 @@
+#include "llmodel.h"
+
+#include <algorithm>
+#include <cassert>
+#include <cstddef>
+#include <cstdint>
+#include <functional>
+#include <iostream>
+#include <optional>
+#include <regex>
+#include <sstream>
+#include <stdexcept>
+#include <string>
+#include <string_view>
+#include <vector>
+
+namespace ranges = std::ranges;
+
+static bool parsePromptTemplate(const std::string &tmpl, std::vector<std::smatch> &placeholders, std::string &err)
+{
+    static const std::regex placeholderRegex(R"(%[1-2](?![0-9]))");
+
+    auto it = std::sregex_iterator(tmpl.begin(), tmpl.end(), placeholderRegex);
+    placeholders.clear();
+    placeholders.insert(placeholders.end(), it, std::sregex_iterator());
+
+    if (placeholders.size() > 2) {
+        err = "ERROR: expected at most two placeholders, got " + std::to_string(placeholders.size());
+        return false;
+    }
+    if (placeholders.size() >= 1 && placeholders[0].str() != "%1") {
+        err = "ERROR: first placeholder must be %1, got " + placeholders[0].str();
+        return false;
+    }
+    if (placeholders.size() >= 2 && placeholders[1].str() != "%2") {
+        err = "ERROR: second placeholder must be %2, got " + placeholders[1].str();
+        return false;
+    }
+    return true;
+}
+
+void LLModel::prompt(const std::string &prompt,
+                     const std::string &promptTemplate,
+                     std::function<bool(int32_t)> promptCallback,
+                     std::function<bool(int32_t, const std::string&)> responseCallback,
+                     bool allowContextShift,
+                     PromptContext &promptCtx,
+                     bool special,
+                     std::optional<std::string_view> fakeReply)
+{
+    if (!isModelLoaded()) {
+        std::cerr << implementation().modelType() << " ERROR: prompt won't work with an unloaded model!\n";
+        return;
+    }
+
+    if (!supportsCompletion()) {
+        std::string errorMessage = "ERROR: this model does not support text completion or chat!";
+        responseCallback(-1, errorMessage);
+        std::cerr << implementation().modelType() << " " << errorMessage << "\n";
+        return;
+    }
+
+    // sanity checks
+    if (promptCtx.n_past > contextLength()) {
+        std::ostringstream ss;
+        ss << "n_past=" << promptCtx.n_past << " is past end of context length=" << contextLength();
+        throw std::out_of_range(ss.str());
+    }
+    if (promptCtx.n_past > promptCtx.tokens.size()) {
+        std::ostringstream ss;
+        ss << "n_past=" << promptCtx.n_past << " is past end of token cache length=" << promptCtx.tokens.size();
+        throw std::out_of_range(ss.str());
+    }
+
+    promptCtx.n_ctx = contextLength();
+    promptCtx.n_batch = std::min(promptCtx.n_batch, LLMODEL_MAX_PROMPT_BATCH);
+
+    if (promptCtx.n_past < promptCtx.tokens.size())
+        promptCtx.tokens.resize(promptCtx.n_past);
+    m_tokenize_last_token = promptCtx.tokens.empty() ? -1 : promptCtx.tokens.back(); // not serialized
+
+    // parse the prompt template
+    std::vector<std::smatch> placeholders;
+    {
+        std::string err;
+        if (!parsePromptTemplate(promptTemplate, placeholders, err)) {
+            responseCallback(-1, err);
+            std::cerr << err << "\n";
+            return;
+        }
+    }
+
+    // tokenize the user prompt
+    std::vector<Token> embd_inp;
+    if (placeholders.empty()) {
+        // this is unusual, but well-defined
+        std::cerr << __func__ << ": prompt template has no placeholder\n";
+        embd_inp = tokenize(promptTemplate, true);
+    } else {
+        // template: beginning of user prompt
+        const auto &phUser = placeholders[0];
+        std::string userPrefix(phUser.prefix());
+        if (!userPrefix.empty())
+            embd_inp = tokenize(userPrefix, true);
+
+        // user input (shouldn't have special token processing)
+        auto tokens = tokenize(prompt, special);
+        embd_inp.insert(embd_inp.end(), tokens.begin(), tokens.end());
+
+        // template: end of user prompt + start of assistant prompt
+        size_t start = phUser.position() + phUser.length();
+        size_t end = placeholders.size() >= 2 ? placeholders[1].position() : promptTemplate.length();
+        auto userToAsst = promptTemplate.substr(start, end - start);
+        if (!userToAsst.empty()) {
+            tokens = tokenize(userToAsst, true);
+            embd_inp.insert(embd_inp.end(), tokens.begin(), tokens.end());
+        }
+    }
+
+    // decode the user prompt
+    if (!decodePrompt(promptCallback, responseCallback, allowContextShift, promptCtx, embd_inp))
+        return; // error
+
+    // decode the assistant's reply, either generated or spoofed
+    if (!fakeReply) {
+        generateResponse(responseCallback, allowContextShift, promptCtx);
+    } else {
+        embd_inp = tokenize(*fakeReply, false);
+        if (!decodePrompt(promptCallback, responseCallback, allowContextShift, promptCtx, embd_inp, true))
+            return; // error
+    }
+
+    // decode the rest of the prompt template
+    // template: end of assistant prompt
+    std::string asstSuffix;
+    if (placeholders.size() >= 2) {
+        size_t start = placeholders[1].position() + placeholders[1].length();
+        asstSuffix = promptTemplate.substr(start);
+    } else {
+        asstSuffix = "\n\n"; // default to a blank link, good for e.g. Alpaca
+    }
+    if (!asstSuffix.empty()) {
+        embd_inp = tokenize(asstSuffix, true);
+        decodePrompt(promptCallback, responseCallback, allowContextShift, promptCtx, embd_inp);
+    }
+}
+
+// returns false on error
+bool LLModel::decodePrompt(std::function<bool(int32_t)> promptCallback,
+                           std::function<bool(int32_t, const std::string&)> responseCallback,
+                           bool allowContextShift,
+                           PromptContext &promptCtx,
+                           std::vector<Token> embd_inp,
+                           bool isResponse) {
+    if ((int) embd_inp.size() > promptCtx.n_ctx - 4) {
+        // FIXME: (Adam) We should find a way to bubble these strings to the UI level to allow for
+        // translation
+        responseCallback(-1, "Your message was too long and could not be processed. Please try again with something shorter.");
+        std::cerr << implementation().modelType() << " ERROR: The prompt is " << embd_inp.size() <<
+            " tokens and the context window is " << promptCtx.n_ctx << "!\n";
+        return false;
+    }
+
+    // FIXME(jared): There are mitigations for this situation, such as making room before
+    // copying the prompt context, or restoring the KV cache when we restore the prompt
+    // context.
+    if (!allowContextShift && promptCtx.n_past + embd_inp.size() > promptCtx.n_ctx) {
+        std::cerr << "LLModel Warning: Not enough space, n_past=" << promptCtx.n_past << ", n_eval=" << embd_inp.size()
+                  << ", n_ctx=" << promptCtx.n_ctx << "\n";
+        return false;
+    }
+
+    // process the prompt in batches
+    size_t i = 0;
+    while (i < embd_inp.size()) {
+        size_t batch_end = std::min(i + promptCtx.n_batch, embd_inp.size());
+        std::vector<Token> batch(embd_inp.begin() + i, embd_inp.begin() + batch_end);
+
+        // Check if the context has run out...
+        if (promptCtx.n_past + int32_t(batch.size()) > promptCtx.n_ctx) {
+            assert(allowContextShift);
+            shiftContext(promptCtx);
+            assert(promptCtx.n_past + int32_t(batch.size()) <= promptCtx.n_ctx);
+        }
+
+        if (!evalTokens(promptCtx, batch)) {
+            std::cerr << implementation().modelType() << " ERROR: Failed to process prompt\n";
+            return false;
+        }
+
+        size_t tokens = batch_end - i;
+        for (size_t t = 0; t < tokens; ++t) {
+            promptCtx.tokens.push_back(batch.at(t));
+            promptCtx.n_past += 1;
+            Token tok = batch.at(t);
+            bool res = isResponse ? responseCallback(tok, tokenToString(tok)) : promptCallback(tok);
+            if (!res)
+                return false;
+        }
+        i = batch_end;
+    }
+
+    return true;
+}
+
+/*
+ * If string s overlaps with the string key such that some prefix of the key is at the end
+ * of the string, return the position in s where the first match starts. Otherwise, return
+ * std::string::npos. Examples:
+ * s = "bfo",  key = "foo" -> 1
+ * s = "fooa", key = "foo" -> npos
+ */
+static std::string::size_type stringsOverlap(const std::string &s, const std::string &key)
+{
+    if (s.empty() || key.empty())
+        throw std::invalid_argument("arguments to stringsOverlap must not be empty");
+
+    for (int start = std::max(0, int(s.size()) - int(key.size())); start < s.size(); start++) {
+        if (s.compare(start, s.size(), key, 0, s.size() - start) == 0)
+            return start;
+    }
+    return std::string::npos;
+}
+
+void LLModel::generateResponse(std::function<bool(int32_t, const std::string&)> responseCallback,
+                               bool allowContextShift,
+                               PromptContext &promptCtx) {
+    static const char *stopSequences[] {
+        "### Instruction", "### Prompt", "### Response", "### Human", "### Assistant", "### Context",
+    };
+
+    // Don't even start if there is no room
+    if (!promptCtx.n_predict)
+        return;
+    if (!allowContextShift && promptCtx.n_past >= promptCtx.n_ctx) {
+        std::cerr << "LLModel Warning: Not enough space, n_past=" << promptCtx.n_past << ", n_ctx=" << promptCtx.n_ctx
+                  << "\n";
+        return;
+    }
+
+    initSampler(promptCtx);
+
+    std::string cachedResponse;
+    std::vector<Token> cachedTokens;
+    int n_predicted = 0;
+
+    // Predict next tokens
+    for (bool stop = false; !stop;) {
+        // Sample next token
+        std::optional<Token> new_tok = sampleToken();
+        std::string new_piece = tokenToString(new_tok.value());
+        cachedTokens.push_back(new_tok.value());
+        cachedResponse += new_piece;
+
+        auto accept = [this, &promptCtx, &new_tok, allowContextShift]() -> bool {
+            // Shift context if out of space
+            if (promptCtx.n_past >= promptCtx.n_ctx) {
+                (void)allowContextShift;
+                assert(allowContextShift);
+                shiftContext(promptCtx);
+                assert(promptCtx.n_past < promptCtx.n_ctx);
+            }
+
+            // Accept the token
+            Token tok = std::exchange(new_tok, std::nullopt).value();
+            if (!evalTokens(promptCtx, { tok })) {
+                // TODO(jared): raise an exception
+                std::cerr << implementation().modelType() << " ERROR: Failed to predict next token\n";
+                return false;
+            }
+
+            promptCtx.tokens.push_back(tok);
+            promptCtx.n_past += 1;
+            return true;
+        };
+
+        // Check for EOS
+        auto lengthLimit = std::string::npos;
+        for (const auto token : endTokens()) {
+            if (new_tok == token) {
+                stop = true;
+                lengthLimit = cachedResponse.size() - new_piece.size();
+            }
+        }
+
+        if (lengthLimit != std::string::npos) {
+            // EOS matched
+        } else if (!isSpecialToken(new_tok.value())) {
+            // Check if the response contains a stop sequence
+            for (const auto &p : stopSequences) {
+                auto match = cachedResponse.find(p);
+                if (match != std::string::npos) stop = true;
+                lengthLimit = std::min(lengthLimit, match);
+                if (match == 0) break;
+            }
+
+            // Check if the response matches the start of a stop sequence
+            if (lengthLimit == std::string::npos) {
+                for (const auto &p : stopSequences) {
+                    auto match = stringsOverlap(cachedResponse, p);
+                    lengthLimit = std::min(lengthLimit, match);
+                    if (match == 0) break;
+                }
+            }
+        } else if (ranges::find(stopSequences, new_piece) < std::end(stopSequences)) {
+            // Special tokens must exactly match a stop sequence
+            stop = true;
+            lengthLimit = cachedResponse.size() - new_piece.size();
+        }
+
+        // Optionally stop if the context will run out
+        if (!allowContextShift && promptCtx.n_past + cachedTokens.size() >= promptCtx.n_ctx) {
+            std::cerr << "LLModel Warning: Not enough space, n_past=" << promptCtx.n_past << ", n_ctx="
+                      << promptCtx.n_ctx << "\n";
+            stop = true;
+        }
+
+        // Empty the cache, up to the length limit
+        std::string::size_type responseLength = 0;
+        while (!cachedTokens.empty()) {
+            Token tok = cachedTokens.front();
+            std::string piece = tokenToString(tok);
+
+            // Stop if the piece (or part of it) does not fit within the length limit
+            if (responseLength + (stop ? 1 : piece.size()) > lengthLimit)
+                break;
+
+            // Remove token from cache
+            assert(cachedResponse.starts_with(piece));
+            cachedTokens.erase(cachedTokens.begin(), cachedTokens.begin() + 1);
+            cachedResponse.erase(cachedResponse.begin(), cachedResponse.begin() + piece.size());
+
+            // Accept the token, if needed (not cached)
+            if (cachedTokens.empty() && new_tok && !accept())
+                return;
+
+            // Send the token
+            if (!responseCallback(tok, piece) || ++n_predicted >= promptCtx.n_predict) {
+                stop = true;
+                break;
+            }
+
+            // FIXME(jared): we could avoid printing partial stop sequences if we didn't have to
+            // output token IDs and could cache a partial token for the next prompt call
+            responseLength += piece.size();
+        }
+        assert(cachedTokens.empty() == cachedResponse.empty());
+
+        // Accept the token, if needed (in cache)
+        if (new_tok) {
+            assert(!cachedTokens.empty() && cachedTokens.back() == new_tok);
+            if (stop) {
+                cachedTokens.pop_back();
+            } else if (!accept()) {
+                return;
+            }
+        }
+    }
+
+    auto &tokens = promptCtx.tokens;
+    if (tokens.size() < cachedTokens.size()) {
+        /* This is theoretically possible if the longest stop sequence is greater than
+         * n_ctx * contextErase tokens. */
+        throw std::runtime_error("shifted too much context, can't go back");
+    }
+
+    auto discard_start = tokens.end() - cachedTokens.size();
+    assert(std::equal(discard_start, tokens.end(), cachedTokens.begin()));
+    tokens.erase(discard_start, tokens.end());
+
+    promptCtx.n_past -= cachedTokens.size();
+}
+
+void LLModel::embed(
+    const std::vector<std::string> &texts, float *embeddings, std::optional<std::string> prefix, int dimensionality,
+    size_t *tokenCount, bool doMean, bool atlas, EmbedCancelCallback *cancelCb
+) {
+    (void)texts;
+    (void)embeddings;
+    (void)prefix;
+    (void)dimensionality;
+    (void)tokenCount;
+    (void)doMean;
+    (void)atlas;
+    (void)cancelCb;
+    throw std::logic_error(std::string(implementation().modelType()) + " does not support embeddings");
+}
+
+void LLModel::embed(
+    const std::vector<std::string> &texts, float *embeddings, bool isRetrieval, int dimensionality, size_t *tokenCount,
+    bool doMean, bool atlas
+) {
+    (void)texts;
+    (void)embeddings;
+    (void)isRetrieval;
+    (void)dimensionality;
+    (void)tokenCount;
+    (void)doMean;
+    (void)atlas;
+    throw std::logic_error(std::string(implementation().modelType()) + " does not support embeddings");
+}

gpt4all-backend/src/utils.h

@@ -0,0 +1,17 @@
+#pragma once
+
+#include <cassert>
+
+#ifdef NDEBUG
+#   ifdef __has_builtin
+#       if __has_builtin(__builtin_unreachable)
+#           define UNREACHABLE() __builtin_unreachable()
+#       else
+#           define UNREACHABLE() do {} while (0)
+#       endif
+#   else
+#       define UNREACHABLE() do {} while (0)
+#   endif
+#else
+#   define UNREACHABLE() assert(!"Unreachable statement was reached")
+#endif

gpt4all-bindings/README.md

@@ -0,0 +1,21 @@
+# GPT4All Language Bindings
+These are the language bindings for the GPT4All backend. They provide functionality to load GPT4All models (and other llama.cpp models), generate text, and (in the case of the Python bindings) embed text as a vector representation.
+
+See their respective folders for language-specific documentation.
+
+### Languages
+- [Python](https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/python) (Nomic official, maintained by [@cebtenzzre](https://github.com/cebtenzzre))
+- [Node.js/Typescript](https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/typescript) (community, maintained by [@jacoobes](https://github.com/jacoobes) and [@iimez](https://github.com/iimez))
+
+<br/>
+<br/>
+
+<details><summary><b>Archived Bindings</b></summary>
+<br/>
+
+The following bindings have been removed from this repository due to lack of maintenance. If adopted, they can be brought back&mdash;feel free to message a developer on Dicsord if you are interested in maintaining one of them. Below are links to their last available version (not necessarily the last working version).
+- C#: [41c9013f](https://github.com/nomic-ai/gpt4all/tree/41c9013fa46a194b3e4fee6ced1b9d1b65e177ac/gpt4all-bindings/csharp)
+- Java: [41c9013f](https://github.com/nomic-ai/gpt4all/tree/41c9013fa46a194b3e4fee6ced1b9d1b65e177ac/gpt4all-bindings/java)
+- Go: [41c9013f](https://github.com/nomic-ai/gpt4all/tree/41c9013fa46a194b3e4fee6ced1b9d1b65e177ac/gpt4all-bindings/golang)
+
+</details>

gpt4all-bindings/cli/README.md

@@ -0,0 +1,43 @@
+# GPT4All Command-Line Interface (CLI)
+
+GPT4All on the command-line.
+
+More details on the [wiki](https://github.com/nomic-ai/gpt4all/wiki/Python-CLI).
+
+## Quickstart
+
+The CLI is based on the `gpt4all` Python bindings and the `typer` package.
+
+The following shows one way to get started with the CLI, the documentation has more information.
+Typically, you will want to replace `python` with `python3` on _Unix-like_ systems and `py -3` on
+_Windows_. Also, it's assumed you have all the necessary Python components already installed.
+
+The CLI is a self-contained Python script named [app.py] ([download][app.py-download]). As long as
+its package dependencies are present, you can download and run it from wherever you like.
+
+[app.py]: https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/cli/app.py
+[app.py-download]: https://raw.githubusercontent.com/nomic-ai/gpt4all/main/gpt4all-bindings/cli/app.py
+
+```shell
+# optional but recommended: create and use a virtual environment
+python -m venv gpt4all-cli
+```
+_Windows_ and _Unix-like_ systems differ slightly in how you activate a _virtual environment_:
+- _Unix-like_, typically: `. gpt4all-cli/bin/activate`
+- _Windows_: `gpt4all-cli\Scripts\activate`
+
+Then:
+```shell
+# pip-install the necessary packages; omit '--user' if using a virtual environment
+python -m pip install --user --upgrade gpt4all typer
+# run the CLI
+python app.py repl
+```
+By default, it will automatically download the `Mistral Instruct` model to `.cache/gpt4all/` in your
+user directory, if necessary.
+
+If you have already saved a model beforehand, specify its path with the `-m`/`--model` argument,
+for example:
+```shell
+python app.py repl --model /home/user/my-gpt4all-models/mistral-7b-instruct-v0.1.Q4_0.gguf
+```

gpt4all-bindings/cli/app.py

@@ -0,0 +1,187 @@
+#!/usr/bin/env python3
+"""GPT4All CLI
+
+The GPT4All CLI is a self-contained script based on the `gpt4all` and `typer` packages. It offers a
+REPL to communicate with a language model similar to the chat GUI application, but more basic.
+"""
+
+import importlib.metadata
+import io
+import sys
+from collections import namedtuple
+from typing_extensions import Annotated
+
+import typer
+from gpt4all import GPT4All
+
+
+MESSAGES = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "Hello there."},
+    {"role": "assistant", "content": "Hi, how can I help you?"},
+]
+
+SPECIAL_COMMANDS = {
+    "/reset": lambda messages: messages.clear(),
+    "/exit": lambda _: sys.exit(),
+    "/clear": lambda _: print("\n" * 100),
+    "/help": lambda _: print("Special commands: /reset, /exit, /help and /clear"),
+}
+
+VersionInfo = namedtuple('VersionInfo', ['major', 'minor', 'micro'])
+VERSION_INFO = VersionInfo(1, 0, 2)
+VERSION = '.'.join(map(str, VERSION_INFO))  # convert to string form, like: '1.2.3'
+
+CLI_START_MESSAGE = f"""
+    
+ ██████  ██████  ████████ ██   ██  █████  ██      ██      
+██       ██   ██    ██    ██   ██ ██   ██ ██      ██      
+██   ███ ██████     ██    ███████ ███████ ██      ██      
+██    ██ ██         ██         ██ ██   ██ ██      ██      
+ ██████  ██         ██         ██ ██   ██ ███████ ███████ 
+                                                          
+
+Welcome to the GPT4All CLI! Version {VERSION}
+Type /help for special commands.
+                                                    
+"""
+
+# create typer app
+app = typer.Typer()
+
+@app.command()
+def repl(
+    model: Annotated[
+        str,
+        typer.Option("--model", "-m", help="Model to use for chatbot"),
+    ] = "mistral-7b-instruct-v0.1.Q4_0.gguf",
+    n_threads: Annotated[
+        int,
+        typer.Option("--n-threads", "-t", help="Number of threads to use for chatbot"),
+    ] = None,
+    device: Annotated[
+        str,
+        typer.Option("--device", "-d", help="Device to use for chatbot, e.g. gpu, amd, nvidia, intel. Defaults to CPU."),
+    ] = None,
+):
+    """The CLI read-eval-print loop."""
+    gpt4all_instance = GPT4All(model, device=device)
+
+    # if threads are passed, set them
+    if n_threads is not None:
+        num_threads = gpt4all_instance.model.thread_count()
+        print(f"\nAdjusted: {num_threads} →", end="")
+
+        # set number of threads
+        gpt4all_instance.model.set_thread_count(n_threads)
+
+        num_threads = gpt4all_instance.model.thread_count()
+        print(f" {num_threads} threads", end="", flush=True)
+    else:
+        print(f"\nUsing {gpt4all_instance.model.thread_count()} threads", end="")
+
+    print(CLI_START_MESSAGE)
+
+    use_new_loop = False
+    try:
+        version = importlib.metadata.version('gpt4all')
+        version_major = int(version.split('.')[0])
+        if version_major >= 1:
+            use_new_loop = True
+    except:
+        pass  # fall back to old loop
+    if use_new_loop:
+        _new_loop(gpt4all_instance)
+    else:
+        _old_loop(gpt4all_instance)
+
+
+def _old_loop(gpt4all_instance):
+    while True:
+        message = input(" ⇢  ")
+
+        # Check if special command and take action
+        if message in SPECIAL_COMMANDS:
+            SPECIAL_COMMANDS[message](MESSAGES)
+            continue
+
+        # if regular message, append to messages
+        MESSAGES.append({"role": "user", "content": message})
+
+        # execute chat completion and ignore the full response since 
+        # we are outputting it incrementally
+        full_response = gpt4all_instance.chat_completion(
+            MESSAGES,
+            # preferential kwargs for chat ux
+            logits_size=0,
+            tokens_size=0,
+            n_past=0,
+            n_ctx=0,
+            n_predict=200,
+            top_k=40,
+            top_p=0.9,
+            min_p=0.0,
+            temp=0.9,
+            n_batch=9,
+            repeat_penalty=1.1,
+            repeat_last_n=64,
+            context_erase=0.0,
+            # required kwargs for cli ux (incremental response)
+            verbose=False,
+            streaming=True,
+        )
+        # record assistant's response to messages
+        MESSAGES.append(full_response.get("choices")[0].get("message"))
+        print() # newline before next prompt
+
+
+def _new_loop(gpt4all_instance):
+    with gpt4all_instance.chat_session():
+        while True:
+            message = input(" ⇢  ")
+
+            # Check if special command and take action
+            if message in SPECIAL_COMMANDS:
+                SPECIAL_COMMANDS[message](MESSAGES)
+                continue
+
+            # if regular message, append to messages
+            MESSAGES.append({"role": "user", "content": message})
+
+            # execute chat completion and ignore the full response since 
+            # we are outputting it incrementally
+            response_generator = gpt4all_instance.generate(
+                message,
+                # preferential kwargs for chat ux
+                max_tokens=200,
+                temp=0.9,
+                top_k=40,
+                top_p=0.9,
+                min_p=0.0,
+                repeat_penalty=1.1,
+                repeat_last_n=64,
+                n_batch=9,
+                # required kwargs for cli ux (incremental response)
+                streaming=True,
+            )
+            response = io.StringIO()
+            for token in response_generator:
+                print(token, end='', flush=True)
+                response.write(token)
+
+            # record assistant's response to messages
+            response_message = {'role': 'assistant', 'content': response.getvalue()}
+            response.close()
+            gpt4all_instance.current_chat_session.append(response_message)
+            MESSAGES.append(response_message)
+            print() # newline before next prompt
+
+
+@app.command()
+def version():
+    """The CLI version command."""
+    print(f"gpt4all-cli v{VERSION}")
+
+
+if __name__ == "__main__":
+    app()

gpt4all-bindings/cli/developer_notes.md

@@ -0,0 +1,25 @@
+# Developing the CLI
+## Documentation
+Documentation can be found in three places:
+- `app.py` docstrings & comments
+- a Readme: `gpt4all-bindings/cli/README.md`
+- the actual CLI documentation: `gpt4all-bindings/python/docs/gpt4all_cli.md`
+
+The _docstrings_ are meant for programmatic use. Since the CLI is primarily geared towards users and
+not to build on top, they're kept terse.
+
+The _Readme_ is mostly meant for users and includes:
+- a link to the _CLI documentation_ (on the [website])
+- a Quickstart section with some guidance on how to get started with a sane setup
+
+The _CLI documentation_ and other documentation are located in the above mentioned `docs/` folder.
+They're in Markdown format and built for the [website]. Of the three, they should be the most
+detailed.
+
+[website]: https://docs.gpt4all.io/gpt4all_cli.html
+
+
+## Versioning
+The version number should now follow the `gpt4all` PyPI package, so compatibility is more clear.
+
+The one place to change it is the `namedtuple` called `VERSION_INFO`.

gpt4all-bindings/python/.gitignore

@@ -0,0 +1,164 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# Cython
+/*.c
+*DO_NOT_MODIFY/

gpt4all-bindings/python/.isort.cfg

@@ -0,0 +1,7 @@
+[settings]
+known_third_party=geopy,nltk,np,numpy,pandas,pysbd,fire,torch
+
+line_length=120
+include_trailing_comma=True
+multi_line_output=3
+use_parentheses=True

gpt4all-bindings/python/CHANGELOG.md

@@ -0,0 +1,72 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## [Unreleased]
+
+### Added
+- Warn on Windows if the Microsoft Visual C++ runtime libraries are not found ([#2920](https://github.com/nomic-ai/gpt4all/pull/2920))
+
+### Changed
+- Rebase llama.cpp on latest upstream as of September 26th ([#2998](https://github.com/nomic-ai/gpt4all/pull/2998))
+- Change the error message when a message is too long ([#3004](https://github.com/nomic-ai/gpt4all/pull/3004))
+- Fix CalledProcessError on Intel Macs since v2.8.0 ([#3045](https://github.com/nomic-ai/gpt4all/pull/3045))
+
+## [2.8.2] - 2024-08-14
+
+### Fixed
+- Fixed incompatibility with Python 3.8 since v2.7.0 and Python <=3.11 since v2.8.1 ([#2871](https://github.com/nomic-ai/gpt4all/pull/2871))
+
+## [2.8.1] - 2024-08-13
+
+### Added
+- Use greedy sampling when temperature is set to zero ([#2854](https://github.com/nomic-ai/gpt4all/pull/2854))
+
+### Changed
+- Search for pip-installed CUDA 11 as well as CUDA 12 ([#2802](https://github.com/nomic-ai/gpt4all/pull/2802))
+- Stop shipping CUBINs to reduce wheel size ([#2802](https://github.com/nomic-ai/gpt4all/pull/2802))
+- Use llama\_kv\_cache ops to shift context faster ([#2781](https://github.com/nomic-ai/gpt4all/pull/2781))
+- Don't stop generating at end of context ([#2781](https://github.com/nomic-ai/gpt4all/pull/2781))
+
+### Fixed
+- Make reverse prompt detection work more reliably and prevent it from breaking output ([#2781](https://github.com/nomic-ai/gpt4all/pull/2781))
+- Explicitly target macOS 12.6 in CI to fix Metal compatibility on older macOS ([#2849](https://github.com/nomic-ai/gpt4all/pull/2849))
+- Do not initialize Vulkan driver when only using CPU ([#2843](https://github.com/nomic-ai/gpt4all/pull/2843))
+- Fix a segfault on exit when using CPU mode on Linux with NVIDIA and EGL ([#2843](https://github.com/nomic-ai/gpt4all/pull/2843))
+
+## [2.8.0] - 2024-08-05
+
+### Added
+- Support GPT-NeoX, Gemma 2, OpenELM, ChatGLM, and Jais architectures (all with Vulkan support) ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
+- Enable Vulkan support for StarCoder2, XVERSE, Command R, and OLMo ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
+- Support DeepSeek-V2 architecture (no Vulkan support) ([#2702](https://github.com/nomic-ai/gpt4all/pull/2702))
+- Add Llama 3.1 8B Instruct to models3.json (by [@3Simplex](https://github.com/3Simplex) in [#2731](https://github.com/nomic-ai/gpt4all/pull/2731) and [#2732](https://github.com/nomic-ai/gpt4all/pull/2732))
+- Support Llama 3.1 RoPE scaling ([#2758](https://github.com/nomic-ai/gpt4all/pull/2758))
+- Add Qwen2-1.5B-Instruct to models3.json (by [@ThiloteE](https://github.com/ThiloteE) in [#2759](https://github.com/nomic-ai/gpt4all/pull/2759))
+- Detect use of a Python interpreter under Rosetta for a clearer error message ([#2793](https://github.com/nomic-ai/gpt4all/pull/2793))
+
+### Changed
+- Build against CUDA 11.8 instead of CUDA 12 for better compatibility with older drivers ([#2639](https://github.com/nomic-ai/gpt4all/pull/2639))
+- Update llama.cpp to commit 87e397d00 from July 19th ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
+
+### Removed
+- Remove unused internal llmodel\_has\_gpu\_device ([#2409](https://github.com/nomic-ai/gpt4all/pull/2409))
+- Remove support for GPT-J models ([#2676](https://github.com/nomic-ai/gpt4all/pull/2676), [#2693](https://github.com/nomic-ai/gpt4all/pull/2693))
+
+### Fixed
+- Fix debug mode crash on Windows and undefined behavior in LLamaModel::embedInternal ([#2467](https://github.com/nomic-ai/gpt4all/pull/2467))
+- Fix CUDA PTX errors with some GPT4All builds ([#2421](https://github.com/nomic-ai/gpt4all/pull/2421))
+- Fix mishandling of inputs greater than n\_ctx tokens after [#1970](https://github.com/nomic-ai/gpt4all/pull/1970) ([#2498](https://github.com/nomic-ai/gpt4all/pull/2498))
+- Fix crash when Kompute falls back to CPU ([#2640](https://github.com/nomic-ai/gpt4all/pull/2640))
+- Fix several Kompute resource management issues ([#2694](https://github.com/nomic-ai/gpt4all/pull/2694))
+- Fix crash/hang when some models stop generating, by showing special tokens ([#2701](https://github.com/nomic-ai/gpt4all/pull/2701))
+- Fix several backend issues ([#2778](https://github.com/nomic-ai/gpt4all/pull/2778))
+  - Restore leading space removal logic that was incorrectly removed in [#2694](https://github.com/nomic-ai/gpt4all/pull/2694)
+  - CUDA: Cherry-pick llama.cpp DMMV cols requirement fix that caused a crash with long conversations since [#2694](https://github.com/nomic-ai/gpt4all/pull/2694)
+
+[Unreleased]: https://github.com/nomic-ai/gpt4all/compare/python-v2.8.2...HEAD
+[2.8.2]: https://github.com/nomic-ai/gpt4all/compare/python-v2.8.1...python-v2.8.2
+[2.8.1]: https://github.com/nomic-ai/gpt4all/compare/python-v2.8.0...python-v2.8.1
+[2.8.0]: https://github.com/nomic-ai/gpt4all/compare/python-v2.7.0...python-v2.8.0

gpt4all-bindings/python/LICENSE.txt

@@ -0,0 +1,19 @@
+Copyright (c) 2023 Nomic, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gpt4all-bindings/python/MANIFEST.in

@@ -0,0 +1 @@
+recursive-include gpt4all/llmodel_DO_NOT_MODIFY *

gpt4all-bindings/python/README.md

@@ -0,0 +1,93 @@
+# Python GPT4All
+
+This package contains a set of Python bindings around the `llmodel` C-API.
+
+Package on PyPI: https://pypi.org/project/gpt4all/
+
+## Documentation
+https://docs.gpt4all.io/gpt4all_python.html
+
+## Installation
+
+The easiest way to install the Python bindings for GPT4All is to use pip:
+
+```
+pip install gpt4all
+```
+
+This will download the latest version of the `gpt4all` package from PyPI.
+
+## Local Build
+
+As an alternative to downloading via pip, you may build the Python bindings from source.
+
+### Prerequisites
+
+You will need a compiler. On Windows, you should install Visual Studio with the C++ Development components. On macOS, you will need the full version of Xcode—Xcode Command Line Tools lacks certain required tools. On Linux, you will need a GCC or Clang toolchain with C++ support.
+
+On Windows and Linux, building GPT4All with full GPU support requires the [Vulkan SDK](https://vulkan.lunarg.com/sdk/home) and the latest [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads).
+
+### Building the python bindings
+
+1. Clone GPT4All and change directory:
+```
+git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
+cd gpt4all/gpt4all-backend
+```
+
+2. Build the backend.
+
+If you are using Windows and have Visual Studio installed:
+```
+cmake -B build
+cmake --build build --parallel --config RelWithDebInfo
+```
+
+For all other platforms:
+```
+cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
+cmake --build build --parallel
+```
+
+`RelWithDebInfo` is a good default, but you can also use `Release` or `Debug` depending on the situation.
+
+2. Install the Python package:
+```
+cd ../gpt4all-bindings/python
+pip install -e .
+```
+
+## Usage
+
+Test it out! In a Python script or console:
+
+```python
+from gpt4all import GPT4All
+model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
+output = model.generate("The capital of France is ", max_tokens=3)
+print(output)
+```
+
+
+GPU Usage
+```python
+from gpt4all import GPT4All
+model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu') # device='amd', device='intel'
+output = model.generate("The capital of France is ", max_tokens=3)
+print(output)
+```
+
+## Troubleshooting a Local Build
+- If you're on Windows and have compiled with a MinGW toolchain, you might run into an error like:
+  ```
+  FileNotFoundError: Could not find module '<...>\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\libllmodel.dll'
+  (or one of its dependencies). Try using the full path with constructor syntax.
+  ```
+  The key phrase in this case is _"or one of its dependencies"_. The Python interpreter you're using
+  probably doesn't see the MinGW runtime dependencies. At the moment, the following three are required:
+  `libgcc_s_seh-1.dll`, `libstdc++-6.dll` and `libwinpthread-1.dll`. You should copy them from MinGW
+  into a folder where Python will see them, preferably next to `libllmodel.dll`.
+
+- Note regarding the Microsoft toolchain: Compiling with MSVC is possible, but not the official way to
+  go about it at the moment. MSVC doesn't produce DLLs with a `lib` prefix, which the bindings expect.
+  You'd have to amend that yourself.

gpt4all-bindings/python/docs/assets/ubuntu.svg

@@ -0,0 +1,5 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg xmlns="http://www.w3.org/2000/svg" width="285" height="285" viewBox="-142.5 -142.5 285 285" xmlns:xlink="http://www.w3.org/1999/xlink">
+<circle fill="#FFFFFF" r="141.732"/><g id="U" fill="#DD4814"><circle cx="-96.3772" r="18.9215"/>
+<path d="M-45.6059,68.395C-62.1655,57.3316-74.4844,40.4175-79.6011,20.6065-73.623,15.7354-69.8047,8.3164-69.8047,0-69.8047-8.3164-73.623-15.7354-79.6011-20.6065-74.4844-40.4175-62.1655-57.3316-45.6059-68.395L-31.7715-45.2212C-45.9824-35.2197-55.2754-18.7026-55.2754,0-55.2754,18.7026-45.9824,35.2197-31.7715,45.2212Z"/></g>
+<use xlink:href="#U" transform="rotate(120)"/><use xlink:href="#U" transform="rotate(240)"/></svg>

gpt4all-bindings/python/docs/css/custom.css

@@ -0,0 +1,5 @@
+.md-content h1,
+.md-content h2 {
+  margin-top: 0.5em;
+  margin-bottom: 0.5em;
+}

gpt4all-bindings/python/docs/gpt4all_api_server/home.md

@@ -0,0 +1,86 @@
+# GPT4All API Server
+
+GPT4All provides a local API server that allows you to run LLMs over an HTTP API. 
+
+## Key Features
+
+- **Local Execution**: Run models on your own hardware for privacy and offline use.
+- **LocalDocs Integration**: Run the API with relevant text snippets provided to your LLM from a [LocalDocs collection](../gpt4all_desktop/localdocs.md).
+- **OpenAI API Compatibility**: Use existing OpenAI-compatible clients and tools with your local models.
+
+## Activating the API Server
+
+1. Open the GPT4All Chat Desktop Application.
+2. Go to `Settings` > `Application` and scroll down to `Advanced`.
+3. Check the box for the `"Enable Local API Server"` setting.
+4. The server listens on port 4891 by default. You can choose another port number in the `"API Server Port"` setting.
+
+## Connecting to the API Server
+
+The base URL used for the API server is `http://localhost:4891/v1` (or `http://localhost:<PORT_NUM>/v1` if you are using a different port number). 
+
+The server only accepts HTTP connections (not HTTPS) and only listens on localhost (127.0.0.1) (e.g. not to the IPv6 localhost address `::1`.)
+
+## Examples
+
+!!! note "Example GPT4All API calls"
+
+    === "cURL"
+
+        ```bash
+        curl -X POST http://localhost:4891/v1/chat/completions -d '{
+        "model": "Phi-3 Mini Instruct",
+        "messages": [{"role":"user","content":"Who is Lionel Messi?"}],
+        "max_tokens": 50,
+        "temperature": 0.28
+        }'
+        ```
+
+    === "PowerShell"
+
+        ```powershell
+        Invoke-WebRequest -URI http://localhost:4891/v1/chat/completions -Method POST -ContentType application/json -Body '{
+        "model": "Phi-3 Mini Instruct",
+        "messages": [{"role":"user","content":"Who is Lionel Messi?"}],
+        "max_tokens": 50,
+        "temperature": 0.28
+        }'
+        ```
+
+## API Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/v1/models` | List available models |
+| GET | `/v1/models/<name>` | Get details of a specific model |
+| POST | `/v1/completions` | Generate text completions |
+| POST | `/v1/chat/completions` | Generate chat completions |
+
+## LocalDocs Integration
+
+You can use LocalDocs with the API server:
+
+1. Open the Chats view in the GPT4All application.
+2. Scroll to the bottom of the chat history sidebar.
+3. Select the server chat (it has a different background color).
+4. Activate LocalDocs collections in the right sidebar.
+
+(Note: LocalDocs can currently only be activated through the GPT4All UI, not via the API itself).
+
+Now, your API calls to your local LLM will have relevant references from your LocalDocs collection retrieved and placed in the input message for the LLM to respond to.
+
+The references retrieved for your API call can be accessed in the API response object at 
+
+`response["choices"][0]["references"]`
+
+The data included in the `references` are:
+
+- `text`: the actual text content from the snippet that was extracted from the reference document
+
+- `author`: the author of the reference document (if available)
+
+- `date`: the date of creation of the reference document (if available)
+
+- `page`: the page number the snippet is from (only available for PDF documents for now)
+
+- `title`: the title of the reference document (if available)

gpt4all-bindings/python/docs/gpt4all_desktop/chats.md

@@ -0,0 +1,124 @@
+# Chats
+
+Chats are conversations with [language models](models.md) that run locally on your device.
+
+!!! note "Example Chats"
+
+    === "Llama 3"
+        Download `Llama 3` and prompt:
+        ```
+        explain why the sky is blue in a way that is correct and makes sense to a child
+        ```
+
+        With the default sampling settings, you should see text resembling the following:
+
+        ```
+        What a great question!
+        So, you know how we can see different colors like red, yellow, green, and orange? Well, when sunlight enters Earth's atmosphere, it starts to interact with tiny particles called molecules of gases like nitrogen (N2) and oxygen (02). These molecules are really small, but they're all around us!
+        Now, here's the cool part: these molecules scatter, or bounce off, shorter wavelengths of light more than longer wavelengths. Think of it like a game of pool where the cue ball hits other balls with different sizes and speeds.
+        The shortest wavelength of visible light is violet (about 400 nanometers), followed by blue (around 450-495 nanometers). These shorter wavelengths are scattered in all directions, kind of like how a basketball player might bounce the ball around on the court. This scattering effect makes the sky appear blue to our eyes!
+        Here's why it doesn't look as bright or intense as a blue crayon: there are many more molecules in the atmosphere that scatter longer wavelengths (like red and orange) than shorter ones, so those colors get spread out and become less noticeable.
+        To sum it up:
+        1. Sunlight enters Earth's atmosphere.
+        2. Tiny gas molecule particles interact with light, scattering shorter wavelengths like blue and violet.
+        3. This scattered blue light reaches our eyes from all directions, making the sky appear blue!
+        Now, go outside and enjoy that beautiful blue sky!
+        ```
+
+    === "Nous Hermes 2 Mistral DPO"
+        Download `Nous Hermes 2 Mistral DPO` and prompt:
+        ```
+        write me a react app i can run from the command line to play a quick game
+        ```
+
+        With the default sampling settings, you should see text and code blocks resembling the following:
+        
+        ```
+        I'll provide you with a simple React app that allows you to play a guessing game. This app will generate a random number between 1 and 10, and ask you to guess it. You can run this app locally using Create React App. Follow the steps below:
+        Open your terminal or command line interface.
+        Navigate to the directory where you want to create the project (e.g., cd Documents/Projects).
+        Run the following commands:
+        ```
+
+        ```bash
+        npx create-react-app guessing-game
+        cd guessing-game
+        ```
+
+        ```
+        Open the project in your preferred code editor by running code . (for VS Code) or any other command to open the folder with your text editor.
+        Replace the content of the src/App.js file with the following code:
+        ```
+
+        ```javascript
+        import React, { useState } from 'react';
+
+        function App() {
+        const [guess, setGuess] = useState('');
+        const [message, setMessage] = useState('');
+        const [numberToGuess, setNumberToGuess] = useState(Math.floor(Math.random() * 10) + 1);
+
+        function handleChange(e) {
+            setGuess(e.target.value);
+        }
+
+        function handleSubmit(e) {
+            e.preventDefault();
+            if (parseInt(guess) === numberToGuess) {
+            setMessage('You guessed it right!');
+            } else {
+            setMessage(`Wrong guess, the correct number was ${numberToGuess}. Try again next time.`);
+            }
+        }
+
+        return (
+            <div>
+            <h1>Guess a Number</h1>
+            <p>{message}</p>
+            <form onSubmit={handleSubmit}>
+                <input type="number" value={guess} onChange={handleChange} />
+                <button type="submit">Submit</button>
+            </form>
+            </div>
+        );
+        }
+
+        export default App;
+        ```
+
+        ```
+        Save the file and go back to your terminal or command line interface. Run npm start. This will start a local development server, and you can access the app in your browser at http://localhost:3000.
+        Play the guessing game by entering a number between 1 and 10 into the input field and clicking "Submit". The app will tell you if your guess is correct or not.
+        Remember that this is just a simple example, and you can expand upon it to make the game more interesting with additional features like high scores, multiple difficulty levels, etc.
+        ```
+
+## New Chat
+
+Choose a model with the dropdown at the top of the Chats page
+
+If you don't have any models, [download one](models.md#download-models). Once you have models, you can start chats by loading your default model, which you can configure in [settings](settings.md#application-settings)
+
+![Choose a model](../assets/three_model_options.png)
+
+## LocalDocs
+
+Open the [LocalDocs](localdocs.md) panel with the button in the top-right corner to bring your files into the chat. With LocalDocs, your chats are enhanced with semantically related snippets from your files included in the model's context.
+
+![Open LocalDocs](../assets/open_local_docs.png)
+
+## Chat History
+
+View your chat history with the button in the top-left corner of the Chats page.
+
+<table>
+<tr>
+    <td>
+    <img src="../assets/closed_chat_panel.png" alt="Close chats" style="width:100%">
+    </td>
+    <td>
+    <img src="../assets/open_chat_panel.png" alt="Open chats" style="width:100%">
+    </td>
+</tr>
+</table>
+
+You can change a chat name or delete it from your chat history at any time.

gpt4all-bindings/python/docs/gpt4all_desktop/cookbook/use-local-ai-models-to-privately-chat-with-Obsidian.md

@@ -0,0 +1,109 @@
+# Using GPT4All to Privately Chat with your Obsidian Vault
+
+Obsidian for Desktop is a powerful management and note-taking software designed to create and organize markdown notes. This tutorial allows you to sync and access your Obsidian note files directly on your computer. By connecting it to LocalDocs, you can integrate these files into your LLM chats for private access and enhanced context.
+
+## Download Obsidian for Desktop
+
+!!! note "Download Obsidian for Desktop"
+
+      1. **Download Obsidian for Desktop**:
+         - Visit the [Obsidian website](https://obsidian.md) and create an account account.
+         - Click the Download button in the center of the homepage
+         - For more help with installing Obsidian see [Getting Started with Obsidian](https://help.obsidian.md/Getting+started/Download+and+install+Obsidian)
+      
+      2. **Set Up Obsidian**:
+         - Launch Obsidian from your Applications folder (macOS), Start menu (Windows), or equivalent location (Linux).
+         - On the welcome screen, you can either create a new vault (a collection of notes) or open an existing one.
+         - To create a new vault, click Create a new vault, name your vault, choose a location on your computer, and click Create.
+   
+   
+      3. **Sign in and Sync**:
+            - Once installed, you can start adding and organizing notes.
+            - Choose the folders you want to sync to your computer.
+   
+
+
+## Connect Obsidian to LocalDocs
+
+!!! note "Connect Obsidian to LocalDocs"
+
+      1. **Open LocalDocs**:
+         - Navigate to the LocalDocs feature within GPT4All.
+
+         <table>
+            <tr>
+               <td>
+                  <!-- Screenshot of LocalDocs interface -->
+                  <img width="1348" alt="LocalDocs interface" src="https://github.com/nomic-ai/gpt4all/assets/132290469/d8fb2d79-2063-45d4-bcce-7299fb75b144">
+               </td>
+            </tr>
+         </table>
+   
+      2. **Add Collection**:
+         - Click on **+ Add Collection** to begin linking your Obsidian Vault.
+      
+         <table>
+            <tr>
+               <td>
+                  <!-- Screenshot of adding collection in LocalDocs -->
+                  <img width="1348" alt="Screenshot of adding collection" src="https://raw.githubusercontent.com/nomic-ai/gpt4all/124ef867a9d9afd9e14d3858cd77bce858f79773/gpt4all-bindings/python/docs/assets/obsidian_adding_collection.png">
+               </td>
+            </tr>
+         </table>
+   
+         - Name your collection
+   
+   
+      3. **Create Collection**:
+         - Click **Create Collection** to initiate the embedding process. Progress will be displayed within the LocalDocs interface.
+   
+      4. **Access Files in Chats**:
+         - Load a model to chat with your files (Llama 3 Instruct is the fastest)
+         - In your chat, open 'LocalDocs' with the button in the top-right corner to provide context from your synced Obsidian notes.
+      
+         <table>
+            <tr>
+               <td>
+                  <!-- Screenshot of accessing LocalDocs in chats -->
+                  <img width="1447" alt="Accessing LocalDocs in chats" src="https://raw.githubusercontent.com/nomic-ai/gpt4all/124ef867a9d9afd9e14d3858cd77bce858f79773/gpt4all-bindings/python/docs/assets/obsidian_docs.png">
+               </td>
+            </tr>
+         </table>
+   
+      5. **Interact With Your Notes:**
+         - Use the model to interact with your files
+         <table>
+            <tr>
+               <td>
+                  <!-- Screenshot of interacting sources -->
+                  <img width="662" alt="osbsidian user interaction" src="https://raw.githubusercontent.com/nomic-ai/gpt4all/124ef867a9d9afd9e14d3858cd77bce858f79773/gpt4all-bindings/python/docs/assets/osbsidian_user_interaction.png">
+               </td>
+            </tr>
+         </table>
+         <table>
+            <tr>
+               <td>
+                  <!-- Screenshot of viewing sources -->
+                  <img width="662" alt="osbsidian GPT4ALL response" src="https://raw.githubusercontent.com/nomic-ai/gpt4all/124ef867a9d9afd9e14d3858cd77bce858f79773/gpt4all-bindings/python/docs/assets/obsidian_response.png">
+               </td>
+            </tr>
+         </table>
+   
+      6. **View Referenced Files**:
+         - Click on **Sources** below LLM responses to see which Obsidian Notes were referenced.
+      
+         <table>
+            <tr>
+               <td>
+                  <!-- Referenced Files  -->
+                  <img width="643" alt="Referenced Files" src="https://raw.githubusercontent.com/nomic-ai/gpt4all/124ef867a9d9afd9e14d3858cd77bce858f79773/gpt4all-bindings/python/docs/assets/obsidian_sources.png">
+               </td>
+            </tr>
+         </table>
+
+## How It Works
+
+Obsidian for Desktop syncs your Obsidian notes to your computer, while LocalDocs integrates these files into your LLM chats using embedding models. These models find semantically similar snippets from your files to enhance the context of your interactions.
+
+To learn more about embedding models and explore further, refer to the [Nomic Python SDK documentation](https://docs.nomic.ai/atlas/capabilities/embeddings).
+

gpt4all-bindings/python/docs/gpt4all_desktop/cookbook/use-local-ai-models-to-privately-chat-with-One-Drive.md

@@ -0,0 +1,112 @@
+# Using GPT4All to Privately Chat with your OneDrive Data
+
+Local and Private AI Chat with your OneDrive Data
+
+OneDrive for Desktop allows you to sync and access your OneDrive files directly on your computer. By connecting your synced directory to LocalDocs, you can start using GPT4All to privately chat with data stored in your OneDrive.
+
+## Download OneDrive for Desktop
+
+!!! note "Download OneDrive for Desktop"
+
+    1. **Download OneDrive for Desktop**:
+    - Visit [Microsoft OneDrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/download).
+    - Press 'download' for your respective device type.
+    - Download the OneDrive for Desktop application.
+    
+    2. **Install OneDrive for Desktop**
+    - Run the installer file you downloaded.
+    - Follow the prompts to complete the installation process.
+    
+    3. **Sign in and Sync**
+    - Once installed, sign in to OneDrive for Desktop with your Microsoft account credentials.
+    - Choose the folders you want to sync to your computer.
+
+## Connect OneDrive to LocalDocs
+
+!!! note "Connect OneDrive to LocalDocs"
+
+    1. **Install GPT4All and Open LocalDocs**:
+    
+        - Go to [nomic.ai/gpt4all](https://nomic.ai/gpt4all) to install GPT4All for your operating system.
+        
+        - Navigate to the LocalDocs feature within GPT4All to configure it to use your synced OneDrive directory.
+
+        <table>
+        <tr>
+            <td>
+                <!-- Placeholder for screenshot of LocalDocs interface -->
+                <img width="1348" alt="Screenshot 2024-07-10 at 10 55 41 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/54254bc0-d9a0-40c4-9fd1-5059abaad583">
+            </td>
+        </tr>
+        </table>
+
+    2. **Add Collection**:
+    
+        - Click on **+ Add Collection** to begin linking your OneDrive folders.
+
+        <table>
+        <tr>
+            <td>
+                <!-- Placeholder for screenshot of adding collection in LocalDocs -->
+               <img width="1348" alt="Screenshot 2024-07-10 at 10 56 29 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/7f12969a-753a-4757-bb9e-9b607cf315ca">
+            </td>
+        </tr>
+        </table>
+
+        - Name the Collection and specify the OneDrive folder path.
+
+    3. **Create Collection**:
+    
+        - Click **Create Collection** to initiate the embedding process. Progress will be displayed within the LocalDocs interface.
+
+    4. **Access Files in Chats**:
+    
+        - Load a model within GPT4All to chat with your files.
+        
+        - In your chat, open 'LocalDocs' using the button in the top-right corner to provide context from your synced OneDrive files.
+
+        <table>
+        <tr>
+            <td>
+                <!-- Placeholder for screenshot of accessing LocalDocs in chats -->
+                <img width="1447" alt="Screenshot 2024-07-10 at 10 58 55 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/b5a67fe6-0d6a-42ae-b3b8-cc0f91cbf5b1">
+            </td>
+        </tr>
+        </table>
+
+    5. **Interact With Your OneDrive**:
+    
+        - Use the model to interact with your files directly from OneDrive.
+        
+        <table>
+        <tr>
+            <td>
+                <!-- Placeholder for screenshot of interacting with sources -->
+                <img width="662" alt="Screenshot 2024-07-10 at 11 04 55 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/2c9815b8-3d1c-4179-bf76-3ddbafb193bf">
+            </td>
+        </tr>
+        </table>
+        
+        <table>
+        <tr>
+            <td>
+                <img width="662" alt="Screenshot 2024-07-11 at 11 21 46 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/ce8be292-b025-415a-bd54-f11868e0cd0a">
+            </td>
+        </tr>
+        </table>
+
+    6. **View Referenced Files**:
+    
+        - Click on **Sources** below responses to see which OneDrive files were referenced.
+
+        <table>
+        <tr>
+            <td>
+              <img width="643" alt="Screenshot 2024-07-11 at 11 22 49 AM" src="https://github.com/nomic-ai/gpt4all/assets/132290469/6fe3f10d-2791-4153-88a7-2198ab3ac945">
+            </td>
+        </tr>
+        </table>
+
+## How It Works
+
+OneDrive for Desktop syncs your OneDrive files to your computer, while LocalDocs maintains a database of these synced files for use by your local GPT4All model. As your OneDrive updates, LocalDocs will automatically detect file changes and stay up to date. LocalDocs leverages [Nomic Embedding](https://docs.nomic.ai/atlas/capabilities/embeddings) models to find semantically similar snippets from your files, enhancing the context of your interactions.