github/gpt4all
1
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all.git synced 2025-09-02 09:06:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

V3 docs max (#2488)

Browse Source 
* new skeleton

Signed-off-by: Max Cembalest <max@nomic.ai>

* v3 docs

Signed-off-by: Max Cembalest <max@nomic.ai>

---------

Signed-off-by: Max Cembalest <max@nomic.ai>
This commit is contained in:
mcembalest
2024-07-01 13:00:14 -04:00
committed by GitHub
parent bd307abfe6
commit 5306595176
57 changed files with 865 additions and 170 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
gpt4all-bindings/python/docs/gpt4all_desktop/chats.md Normal file
View File

@@ -0,0 +1,124 @@
# Chats
Chats are conversations with [language models](models.md) that run locally on your device.
!!! note "Example Chats"
=== "Llama 3"
Download `Llama 3` and prompt:
```
explain why the sky is blue in a way that is correct and makes sense to a child
```
With the default sampling settings, you should see text resembling the following:
```
What a great question!
So, you know how we can see different colors like red, yellow, green, and orange? Well, when sunlight enters Earth's atmosphere, it starts to interact with tiny particles called molecules of gases like nitrogen (N2) and oxygen (02). These molecules are really small, but they're all around us!
Now, here's the cool part: these molecules scatter, or bounce off, shorter wavelengths of light more than longer wavelengths. Think of it like a game of pool where the cue ball hits other balls with different sizes and speeds.
The shortest wavelength of visible light is violet (about 400 nanometers), followed by blue (around 450-495 nanometers). These shorter wavelengths are scattered in all directions, kind of like how a basketball player might bounce the ball around on the court. This scattering effect makes the sky appear blue to our eyes!
Here's why it doesn't look as bright or intense as a blue crayon: there are many more molecules in the atmosphere that scatter longer wavelengths (like red and orange) than shorter ones, so those colors get spread out and become less noticeable.
To sum it up:
1. Sunlight enters Earth's atmosphere.
2. Tiny gas molecule particles interact with light, scattering shorter wavelengths like blue and violet.
3. This scattered blue light reaches our eyes from all directions, making the sky appear blue!
Now, go outside and enjoy that beautiful blue sky!
```
=== "Nous Hermes 2 Mistral DPO"
Download `Nous Hermes 2 Mistral DPO` and prompt:
```
write me a react app i can run from the command line to play a quick game
```
With the default sampling settings, you should see text and code blocks resembling the following:
```
I'll provide you with a simple React app that allows you to play a guessing game. This app will generate a random number between 1 and 10, and ask you to guess it. You can run this app locally using Create React App. Follow the steps below:
Open your terminal or command line interface.
Navigate to the directory where you want to create the project (e.g., cd Documents/Projects).
Run the following commands:
```
```bash
npx create-react-app guessing-game
cd guessing-game
```
```
Open the project in your preferred code editor by running code . (for VS Code) or any other command to open the folder with your text editor.
Replace the content of the src/App.js file with the following code:
```
```javascript
import React, { useState } from 'react';
function App() {
const [guess, setGuess] = useState('');
const [message, setMessage] = useState('');
const [numberToGuess, setNumberToGuess] = useState(Math.floor(Math.random() * 10) + 1);
function handleChange(e) {
setGuess(e.target.value);
}
function handleSubmit(e) {
e.preventDefault();
if (parseInt(guess) === numberToGuess) {
setMessage('You guessed it right!');
} else {
setMessage(`Wrong guess, the correct number was ${numberToGuess}. Try again next time.`);
}
}
return (
<div>
<h1>Guess a Number</h1>
<p>{message}</p>
<form onSubmit={handleSubmit}>
<input type="number" value={guess} onChange={handleChange} />
<button type="submit">Submit</button>
</form>
</div>
);
}
export default App;
```
```
Save the file and go back to your terminal or command line interface. Run npm start. This will start a local development server, and you can access the app in your browser at http://localhost:3000.
Play the guessing game by entering a number between 1 and 10 into the input field and clicking "Submit". The app will tell you if your guess is correct or not.
Remember that this is just a simple example, and you can expand upon it to make the game more interesting with additional features like high scores, multiple difficulty levels, etc.
```
## New Chat
Choose a model with the dropdown at the top of the Chats page
If you don't have any models, [download one](models.md#download-models). Once you have models, you can start chats by loading your default model, which you can configure in [settings](settings.md#application-settings)
![Choose a model](../assets/three_model_options.png)
## LocalDocs
Open the [LocalDocs](localdocs.md) panel with the button in the top-right corner to bring your files into the chat. With LocalDocs, your chats are enhanced with semantically related snippets from your files included in the model's context.
![Open LocalDocs](../assets/open_local_docs.png)
## Chat History
View your chat history with the button in the top-left corner of the Chats page.
<table>
<tr>
<td>
<img src="../assets/closed_chat_panel.png" alt="Close chats" style="width:100%">
</td>
<td>
<img src="../assets/open_chat_panel.png" alt="Open chats" style="width:100%">
</td>
</tr>
</table>
You can change a chat name or delete it from your chat history at any time.

48
gpt4all-bindings/python/docs/gpt4all_desktop/localdocs.md Normal file
View File

@@ -0,0 +1,48 @@
# LocalDocs
LocalDocs brings the information you have from files on-device into your LLM chats - **privately**.
## Create LocalDocs
!!! note "Create LocalDocs"
1. Click `+ Add Collection`.
2. Name your collection and link it to a folder.
<table>
<tr>
<td>
<img src="../assets/new_docs_annotated.png" alt="new GOT Docs" style="width:100%">
</td>
<td>
<img src="../assets/new_docs_annotated_filled.png" alt="new GOT Docs filled out" style="width:100%">
</td>
</tr>
</table>
3. Click `Create Collection`. Progress for the collection is displayed on the LocalDocs page.
![Embedding in progress](../assets/baelor.png)
You will see a green `Ready` indicator when the entire collection is ready.
Note: you can still chat with the files that are ready before the entire collection is ready.
![Embedding complete](../assets/got_done.png)
Later on if you modify your LocalDocs settings you can rebuild your collections with your new settings.
4. In your chats, open `LocalDocs` with button in top-right corner to give your LLM context from those files.
![LocalDocs result](../assets/syrio_snippets.png)
5. See which files were referenced by clicking `Sources` below the LLM responses.
![Sources](../assets/open_sources.png)
## How It Works
A LocalDocs collection uses Nomic AI's free and fast on-device embedding models to index your folder into text snippets that each get an **embedding vector**. These vectors allow us to find snippets from your files that are semantically similar to the questions and prompts you enter in your chats. We then include those semantically similar snippets in the prompt to the LLM.
To try the embedding models yourself, we recommend using the [Nomic Python SDK](https://docs.nomic.ai/atlas/guides/embeddings)

79
gpt4all-bindings/python/docs/gpt4all_desktop/models.md Normal file
View File

@@ -0,0 +1,79 @@
# Models
GPT4All is optimized to run LLMs in the 3-13B parameter range on consumer-grade hardware.
LLMs are downloaded to your device so you can run them locally and privately. With our backend anyone can interact with LLMs efficiently and securely on their own hardware.
## Download Models
!!! note "Download Models"
<div style="text-align: center; margin-top: 20px;">
<table style="margin-left: auto; margin-right: auto;">
<tr>
<td style="text-align: right; padding-right: 10px;">1.</td>
<td style="text-align: left;">Click `Models` in the menu on the left (below `Chats` and above `LocalDocs`)</td>
<td><img src="../assets/models_page_icon.png" alt="Models Page Icon" style="width: 80px; height: auto;"></td>
</tr>
<tr>
<td style="text-align: right; padding-right: 10px;">2.</td>
<td style="text-align: left;">Click `+ Add Model` to navigate to the `Explore Models` page</td>
<td><img src="../assets/add.png" alt="Add Model button" style="width: 100px; height: auto;"></td>
</tr>
<tr>
<td style="text-align: right; padding-right: 10px;">3.</td>
<td style="text-align: left;">Search for models available online</td>
<td><img src="../assets/explore.png" alt="Explore Models search" style="width: 120px; height: auto;"></td>
</tr>
<tr>
<td style="text-align: right; padding-right: 10px;">4.</td>
<td style="text-align: left;">Hit `Download` to save a model to your device</td>
<td><img src="../assets/download.png" alt="Download Models button" style="width: 120px; height: auto;"></td>
</tr>
<tr>
<td style="text-align: right; padding-right: 10px;">5.</td>
<td style="text-align: left;">Once the model is downloaded you will see it in `Models`.</td>
<td><img src="../assets/installed_models.png" alt="Download Models button" style="width: 120px; height: auto;"></td>
</tr>
</table>
</div>
## Explore Models
GPT4All connects you with LLMs from HuggingFace with a [`llama.cpp`](https://github.com/ggerganov/llama.cpp) backend so that they will run efficiently on your hardware. Many of these models can be identified by the file type `.gguf`.
![Explore models](../assets/search_mistral.png)
## Example Models
Many LLMs are available at various sizes, quantizations, and licenses.
- LLMs with more parameters tend to be better at coherently responding to instructions
- LLMs with a smaller quantization (e.g. 4bit instead of 16bit) are much faster and less memory intensive, and tend to have slightly worse performance
- Licenses vary in their terms for personal and commercial use
Here are a few examples:
| Model| Filesize| RAM Required| Parameters| Developer| License| MD5 Sum (Unique Hash)|
|------|---------|-------------|-----------|----------|--------|----------------------|
| Llama 3 Instruct | 4.66 GB| 8 GB| 8 Billion| Meta| [Llama 3 License](https://llama.meta.com/llama3/license/)| c87ad09e1e4c8f9c35a5fcef52b6f1c9|
| Nous Hermes 2 Mistral DPO| 4.21 GB| 8 GB| 7 Billion| Mistral & Nous Research | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)| Coa5f6b4eabd3992da4d7fb7f020f921eb|
| Phi-3 Mini Instruct | 2.03 GB| 4 GB| 4 billion| Microsoft| [MIT](https://opensource.org/license/mit)| f8347badde9bfc2efbe89124d78ddaf5|
| Mini Orca (Small)| 1.84 GB| 4 GB| 3 billion| Microsoft | [CC-BY-NC-SA-4.0](https://spdx.org/licenses/CC-BY-NC-SA-4.0)| 0e769317b90ac30d6e09486d61fefa26|
| GPT4All Snoozy| 7.36 GB| 16 GB| 13 billion| Nomic AI| [GPL](https://www.gnu.org/licenses/gpl-3.0.en.html)| 40388eb2f8d16bb5d08c96fdfaac6b2c|
### Search Results
You can click the gear icon in the search bar to sort search results by their # of likes, # of downloads, or date of upload (all from HuggingFace).
![Sort search results](../assets/search_settings.png)
## Connect Model APIs
You can add your API key for remote model providers.
**Note**: this does not download a model file to your computer to use securely. Instead, this way of interacting with models has your prompts leave your computer to the API provider and returns the response to your computer.
![Connect APIs](../assets/add_model_gpt4.png)

40
gpt4all-bindings/python/docs/gpt4all_desktop/quickstart.md Normal file
View File

@@ -0,0 +1,40 @@
# GPT4All Desktop
The GPT4All Desktop Application allows you to download and run large language models (LLMs) locally & privately on your device.
With GPT4All, you can chat with models, turn your local files into information sources for models [(LocalDocs)](localdocs.md), or browse models available online to download onto your device.
## Quickstart
!!! note "Quickstart"
1. Install GPT4All for your operating system and open the application.
<div style="text-align: center; margin-top: 20px;">
[Download for Windows](https://gpt4all.io/installers/gpt4all-installer-win64.exe) &nbsp;&nbsp;&nbsp;&nbsp;
[Download for Mac](https://gpt4all.io/installers/gpt4all-installer-darwin.dmg) &nbsp;&nbsp;&nbsp;&nbsp;
[Download for Linux](https://gpt4all.io/installers/gpt4all-installer-linux.run)
</div>
2. Hit `Start Chatting`. ![GPT4All home page](../assets/gpt4all_home.png)
3. Click `+ Add Model`.
4. Download a model. We recommend starting with Llama 3, but you can [browse more models](models.md). ![Download a model](../assets/download_llama.png)
5. Once downloaded, go to Chats (below Home and above Models in the menu on the left).
6. Click "Load Default Model" (will be Llama 3 or whichever model you downloaded).
<table>
<tr>
<td>
<img src="../assets/before_first_chat.png" alt="Before first chat" style="width:100%">
</td>
<td>
<img src="../assets/new_first_chat.png" alt="New first chat" style="width:100%">
</td>
</tr>
</table>
7. Try the [example chats](chats.md) or your own prompts!

75
gpt4all-bindings/python/docs/gpt4all_desktop/settings.md Normal file
View File

@@ -0,0 +1,75 @@
# Settings
## Application Settings
!!! note "General Application Settings"
| Setting | Description | Default Value |
| --- | --- | --- |
| **Theme** | Color theme for the application. Options are `Light`, `Dark`, and `LegacyDark` | `Light` |
| **Font Size** | Font size setting for text throughout the application. Options are Small, Medium, and Large | Small |
| **Device** | Device that will run your models. Options are `Auto` (GPT4All chooses), `Metal` (Apple Silicon M1+), `CPU`, and `GPU` | `Auto` |
| **Default Model** | Choose your preferred LLM to load by default on startup| Auto |
| **Download Path** | Select a destination on your device to save downloaded models | Windows: `C:\Users\{username}\AppData\Local\nomic.ai\GPT4All`<br><br>Mac: `/Users/{username}/Library/Application Support/nomic.ai/GPT4All/`<br><br>Linux: `/home/{username}/.local/share/nomic.ai/GPT4All` |
| **Enable Datalake** | Opt-in to sharing interactions with GPT4All community (**anonymous** and **optional**) | Off |
!!! note "Advanced Application Settings"
| Setting | Description | Default Value |
| --- | --- | --- |
| **CPU Threads** | Number of concurrently running CPU threads (more can speed up responses) | 4 |
| **Save Chat Context** | Save chat context to disk to pick up exactly where a model left off. | Off |
| **Enable Local Server** | Allow any application on your device to use GPT4All via an OpenAI-compatible GPT4All API | Off |
| **API Server Port** | Local HTTP port for the local API server | 4891 |
## Model Settings
!!! note "Model / Character Settings"
| Setting | Description | Default Value |
| --- | --- | --- |
| **Name** | Unique name of this model / character| set by model uploader |
| **System Prompt** | General instructions for the chats this model will be used for | set by model uploader |
| **Prompt Template** | Format of user <-> assistant interactions for the chats this model will be used for | set by model uploader |
### Clone
You can **clone** an existing model, which allows you to save a configuration of a model file with different prompt templates and sampling settings.
### Sampling Settings
!!! note "Model Sampling Settings"
| Setting | Description | Default Value |
|----------------------------|------------------------------------------|-----------|
| **Context Length** | Maximum length of input sequence in tokens | 2048 |
| **Max Length** | Maximum length of response in tokens | 4096 |
| **Prompt Batch Size** | Token batch size for parallel processing | 128 |
| **Temperature** | Lower temperature gives more likely generations | 0.7 |
| **Top P** | Prevents choosing highly unlikely tokens | 0.4 |
| **Top K** | Size of selection pool for tokens | 40 |
| **Min P** | Minimum relative probability | 0 |
| **Repeat Penalty Tokens** | Length to apply penalty | 64 |
| **Repeat Penalty** | Penalize repetitiveness | 1.18 |
| **GPU Layers** | How many model layers to load into VRAM | 32 |
## LocalDocs Settings
!!! note "General LocalDocs Settings"
| Setting | Description | Default Value |
| --- | --- | --- |
| **Allowed File Extensions** | Choose which file types will be indexed into LocalDocs collections as text snippets with embedding vectors | `.txt`, `.pdf`, `.md`, `.rst` |
| **Use Nomic Embed API** | Use Nomic API to create LocalDocs collections fast and off-device; [Nomic API Key](https://atlas.nomic.ai/) required | Off |
| **Embeddings Device** | Device that will run embedding models. Options are `Auto` (GPT4All chooses), `Metal` (Apple Silicon M1+), `CPU`, and `GPU` | `Auto` |
| **Show Sources** | Titles of source files retrieved by LocalDocs will be displayed directly in your chats.| On |
!!! note "Advanced LocalDocs Settings"
Note that increasing these settings can increase the likelihood of factual responses, but may result in slower generation times.
| Setting | Description | Default Value |
| --- | --- | --- |
| **Document Snippet Size** | Number of string characters per document snippet | 512 |
| **Maximum Document Snippets Per Prompt** | Upper limit for the number of snippets from your files LocalDocs can retrieve for LLM context | 3 |