mirror of
https://github.com/nomic-ai/gpt4all.git
synced 2025-10-02 10:38:47 +00:00
c4706d0c14
* porting over replit code model to gpt4all * replaced memory with kv_self struct * continuing debug * welp it built but lot of sus things * working model loading and somewhat working generate.. need to format response? * revert back to semi working version * finally got rid of weird formatting * figured out problem is with python bindings - this is good to go for testing * addressing PR feedback * output refactor * fixed prompt reponse collection * cleanup * addressing PR comments * building replit backend with new ggmlver code * chatllm replit and clean python files * cleanup * updated replit to match new llmodel api * match llmodel api and change size_t to Token * resolve PR comments * replit model commit comment
Python GPT4All
This package contains a set of Python bindings around the llmodel C-API.
Documentation
docs.gpt4all.io
Installation
pip install gpt4all
Local Build Instructions
NOTE: If you are doing this on a Windows machine, you must build the GPT4All backend using MinGW64 compiler.
- Setup
llmodel
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all
cd gpt4all/gpt4all-backend/
mkdir build
cd build
cmake ..
cmake --build . --parallel
Confirm that libllmodel.* exists in gpt4all-backend/build.
- Setup Python package
cd ../../gpt4all-bindings/python
pip3 install -e .
- Test it out! In a Python script or console:
from gpt4all import GPT4All
gptj = GPT4All("ggml-gpt4all-j-v1.3-groovy")
messages = [{"role": "user", "content": "Name 3 colors"}]
gptj.chat_completion(messages)