github/gpt4all
1
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all.git synced 2025-09-07 19:40:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

python: add list_gpus to the GPT4All API (#2194)

Browse Source 
Other changes:
* fix memory leak in llmodel_available_gpu_devices
* drop model argument from llmodel_available_gpu_devices
* breaking: make GPT4All/Embed4All arguments past model_name keyword-only

Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This commit is contained in:
Jared Van Bortel
2024-04-04 14:52:13 -04:00
committed by GitHub
parent 790320e170
commit 1b84a48c47
8 changed files with 91 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
gpt4all-backend/llmodel_c.h
View File

@@ -48,9 +48,9 @@ struct llmodel_prompt_context {
};
struct llmodel_gpu_device {
int index = 0;
int type = 0; // same as VkPhysicalDeviceType
size_t heapSize = 0;
int index;
int type; // same as VkPhysicalDeviceType
size_t heapSize;
const char * name;
const char * vendor;
};
@@ -241,9 +241,10 @@ const char *llmodel_get_implementation_search_path();
/**
* Get a list of available GPU devices given the memory required.
* @param memoryRequired The minimum amount of VRAM, in bytes
* @return A pointer to an array of llmodel_gpu_device's whose number is given by num_devices.
*/
struct llmodel_gpu_device* llmodel_available_gpu_devices(llmodel_model model, size_t memoryRequired, int* num_devices);
struct llmodel_gpu_device* llmodel_available_gpu_devices(size_t memoryRequired, int* num_devices);
/**
* Initializes a GPU device based on a specified string criterion.