github/gpt4all
1
0
Fork 0
mirror of https://github.com/nomic-ai/gpt4all.git synced 2025-09-06 19:10:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

add requiredMem method to llmodel impls

Browse Source 
most of these can just shortcut out of the model loading logic llama is a bit worse to deal with because we submodule it so I have to at least parse the hparams, and then I just use the size on disk as an estimate for the mem size (which seems reasonable since we mmap() the llama files anyway)
This commit is contained in:
Aaron Miller
2023-06-26 12:17:34 -07:00
committed by AT
parent dead954134
commit b19a3e5b2c
14 changed files with 154 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
gpt4all-backend/llmodel_c.cpp
View File

@@ -60,6 +60,12 @@ void llmodel_model_destroy(llmodel_model model) {
delete reinterpret_cast<LLModelWrapper*>(model);
}
size_t llmodel_required_mem(llmodel_model model, const char *model_path)
{
LLModelWrapper *wrapper = reinterpret_cast<LLModelWrapper*>(model);
return wrapper->llModel->requiredMem(model_path);
}
bool llmodel_loadModel(llmodel_model model, const char *model_path)
{
LLModelWrapper *wrapper = reinterpret_cast<LLModelWrapper*>(model);