Commit Graph

326 Commits

Author SHA1 Message Date
Zach Nussbaum
55fef489ad fix: wip mem xf 2023-06-02 23:02:25 +00:00
Zach Nussbaum
3677935ce8 wip: train of mem attn 2023-05-18 19:44:30 +00:00
Zach Nussbaum
5f28e60a9a feat: mem attn forward pass on cuda 2023-05-18 19:44:04 +00:00
Zach Nussbaum
18b04347f5 fix: prep index for mem attn 2023-05-18 19:43:38 +00:00
Zach Nussbaum
60f2f99cd7 feat: mem transformer 2023-05-17 20:42:26 +00:00
Zach Nussbaum
cfddd78eb4 fix: retrieval 2023-05-17 20:41:53 +00:00
Zach Nussbaum
db90a15911 feat: eval pythia 2023-05-17 20:41:20 +00:00
Zach Nussbaum
fd3f9da403 chore: configs 2023-05-15 14:17:13 +00:00
Zach Nussbaum
e8ccaea0bf feat: current working train 2023-05-15 14:17:02 +00:00
Zach Nussbaum
a69475a5f1 feat: updates to model + config 2023-05-15 14:16:48 +00:00
Zach Nussbaum
4d5e48a7a9 Merge branch 'junior' of https://github.com/nomic-ai/gpt4all into junior 2023-05-05 02:18:10 +00:00
Zach Nussbaum
71db2b664a feat: forward for pythiaseek 2023-05-04 03:16:33 +00:00
Zach Nussbaum
d1b64d7eed refactor: imports 2023-05-04 03:16:26 +00:00
Zach Nussbaum
0402d3e28a feat: barebones pythiaseek 2023-05-03 22:05:55 +00:00
Zach Nussbaum
2771f96cb6 refactor: move to folder 2023-05-03 21:52:40 +00:00
Zach Nussbaum
bd6e471555 feat: cosine alpha schedule 2023-05-03 21:39:14 +00:00
Zach Nussbaum
27a9b2b10c fix: option for no schedule 2023-05-03 21:39:05 +00:00
Zach Nussbaum
aa6763daa8 fix: update config 2023-05-03 21:33:52 +00:00
Zach Nussbaum
3fa80f8c09 fix: remove schedule 2023-05-03 21:33:29 +00:00
Zach Nussbaum
06228d9b67 Merge branch 'junior' of https://github.com/nomic-ai/gpt4all into junior 2023-05-03 01:52:09 +00:00
Zach Nussbaum
73849d95f1 Merge branch 'junior' of https://github.com/nomic-ai/gpt4all into junior 2023-05-02 19:45:06 +00:00
Zach Nussbaum
d61cd55772 fix: alpha, projection 2023-05-02 19:38:09 +00:00
Zach Nussbaum
2c8e1096c5
Merge pull request #472 from berkantay/main
Update README.md
2023-05-02 10:15:40 -04:00
Zach Nussbaum
201505dd62 Merge branch 'junior' of https://github.com/nomic-ai/gpt4all into junior 2023-05-02 14:11:06 +00:00
Zach Nussbaum
00f04360d2 fix: config for index building 2023-05-01 21:39:55 +00:00
Zach Nussbaum
0f61cd8b42 fix: retrieval dataset only has train split 2023-05-01 21:39:40 +00:00
Zach Nussbaum
3736eda56a feat: eval for retrieval 2023-05-01 21:39:21 +00:00
Zach Nussbaum
1b3f18bef2 fix: import path 2023-05-01 21:39:09 +00:00
Zach Nussbaum
0c0a56acab feat: data preprocessing 2023-05-01 21:38:46 +00:00
Zach Nussbaum
c9dd9152c3 feat: model def + metrics 2023-05-01 21:38:36 +00:00
Zach Nussbaum
48e07be9e9 feat: training script 2023-05-01 21:38:23 +00:00
Zach Nussbaum
80d810322a fix: lr schedule 2023-05-01 21:38:01 +00:00
Berkant
aefea2e713
Update README.md
README.md typo fix.
2023-04-30 01:07:14 +03:00
Zach Nussbaum
8a917ad4e1 chore: create data folder 2023-04-25 21:28:56 +00:00
Zach Nussbaum
a58d1eb3bd refactor: move file around 2023-04-25 21:28:42 +00:00
Zach Nussbaum
da5ce0a181 chore: ignore large arrow files 2023-04-25 20:36:04 +00:00
Zach Nussbaum
b0f92b610e refactor: clean up embed texts 2023-04-25 20:34:49 +00:00
Zach Nussbaum
c20379f7e9 refactor: clean up prep index 2023-04-25 20:34:38 +00:00
Zach Nussbaum
f2161f7e59 docs: prep index barebones 2023-04-25 20:34:26 +00:00
Zach Nussbaum
586a8abc06 fix: allow for print for fns that are used in both dist and single 2023-04-25 20:34:12 +00:00
Zach Nussbaum
7832707c37 fix: index -> id 2023-04-25 20:33:49 +00:00
AT
b00d338c1e
Update README.md 2023-04-25 09:02:34 -04:00
AT
a7ada4e4b0
Update README.md 2023-04-25 09:02:11 -04:00
AT
8b10e533bb
Update README.md 2023-04-25 09:01:31 -04:00
Zach Nussbaum
869829a065 nearly have the neighbor caching working, but combinging info into final dataset is challenging 2023-04-23 22:28:57 +00:00
Zach Nussbaum
cc3cc3f7e9 we now get the neighbors embeddings from the disk index 2023-04-23 21:04:43 +00:00
Zach Nussbaum
240feae277 added initial files for dataset prep and ingestion for gpt4all jr 2023-04-23 20:02:22 +00:00
Zach Nussbaum
84acbc8225 chore: ignore swp files 2023-04-23 17:34:49 +00:00
Zach Nussbaum
9bc88fb33d feat: add options to specify different datasets 2023-04-22 19:53:47 +00:00
Zach Nussbaum
8cb3c8de35 chore: reqs, ignore, readme 2023-04-22 19:40:06 +00:00