github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-07 03:52:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

[lora] add lora APIs for booster, support lora for TorchDDP (#4981)

Browse Source 
* add apis and peft requirement

* add liscense and implement apis

* add checkpointio apis

* add torchddp fwd_bwd test

* add support_lora methods

* add checkpointio test and debug

* delete unneeded codes

* remove peft from LICENSE

* add concrete methods for enable_lora

* simplify enable_lora api

* fix requirements
This commit is contained in:
Baizhou Zhang
2023-10-31 15:19:37 +08:00
committed by Hongxin Liu
parent c1594e4bad
commit 14b0d4c7e5
11 changed files with 265 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
requirements/requirements-test.txt
View File

@@ -5,7 +5,7 @@ git+https://github.com/hpcaitech/pytest-testmon
torchvision
timm
titans
torchaudio
torchaudio>=0.13.1
torchx-nightly==2022.6.29 # torchrec 0.2.0 requires torchx-nightly. This package is updated every day. We fix the version to a specific date to avoid breaking changes.
torchrec==0.2.0
contexttimer
@@ -18,4 +18,5 @@ flash_attn
datasets
pydantic
ray
peft
#auto-gptq now not support torch1.12