github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-05-02 05:35:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,706 Commits 43 Branches 49 Tags 233 MiB
64f74a157e
Commit Graph

13 Commits

Author SHA1 Message Date
flybird11111
64f74a157e
[NPU]support npu () 
* support npu

* support pretrain

support pretrain

fix

* support lora

fix

fix

* support chatglm

fix

fxi

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

fix

fix

fix

* Update train.py

* Update train.py
 2024-11-20 15:28:35 +08:00
Hongxin Liu
da15fdb9ca
[doc] add lazy init docs () 2023-09-27 10:24:04 +08:00
Hongxin Liu
4965c0dabd
[lazy] support from_pretrained () 
* [lazy] patch from pretrained

* [lazy] fix from pretrained and add tests

* [devops] update ci
 2023-09-26 11:04:11 +08:00
Hongxin Liu
3e05c07bb8
[lazy] support torch 2.0 () 
* [lazy] support _like methods and clamp

* [lazy] pass transformers models

* [lazy] fix device move and requires grad

* [lazy] fix requires grad and refactor api

* [lazy] fix requires grad
 2023-09-21 16:30:23 +08:00
Hongxin Liu
079bf3cb26
[misc] update pre-commit and run all files () 
* [misc] update pre-commit

* [misc] run pre-commit

* [misc] remove useless configuration files

* [misc] ignore cuda for clang-format
 2023-09-19 14:20:26 +08:00
Hongxin Liu
890774b2fb [shardformer] support lazy init () 
* [shardformer] support lazy init

* [shardformer] linear support lazy init

* [shardformer] embedding support lazy init

* [shardformer] norm support lazy init

* [shardformer] fused linear support lazy init

* [test] update shardformer test layer

* [test] shardformer with lazy init fit ddp

* [lazy] hotfix deepcopy of param

* [shardformer] fix bert policy and update test

* [shardformer] fix bloom policy and update test

* [shardformer] fix opt policy and update test

* [shardformer] fix t5 policy and update test

* [shardformer] fix gpt2 policy and update test

* [shardformer] fix llama policy and update test
 2023-08-15 23:25:14 +08:00
Hongxin Liu
fc5cef2c79
[lazy] support init on cuda () 
* [lazy] support init on cuda

* [test] update lazy init test

* [test] fix transformer version
 2023-07-19 16:43:01 +08:00
Frank Lee
c4b1b65931 [test] fixed tests failed due to dtensor change () 
* [test] fixed tests failed due to dtensor change

* polish code
 2023-07-04 16:05:01 +08:00
Frank Lee
8eb09a4c69 [shardformer] support module saving and loading () 
* [shardformer] support module saving and loading

* polish code
 2023-07-04 16:05:01 +08:00
Frank Lee
ddcf58cacf
Revert "[sync] sync feature/shardformer with develop" 2023-06-09 09:41:27 +08:00
Frank Lee
eb39154d40
[dtensor] updated api and doc () 2023-06-08 10:18:17 +08:00
Hongxin Liu
9c88b6cbd1
[lazy] fix compatibility problem on torch 1.13 () 2023-06-07 11:10:12 +08:00
Hongxin Liu
dbb32692d2
[lazy] refactor lazy init () 
* [lazy] remove old lazy init

* [lazy] refactor lazy init folder structure

* [lazy] fix lazy tensor deepcopy

* [test] update lazy init test
 2023-06-05 14:20:47 +08:00