Jianghai c5ea728016 [pipeline] add bert_for_pretraining bert_lmhead forward and policy (#4172) ...

* add pipeline policy and bert forward to be done

* add bertmodel pipeline forward and make tests

* add Bert_Policy and test for policy

* update formatting

* update formatting

* update the code

* fix bugs

* fix name confilt

* add bloom model and policy ,revise the base class of policy

* revise

* revision

* add bert_for_pretraining

* add bert_for_pretraining forward and policy

* fix typos

* cancel warning

* change the imediate output to default dict

* change the default output of get_shared_params

2023-08-15 23:25:14 +08:00

..

_analyzer

[example] add train resnet/vit with booster example (#3694)

2023-05-08 10:42:30 +08:00

_C

[setup] support pre-build and jit-build of cuda kernels (#2374)

2023-01-06 20:50:26 +08:00

amp

[bf16] add bf16 support (#3882)

2023-06-05 15:58:31 +08:00

auto_parallel

[NFC] polish runtime_preparation_pass style (#4266)

2023-07-26 14:12:57 +08:00

autochunk

fix typo colossalai/auto_parallel autochunk fx/passes etc. (#3808)

2023-05-24 09:01:50 +08:00

booster

[zero] support shard optimizer state dict of zero (#4194)

2023-07-31 22:13:29 +08:00

builder

…

checkpoint_io

[checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin (#4302)

2023-07-21 14:39:01 +08:00

cli

fix localhost measurement (#4320)

2023-08-01 10:14:00 +08:00

cluster

[cluster] add process group mesh (#4039)

2023-08-15 23:25:14 +08:00

communication

[NFC] fix: format (#4270)

2023-07-26 14:12:57 +08:00

context

[CI] fix some spelling errors (#3707)

2023-05-10 17:12:03 +08:00

device

[format] applied code formatting on changed files in pull request 4152 (#4157)

2023-07-04 16:07:47 +08:00

engine

[nfc]fix ColossalaiOptimizer is not defined (#4122)

2023-06-30 17:23:22 +08:00

fx

[nfc] fix typo colossalai/cli fx kernel (#3847)

2023-06-02 15:02:45 +08:00

interface

[pipeline] refactor 1f1b schedule (#4115)

2023-08-15 23:25:14 +08:00

kernel

[coloattention] fix import error (#4380)

2023-08-04 16:28:41 +08:00

lazy

[lazy] support init on cuda (#4269)

2023-07-19 16:43:01 +08:00

logging

…

nn

[doc] add Series A Funding and NeurIPS news (#4377)

2023-08-04 17:42:07 +08:00

pipeline

[pipeline] add bert_for_pretraining bert_lmhead forward and policy (#4172)

2023-08-15 23:25:14 +08:00

registry

…

shardformer

[shardformer] fix type hint

2023-08-15 23:25:14 +08:00

tensor

[dtensor] fixed readme file name and removed deprecated file (#4162)

2023-07-04 18:21:11 +08:00

testing

Next commit [checkpointio] Unsharded Optimizer Checkpoint for Gemini Plugin (#4141)

2023-07-07 16:33:06 +08:00

trainer

fix typo with colossalai/trainer utils zero (#3908)

2023-06-07 16:08:37 +08:00

utils

[test] remove useless tests (#4359)

2023-08-01 18:52:14 +08:00

zero

[hotfix] fix unsafe async comm in zero (#4404)

2023-08-11 15:09:24 +08:00

__init__.py

…

constants.py

…

core.py

…

global_variables.py

[NFC] polish colossalai/global_variables.py code style (#3259)

2023-03-29 15:22:21 +08:00

initialize.py

[nfc] fix typo colossalai/zero (#3923)

2023-06-08 00:01:29 +08:00