github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-02 01:28:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

[NFC] Update roberta/README.md (#2350)

Browse Source
This commit is contained in:
Ikko Eltociear Ashimine
2023-01-06 11:09:14 +09:00
committed by GitHub
parent 35e22be2f6
commit 5e4bced0a3
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/language/roberta/README.md
View File

@@ -33,7 +33,7 @@ service ssh restart
```bash ```bash
cd preprocessing cd preprocessing
``` ```
following the `README.md`, preprocess orginal corpus to h5py+numpy following the `README.md`, preprocess original corpus to h5py+numpy
## 2. Pretrain ## 2. Pretrain
@@ -44,7 +44,7 @@ following the `README.md`, load the h5py generated by preprocess of step 1 to pr
## 3. Finetune ## 3. Finetune
The checkpoint produced by this repo can replace `pytorch_model.bin` from [hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/tree/main) directly. Then use transfomers from HuggingFace to finetune downstream application. The checkpoint produced by this repo can replace `pytorch_model.bin` from [hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large/tree/main) directly. Then use transfomers from Hugging Face to finetune downstream application.
## Contributors ## Contributors
The repo is contributed by AI team from [Moore Threads](https://www.mthreads.com/). If you find any problems for pretraining, please file an issue or send an email to yehua.zhang@mthreads.com. At last, welcome any form of contribution! The repo is contributed by AI team from [Moore Threads](https://www.mthreads.com/). If you find any problems for pretraining, please file an issue or send an email to yehua.zhang@mthreads.com. At last, welcome any form of contribution!
@@ -55,4 +55,4 @@ The repo is contributed by AI team from [Moore Threads](https://www.mthreads.com
author={Yehua Zhang, Chen Zhang}, author={Yehua Zhang, Chen Zhang},
year={2022} year={2022}
} }
``` ```