mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2025-09-09 13:00:52 +00:00
[tutorial] polish all README (#1946)
This commit is contained in:
@@ -4,6 +4,17 @@ This is an example showing how to run OPT generation. The OPT model is implement
|
||||
|
||||
It supports tensor parallelism, batching and caching.
|
||||
|
||||
## 🚀Quick Start
|
||||
1. Run inference with OPT 125M
|
||||
```bash
|
||||
docker hpcaitech/tutorial:opt-inference
|
||||
docker run -it --rm --gpus all --ipc host -p 7070:7070 hpcaitech/tutorial:opt-inference
|
||||
```
|
||||
2. Start the http server inside the docker container with tensor parallel size 2
|
||||
```bash
|
||||
python opt_fastapi.py opt-125m --tp 2 --checkpoint /data/opt-125m
|
||||
```
|
||||
|
||||
# How to run
|
||||
|
||||
Run OPT-125M:
|
||||
|
@@ -15,6 +15,7 @@ limitations under the License.
|
||||
-->
|
||||
# Train OPT model with Colossal-AI
|
||||
|
||||
|
||||
## OPT
|
||||
Meta recently released [Open Pretrained Transformer (OPT)](https://github.com/facebookresearch/metaseq), a 175-Billion parameter AI language model, which stimulates AI programmers to perform various downstream tasks and application deployments.
|
||||
|
||||
@@ -26,7 +27,21 @@ the tokenization). This training script is adapted from the [HuggingFace Languag
|
||||
## Our Modifications
|
||||
We adapt the OPT training code to ColossalAI by leveraging Gemini and ZeRO DDP.
|
||||
|
||||
## Quick Start
|
||||
## 🚀Quick Start for Tutorial
|
||||
1. Install the dependency
|
||||
```bash
|
||||
pip install datasets accelerate
|
||||
```
|
||||
2. Run finetuning with synthetic datasets with one GPU
|
||||
```bash
|
||||
bash ./run_clm_synthetic.sh
|
||||
```
|
||||
3. Run finetuning with 4 GPUs
|
||||
```bash
|
||||
bash ./run_clm_synthetic.sh 16 0 125m 4
|
||||
```
|
||||
|
||||
## Quick Start for Practical Use
|
||||
You can launch training by using the following bash script
|
||||
|
||||
```bash
|
||||
|
Reference in New Issue
Block a user