From 55dcd3051adfe7e1b12743a0ce4797e2bc0ac005 Mon Sep 17 00:00:00 2001 From: BlueRum <70618399+ht-zhou@users.noreply.github.com> Date: Tue, 7 Mar 2023 10:21:25 +0800 Subject: [PATCH] [chatgpt] fix readme (#3025) --- applications/ChatGPT/examples/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/applications/ChatGPT/examples/README.md b/applications/ChatGPT/examples/README.md index bf3daf5ec..39a769110 100644 --- a/applications/ChatGPT/examples/README.md +++ b/applications/ChatGPT/examples/README.md @@ -15,9 +15,9 @@ Use these code to train your reward model. ```shell # Naive reward model training -python train_reward_model.py --pretrain -# if to use LoRA -python train_reward_model.py --pretrain --lora_rank 16 +python train_reward_model.py --pretrain --model --strategy naive +# use colossalai_zero2 +torchrun --standalone --nproc_per_node=2 train_reward_model.py --pretrain --model --strategy colossalai_zero2 ``` ## Train with dummy prompt data (Stage 3) @@ -44,7 +44,7 @@ DDP strategy and ColossalAI strategy support multi GPUs training: # run DDP on 2 GPUs torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy ddp # run ColossalAI on 2 GPUs -torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy colossalai +torchrun --standalone --nproc_per_node=2 train_dummy.py --strategy colossalai_zero2 ``` ## Train with real prompt data (Stage 3) @@ -63,7 +63,7 @@ python train_prompts.py prompts.csv --strategy naive # run DDP on 2 GPUs torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy ddp # run ColossalAI on 2 GPUs -torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai +torchrun --standalone --nproc_per_node=2 train_prompts.py prompts.csv --strategy colossalai_zero2 ``` ## Inference example(After Stage3)