[example] update Grok-1 inference (#5495)

* revise grok-1 example

* remove unused arg in scripts

* prevent re-installing torch

* update readme

* revert modifying colossalai requirements

* add perf

* trivial

* add tokenizer url
This commit is contained in:
Yuanheng Zhao
2024-03-24 20:24:11 +08:00
committed by GitHub
parent 6df844b8c4
commit 5fcd7795cd
7 changed files with 69 additions and 43 deletions

View File

@@ -1,11 +1,9 @@
#!/usr/bin/env bash
PRETRAINED=${1:-"hpcaitech/grok-1"}
TOKENIZER=${2:-"tokenizer.model"}
torchrun --standalone --nproc_per_node 8 inference_tp.py --pretrained "$PRETRAINED" \
--tokenizer "$TOKENIZER" \
--max_new_tokens 64 \
--max_new_tokens 100 \
--text "The company's annual conference, featuring keynote speakers and exclusive product launches, will be held at the Los Angeles Convention Center from October 20th to October 23rd, 2021. Extract the date mentioned in the above sentence." \
"将以下句子翻译成英语。 我喜欢看电影和读书。" \
"All books have the same weight, 10 books weigh 5kg, what is the weight of 2 books?"