mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2025-09-25 11:44:03 +00:00
ed97d3a5d3
* fix readme * fix readme, tokenization fully tested * fix readme, tokenization fully tested * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: root <root@notebook-8f919155-6035-47b4-9c6f-1be133b9e2c9-0.notebook-8f919155-6035-47b4-9c6f-1be133b9e2c9.colossal-ai.svc.cluster.local> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
24 lines
283 B
Plaintext
Executable File
24 lines
283 B
Plaintext
Executable File
transformers==4.39.3
|
|
tqdm
|
|
datasets==2.14.7
|
|
loralib
|
|
colossalai>=0.4.0
|
|
torch>=2.1.0
|
|
langchain
|
|
tokenizers
|
|
fastapi
|
|
sse_starlette
|
|
wandb
|
|
sentencepiece
|
|
gpustat
|
|
packaging
|
|
autoflake==2.2.1
|
|
black==23.9.1
|
|
tensorboard
|
|
six==1.16.0
|
|
datasets
|
|
ninja==1.11.1
|
|
sentencepiece==0.1.99
|
|
flash-attn
|
|
tiktoken
|