github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-13 05:01:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

[NFC] fix typo applications/ and colossalai/ (#3735)

Browse Source
This commit is contained in:
digger-yu
2023-05-15 11:46:25 +08:00
committed by GitHub
parent 1f73609adb
commit ad6460cf2c
10 changed files with 17 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
applications/Chat/inference/README.md
View File

@@ -75,7 +75,7 @@ E.g. you can set `export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH`.
Please ensure you have downloaded HF-format model weights of LLaMA models first.
Then you can follow [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa). This lib provides efficient CUDA kernels and weight convertion script.
Then you can follow [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa). This lib provides efficient CUDA kernels and weight conversion script.
After installing this lib, we may convert the original HF-format LLaMA model weights to 4-bit version.

2
applications/Chat/inference/benchmark.py
View File

@@ -123,7 +123,7 @@ if __name__ == "__main__":
start = time()
for instruction in instructions:
print(f"Instruction: {instruction}")
resp, tokens = evaluate(model, tokenizer, instruction, temparature=0.2, num_beams=1)
resp, tokens = evaluate(model, tokenizer, instruction, temperature=0.2, num_beams=1)
total_tokens += tokens
print(f"Response: {resp}")
print('\n----------------------------\n')