github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-10-09 20:14:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,202 Commits 59 Branches 51 Tags
a8fd3b034235e1fa987a1ae85a9a2b465ee6128f
Commit Graph

2 Commits

Author SHA1 Message Date
yuehuayingxueluo
90cd5227a3 [Fix/Inference]Fix vllm benchmark (#5630) 
* Fix bugs about OOM when running vllm-0.4.0

* rm used params

* change generation_config

* change benchmark log file name
 2024-04-24 14:51:36 +08:00
Yuanheng Zhao
04863a9b14 [example] Update Llama Inference example (#5629) 
* [example] add infernece benchmark llama3

* revise inference config - arg

* remove unused args

* add llama generation demo script

* fix init rope in llama policy

* add benchmark-llama3 - cleanup
 2024-04-23 22:23:07 +08:00