github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-29 05:26:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,201 Commits 59 Branches 51 Tags
90cd5227a348dfe506e95b2e49f2a8dcd34fdbca
Commit Graph

2 Commits

Author SHA1 Message Date
yuehuayingxueluo
90cd5227a3 [Fix/Inference]Fix vllm benchmark (#5630) 
* Fix bugs about OOM when running vllm-0.4.0

* rm used params

* change generation_config

* change benchmark log file name
 2024-04-24 14:51:36 +08:00
Yuanheng Zhao
04863a9b14 [example] Update Llama Inference example (#5629) 
* [example] add infernece benchmark llama3

* revise inference config - arg

* remove unused args

* add llama generation demo script

* fix init rope in llama policy

* add benchmark-llama3 - cleanup
 2024-04-23 22:23:07 +08:00