3 Commits

Author	SHA1	Message	Date
yuehuayingxueluo	35382a7fbf	[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365) ... * fused the gate and up proj in mlp * fix code styles * opt auto_grad * rollback test_inference_engine.py * modifications based on the review feedback. * fix bugs in flash attn * Change reshape to view * fix test_rmsnorm_triton.py	2024-02-06 19:38:25 +08:00
Jianghai	7ddd8b37f0	fix (#5311)	2024-01-26 15:02:12 +08:00
Jianghai	c647e00e3c	[Inference]Add fused rotary kernel and get cos cache kernel (#5302) ... * add fused rotary and get cos cache func * staged * fix bugs * fix bugs	2024-01-24 16:20:42 +08:00