Runyu Lu e37ee2fb65 [Feat]Tensor Model Parallel Support For Inference (#5563) ...

* tensor parallel support naive source

* [fix]precision, model load and refactor the framework

* add tp unit test

* docstring

* fix do_sample

2024-04-18 16:56:46 +08:00

..

layers

[Inference]Fused the gate and up proj in mlp，and optimized the autograd process. (#5365)

2024-02-06 19:38:25 +08:00

models

[Feat]Tensor Model Parallel Support For Inference (#5563)

2024-04-18 16:56:46 +08:00

policy

[Feat]Tensor Model Parallel Support For Inference (#5563)

2024-04-18 16:56:46 +08:00

__init__.py

[doc] updated inference readme (#5343)

2024-02-02 14:31:10 +08:00