4 Commits

Author	SHA1	Message	Date
BurkeHulk	e88190184a	support fp8 communication in pipeline parallelism	2024-07-12 15:25:25 +08:00
BurkeHulk	1e1959467e	fix scaling algorithm in FP8 casting	2024-07-12 15:23:37 +08:00
HangXu	f5a52e1600	fp8 operators for compressed communication ... cast_to_fp8, cast_from_fp8, all_reduce_fp8	2024-07-01 13:44:21 +08:00
linsj20	91fa553775	[Feature] qlora support (#5586) ... * [feature] qlora support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * qlora follow commit * migrate qutization folder to colossalai/ * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fixes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2024-04-28 10:51:27 +08:00