mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2025-04-27 19:36:13 +00:00
df5e9c53cf
* Add dpo. Fix sft, ppo, lora. Refactor all * fix and tested ppo * 2 nd round refactor * add ci tests * fix ci * fix ci * fix readme, style * fix readme style * fix style, fix benchmark * reproduce benchmark result, remove useless files * rename to ColossalChat * use new image * fix ci workflow * fix ci * use local model/tokenizer for ci tests * fix ci * fix ci * fix ci * fix ci timeout * fix rm progress bar. fix ci timeout * fix ci * fix ci typo * remove 3d plugin from ci temporary * test environment * cannot save optimizer * support chat template * fix readme * fix path * test ci locally * restore build_or_pr * fix ci data path * fix benchmark * fix ci, move ci tests to 3080, disable fast tokenizer * move ci to 85 * support flash attention 2 * add all-in-one data preparation script. Fix colossal-llama2-chat chat template * add hardware requirements * move ci test data * fix save_model, add unwrap * fix missing bos * fix missing bos; support grad accumulation with gemini * fix ci * fix ci * fix ci * fix llama2 chat template config * debug sft * debug sft * fix colossalai version requirement * fix ci * add sanity check to prevent NaN loss * fix requirements * add dummy data generation script * add dummy data generation script * add dummy data generation script * add dummy data generation script * update readme * update readme * update readme and ignore * fix logger bug * support parallel_output * modify data preparation logic * fix tokenization * update lr * fix inference * run pre-commit --------- Co-authored-by: Tong Li <tong.li352711588@gmail.com>
44 lines
1.1 KiB
Python
Executable File
44 lines
1.1 KiB
Python
Executable File
from setuptools import find_packages, setup
|
|
|
|
|
|
def fetch_requirements(path):
|
|
with open(path, "r") as fd:
|
|
return [r.strip() for r in fd.readlines()]
|
|
|
|
|
|
def fetch_readme():
|
|
with open("README.md", encoding="utf-8") as f:
|
|
return f.read()
|
|
|
|
|
|
def fetch_version():
|
|
with open("version.txt", "r") as f:
|
|
return f.read().strip()
|
|
|
|
|
|
setup(
|
|
name="coati",
|
|
version=fetch_version(),
|
|
packages=find_packages(
|
|
exclude=(
|
|
"tests",
|
|
"benchmarks",
|
|
"*.egg-info",
|
|
)
|
|
),
|
|
description="Colossal-AI Talking Intelligence",
|
|
long_description=fetch_readme(),
|
|
long_description_content_type="text/markdown",
|
|
license="Apache Software License 2.0",
|
|
url="https://github.com/hpcaitech/Coati",
|
|
install_requires=fetch_requirements("requirements.txt"),
|
|
python_requires=">=3.7",
|
|
classifiers=[
|
|
"Programming Language :: Python :: 3",
|
|
"License :: OSI Approved :: Apache Software License",
|
|
"Environment :: GPU :: NVIDIA CUDA",
|
|
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
|
"Topic :: System :: Distributed Computing",
|
|
],
|
|
)
|