From 6df844b8c4946c734115b7e180b292888d857bc1 Mon Sep 17 00:00:00 2001
From: binmakeswell <binmakeswell@gmail.com>
Date: Fri, 22 Mar 2024 15:48:12 +0800
Subject: [PATCH] [release] grok-1 314b inference (#5490)
* [release] grok-1 inference
* [release] grok-1 inference
* [release] grok-1 inference
---
README.md | 8 ++++++++
docs/README-zh-Hans.md | 8 ++++++++
examples/language/grok-1/README.md | 5 +++++
3 files changed, 21 insertions(+)
diff --git a/README.md b/README.md
index 7c234b15e..d79ec9a66 100644
--- a/README.md
+++ b/README.md
@@ -25,6 +25,7 @@
</div>
## Latest News
+* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
* [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0)
* [2024/03] [Open-Sora：Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora)
* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer)
@@ -72,6 +73,7 @@
<li>
<a href="#Inference">Inference</a>
<ul>
+ <li><a href="#Grok-1">Grok-1: 314B model of PyTorch + HuggingFace Inference</a></li>
<li><a href="#SwiftInfer">SwiftInfer:Breaks the Length Limit of LLM for Multi-Round Conversations with 46% Acceleration</a></li>
<li><a href="#GPT-3-Inference">GPT-3</a></li>
<li><a href="#OPT-Serving">OPT-175B Online Serving for Text Generation</a></li>
@@ -365,6 +367,12 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt
## Inference
+### Grok-1
+An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1 Inference.
+[[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
+[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
+[[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1)
+
<p id="SwiftInfer" align="center">
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/>
</p>
diff --git a/docs/README-zh-Hans.md b/docs/README-zh-Hans.md
index 93045ea6a..352564446 100644
--- a/docs/README-zh-Hans.md
+++ b/docs/README-zh-Hans.md
@@ -24,6 +24,7 @@
</div>
## 新闻
+* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
* [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0)
* [2024/03] [Open-Sora：Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora)
* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer)
@@ -71,6 +72,7 @@
<li>
<a href="#推理">推理</a>
<ul>
+ <li><a href="#Grok-1">Grok-1: 3140亿参数PyTorch + HuggingFace推理</a></li>
<li><a href="#SwiftInfer">SwiftInfer:打破LLM多轮对话的长度限制，推理加速46%</a></li>
<li><a href="#GPT-3-Inference">GPT-3</a></li>
<li><a href="#OPT-Serving">1750亿参数OPT在线推理服务</a></li>
@@ -358,6 +360,12 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的
## 推理
+### Grok-1
+方便易用的Python + PyTorch + HuggingFace Grok-1 推理
+[[代码]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
+[[博客]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
+[[HuggingFace Grok-1 PyTorch模型权重]](https://huggingface.co/hpcai-tech/grok-1)
+
<p id="SwiftInfer" align="center">
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/>
</p>
diff --git a/examples/language/grok-1/README.md b/examples/language/grok-1/README.md
index c523f9412..1d234e8eb 100644
--- a/examples/language/grok-1/README.md
+++ b/examples/language/grok-1/README.md
@@ -1,5 +1,10 @@
# Grok-1 Inference
+An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1.
+[[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
+[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
+[[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1)
+
## Install
```bash