mirror of
https://github.com/hpcaitech/ColossalAI.git
synced 2025-09-02 17:46:42 +00:00
Merge branch 'main' into sync/npu
This commit is contained in:
@@ -23,7 +23,7 @@
|
||||
|
||||
Booster 插件是管理并行配置的重要组件(eg:gemini 插件封装了 gemini 加速方案)。目前支持的插件如下:
|
||||
|
||||
**_HybridParallelPlugin:_** HybirdParallelPlugin 插件封装了混合并行的加速解决方案。它提供的接口可以在张量并行,流水线并行以及两种数据并行方法(DDP, Zero)间进行任意的组合。
|
||||
**_HybridParallelPlugin:_** HybridParallelPlugin 插件封装了混合并行的加速解决方案。它提供的接口可以在张量并行,流水线并行以及两种数据并行方法(DDP, Zero)间进行任意的组合。
|
||||
|
||||
**_GeminiPlugin:_** GeminiPlugin 插件封装了 gemini 加速解决方案,即基于块内存管理的 ZeRO 优化方案。
|
||||
|
||||
|
@@ -73,4 +73,4 @@ model, *_ = booster.boost(model)
|
||||
| Blip2Model | transformers |
|
||||
| Blip2ForConditionalGeneration | transformers |
|
||||
|
||||
<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_iniy.py -->
|
||||
<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_init.py -->
|
||||
|
@@ -174,6 +174,18 @@ Author: [Baizhou Zhang](https://github.com/Fridge003), [Bin Jia](https://github.
|
||||
<td nowrap="nowrap" align="center">❌</td>
|
||||
<td nowrap="nowrap" align="center">❌</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td nowrap="nowrap">Falcon</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">❌</td>
|
||||
<td nowrap="nowrap" align="center">✔️</td>
|
||||
<td nowrap="nowrap" align="center">❌</td>
|
||||
<td nowrap="nowrap" align="center">❌</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td colspan="39"></td>
|
||||
</tr>
|
||||
|
@@ -2,7 +2,7 @@
|
||||
|
||||
环境要求:
|
||||
|
||||
- PyTorch >= 1.11 (PyTorch 2.x 正在适配中)
|
||||
- PyTorch >= 1.11 并且 PyTorch <= 2.1
|
||||
- Python >= 3.7
|
||||
- CUDA >= 11.0
|
||||
- [NVIDIA GPU Compute Capability](https://developer.nvidia.com/cuda-gpus) >= 7.0 (V100/RTX20 and higher)
|
||||
|
Reference in New Issue
Block a user