Merge branch 'main' into sync/npu

This commit is contained in:
ver217
2024-01-18 12:05:21 +08:00
152 changed files with 8641 additions and 2138 deletions

View File

@@ -73,4 +73,4 @@ And some models are not supported at all which will raise an error. We tested mo
| Blip2Model | transformers |
| Blip2ForConditionalGeneration | transformers |
<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_iniy.py -->
<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_init.py -->

View File

@@ -167,7 +167,7 @@ plugin = HybridParallelPlugin(tp_size=1,
booster = Booster(plugin=plugin)
```
Boost these train componts with the booster created.
Boost these train components with the booster created.
```python
model, optimizer, _criterion, _, lr_scheduler = booster.boost(model,
optimizer,

View File

@@ -178,6 +178,18 @@ Model/Feature Compatibility Matrix:
<td nowrap="nowrap" align="center"></td>
<td nowrap="nowrap" align="center"></td>
</tr>
<tr>
<td nowrap="nowrap">Falcon</td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center"></td>
<td nowrap="nowrap" align="center">✔️</td>
<td nowrap="nowrap" align="center"></td>
<td nowrap="nowrap" align="center"></td>
</tr>
<tr>
<td colspan="39"></td>
</tr>

View File

@@ -1,7 +1,7 @@
# Setup
Requirements:
- PyTorch >= 1.11 (PyTorch 2.x in progress)
- PyTorch >= 1.11 and PyTorch <= 2.1
- Python >= 3.7
- CUDA >= 11.0
- [NVIDIA GPU Compute Capability](https://developer.nvidia.com/cuda-gpus) >= 7.0 (V100/RTX20 and higher)