Overview
This directory includes two parts: Using the Booster API finetune Huggingface Bert and AlBert models and benchmarking Bert and AlBert models with different Booster Plugin.
Finetune
bash test_ci.sh
Bert-Finetune Results
| Plugin |
Accuracy |
F1-score |
GPU number |
| torch_ddp |
84.4% |
88.6% |
2 |
| torch_ddp_fp16 |
84.7% |
88.8% |
2 |
| gemini |
84.0% |
88.4% |
2 |
| hybrid_parallel |
84.5% |
88.6% |
4 |
Benchmark
bash benchmark.sh
Now include these metrics in benchmark: CUDA mem occupy, throughput and the number of model parameters. If you have custom metrics, you can add them to benchmark_util.
Results
Bert
|
max cuda mem |
throughput(sample/s) |
params |
| ddp |
21.44 GB |
3.0 |
82M |
| ddp_fp16 |
16.26 GB |
11.3 |
82M |
| gemini |
11.0 GB |
12.9 |
82M |
| low_level_zero |
11.29 G |
14.7 |
82M |
AlBert
|
max cuda mem |
throughput(sample/s) |
params |
| ddp |
OOM |
|
|
| ddp_fp16 |
OOM |
|
|
| gemini |
69.39 G |
1.3 |
208M |
| low_level_zero |
56.89 G |
1.4 |
208M |
d202cc28c0