github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-09-28 21:17:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
3defa32aee5c8cac42b0625df258254d11cfaad7
ColossalAI/docs/colossalai
History
Frank Lee 3defa32aee Support TP-compatible Torch AMP and Update trainer API (#27) 
* Add gradient accumulation, fix lr scheduler

* fix FP16 optimizer and adapted torch amp with tensor parallel (#18)

* fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes

* fixed trainer

* Revert "fixed trainer"

This reverts commit 2e0b0b7699.

* improved consistency between trainer, engine and schedule (#23)

Co-authored-by: 1SAA <c2h214748@gmail.com>

Co-authored-by: 1SAA <c2h214748@gmail.com>
Co-authored-by: ver217 <lhx0217@gmail.com>
2021-11-18 19:45:06 +08:00
..
colossalai.builder.builder.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.builder.pipeline.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.builder.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.checkpointing.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.communication.collective.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.communication.p2p.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.communication.ring.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.communication.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.communication.utils.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.constants.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.config.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.parallel_context.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.parallel_mode.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_1d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_2d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_2p5d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_3d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_data.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_pipeline.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_sequence.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.initializer_tensor.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.process_group_initializer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.process_group_initializer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.random.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.random.seed_manager.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.context.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.core.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.engine.amp.amp_type.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.engine.amp.grad_scaler.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.engine.amp.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.engine.gradient_handler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.engine.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.engine.schedule.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.initialize.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.logging.logging.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.logging.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.base_dataset.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.caltech101_dataset.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.cifar10_dataset.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.rst
update documentation
2021-10-29 09:29:20 +08:00
colossalai.nn.data.sampler.base_sampler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.sampler.data_parallel_sampler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.data.sampler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.base_layer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_1d.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_1d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_2d.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_2d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_2p5d.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_2p5d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_3d.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_3d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_sequence.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_sequence.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_vision_transformer.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.parallel_vision_transformer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_resnet.basic_block.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_resnet.bottleneck.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_resnet.conv.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_resnet.reslayer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_resnet.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_vision_transformer.layers.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.vanilla_vision_transformer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.wrapper.lambda_wrapper.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.layer.wrapper.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.base_loss.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.cross_entropy_1d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.cross_entropy_2d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.cross_entropy_2p5d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.cross_entropy_3d.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.loss.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.cosine.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.delayed.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.linear.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.multistep.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.onecycle.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.poly.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.lr_scheduler.torch.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.model.base_model.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.model.rst
update documentation
2021-10-29 09:29:20 +08:00
colossalai.nn.model.vanilla_resnet.resnet.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.model.vanilla_resnet.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.model.vision_transformer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.model.vision_transformer.vision_transformer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.multi_tensor_apply.multi_tensor_apply.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.multi_tensor_apply.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.fp16_optimizer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.fused_adam.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.fused_lamb.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.fused_sgd.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.lamb.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.lars.rst
update documentation
2021-10-29 09:29:20 +08:00
colossalai.nn.optimizer.loss_scaler.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.rst
update documentation
2021-10-29 09:29:20 +08:00
colossalai.nn.optimizer.zero_redundancy_optimizer_level_1.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.zero_redundancy_optimizer_level_2.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.optimizer.zero_redundancy_optimizer_level_3.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.nn.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.registry.registry.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.registry.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.trainer.hooks.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.trainer.metric.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.trainer.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.utils.activation_checkpoint.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.utils.checkpointing.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.utils.common.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.utils.cuda.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.utils.memory.rst
Migrated project
2021-10-28 18:21:23 +02:00
colossalai.utils.rst
Support TP-compatible Torch AMP and Update trainer API (#27)
2021-11-18 19:45:06 +08:00
colossalai.utils.timer.rst
Migrated project
2021-10-28 18:21:23 +02:00