github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-03 18:46:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

[misc] remove useless condition

Browse Source
This commit is contained in:
haze188 2024-07-25 08:40:53 +00:00 committed by Hongxin Liu
parent 70793ce9ed
commit 7e737df5ad
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
colossalai/shardformer/policies/mixtral.py
View File

@ -61,7 +61,7 @@ class MixtralPolicy(Policy):
policy[attn_cls] = ModulePolicyDescription(
attribute_replacement=decoder_attribute_replacement,
)
if self.shard_config.enable_flash_attention or self.shard_config.enable_sequence_parallelism:
if self.shard_config.enable_sequence_parallelism:
if self.pipeline_stage_manager is not None:
# NOTE: we are replacing model forward for both sequence parallelism and pipeline parallelism
# if both are enabled, one of them will be ignored