github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-07-31 15:25:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix llama pretrain (#5287)

Browse Source
This commit is contained in:
flybird11111 2024-01-19 17:49:02 +08:00 committed by GitHub
parent 6a56967855
commit f7e3f82a7e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
examples/language/llama2/pretrain.py
View File

@ -273,11 +273,10 @@ def main():
dataloader.sampler.set_start_index(sampler_start_idx)
for epoch in range(start_epoch, args.num_epochs):
dataloader.sampler.set_epoch(epoch)
step_nums = num_steps_per_epoch - start_step
dataloader_iter = iter(dataloader)
with tqdm(
range(step_nums),
range(start_step, num_steps_per_epoch),
desc=f"Epoch {epoch}",
disable=not print_flag,
total=num_steps_per_epoch,