github/ColossalAI
1
0
Fork 0
mirror of https://github.com/hpcaitech/ColossalAI.git synced 2025-06-22 13:41:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

add how to build tfrecord dataset (#48)

Browse Source
This commit is contained in:
ver217 2021-12-02 16:31:23 +08:00 committed by GitHub
parent 4da256a584
commit eb2f8b1f6b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
examples/vit-b16/README.md
View File

@ -47,3 +47,8 @@ Since Mixup is used as data augmentation, we define the loss function of Mixup h
`hooks.py`
We define hook functions that record running information to help debugging.
# How to build TFRecords dataset
As we use [DALI](https://github.com/NVIDIA/DALI) to read data, we use the TFRecords dataset instead of raw Imagenet dataset. If you don't have TFRecords dataset, follow [imagenet-tools](https://github.com/ver217/imagenet-tools) to build one.