[legacy] clean up legacy code (#4743)

* [legacy] remove outdated codes of pipeline (#4692)

* [legacy] remove cli of benchmark and update optim (#4690)

* [legacy] remove cli of benchmark and update optim

* [doc] fix cli doc test

* [legacy] fix engine clip grad norm

* [legacy] remove outdated colo tensor (#4694)

* [legacy] remove outdated colo tensor

* [test] fix test import

* [legacy] move outdated zero to legacy (#4696)

* [legacy] clean up utils (#4700)

* [legacy] clean up utils

* [example] update examples

* [legacy] clean up amp

* [legacy] fix amp module

* [legacy] clean up gpc (#4742)

* [legacy] clean up context

* [legacy] clean core, constants and global vars

* [legacy] refactor initialize

* [example] fix examples ci

* [example] fix examples ci

* [legacy] fix tests

* [example] fix gpt example

* [example] fix examples ci

* [devops] fix ci installation

* [example] fix examples ci

docs/README.md

@@ -108,5 +108,5 @@ We support `autodoc` to extract the docstring and transform it into a Web elemen
 You just need to add `{{ autodoc:<mod-name> }}` in your markdown as a single line. An example is given below and you can see the outcome in [this PR](https://github.com/hpcaitech/ColossalAI-Documentation/pull/175).
 
 ```markdown
-{{ autodoc:colossalai.amp.apex_amp.convert_to_apex_amp }}
+{{ autodoc:colossalai.legacy.amp.apex_amp.convert_to_apex_amp }}
 ```

docs/source/en/advanced_tutorials/add_your_parallel.md

@@ -31,7 +31,7 @@ global context for users to easily manage their process groups. If you wish to a
 define a new class and set it in your configuration file. To define your own way of creating process groups, you can
 follow the steps below to create a new distributed initialization.
 
-1. Add your parallel mode in `colossalai.context.parallel_mode.ParallelMode`.
+1. Add your parallel mode in `colossalai.legacy.context.parallel_mode.ParallelMode`.
     ```python
     class ParallelMode(Enum):
         GLOBAL = 'global'

docs/source/en/advanced_tutorials/train_gpt_using_hybrid_parallelism.md

@@ -37,7 +37,7 @@ import torch.nn as nn
 from colossalai import nn as col_nn
 from colossalai.amp import AMP_TYPE
 from colossalai.legacy.builder.pipeline import partition_uniform
-from colossalai.context.parallel_mode import ParallelMode
+from colossalai.legacy.context.parallel_mode import ParallelMode
 from colossalai.core import global_context as gpc
 from colossalai.legacy.engine.schedule import (InterleavedPipelineSchedule,
                                         PipelineSchedule)

docs/source/en/basics/command_line_tool.md

@@ -30,24 +30,4 @@ This command will inform you information regarding the version compatibility and
 To launch distributed jobs on single or multiple nodes, the command `colossalai run` can be used for process launching.
 You may refer to [Launch Colossal-AI](./launch_colossalai.md) for more details.
 
-## Tensor Parallel Micro-Benchmarking
-
-As Colossal-AI provides an array of tensor parallelism methods, it is not intuitive to choose one for your hardware and
-model. Therefore, we provide a simple benchmarking to evaluate the performance of various tensor parallelisms on your system.
-This benchmarking is run on a simple MLP model where the input data is of the shape `(batch_size, seq_length, hidden_size)`.
-Based on the number of GPUs, the CLI will look for all possible tensor parallel configurations and display the benchmarking results.
-You can customize the benchmarking configurations by checking out `colossalai benchmark --help`.
-
-```shell
-# run on 4 GPUs
-colossalai benchmark --gpus 4
-
-# run on 8 GPUs
-colossalai benchmark --gpus 8
-```
-
-:::caution
-
-Only single-node benchmarking is supported currently.
-
-:::
+<!-- doc-test-command: echo  -->

docs/source/zh-Hans/advanced_tutorials/add_your_parallel.md

@@ -24,7 +24,7 @@
 并行通常由进程组来管理，参与相同并行算法的进程被置于同一进程组。对于不同的并行算法，需要创建不同的进程组。
 Colossal-AI 为用户提供了一个全局 context，使他们能够轻松地管理进程组。如果你想添加新的进程组，你可以很容易地定义一个新的类并在你的配置文件中设置它。为了定义你自己的进程组创建方式，你可以按照下面的步骤来创建一个新的分布式初始化。
 
-1. 在 `colossalai.context.parallel_mode.ParallelMode` 中添加你自己的并行模式。
+1. 在 `colossalai.legacy.context.parallel_mode.ParallelMode` 中添加你自己的并行模式。
     ```python
     class ParallelMode(Enum):
         GLOBAL = 'global'

docs/source/zh-Hans/advanced_tutorials/train_gpt_using_hybrid_parallelism.md

@@ -37,7 +37,7 @@ import torch.nn as nn
 from colossalai import nn as col_nn
 from colossalai.amp import AMP_TYPE
 from colossalai.legacy.builder.pipeline import partition_uniform
-from colossalai.context.parallel_mode import ParallelMode
+from colossalai.legacy.context.parallel_mode import ParallelMode
 from colossalai.core import global_context as gpc
 from colossalai.legacy.engine.schedule import (InterleavedPipelineSchedule,
                                         PipelineSchedule)

docs/source/zh-Hans/basics/command_line_tool.md

@@ -26,22 +26,4 @@ Colossal-AI给用户提供了命令行工具，目前命令行工具可以用来
 
 在分布式训练时，我们可以使用`colossalai run`来启动单节点或者多节点的多进程，详细的内容可以参考[启动 Colossal-AI](./launch_colossalai.md)。
 
-## 张量并行基准测试
-
-Colossal-AI提供了多种张量并行，想要充分理解这些方法需要一定的学习成本，对于新手来说很难靠经验选择一个并行方式。
-所以我们提供了一个简单的基准测试，能够让用户在自己的机器上测试不同张量并行的性能。这个基准测试跑一个并行的MLP模型，
-输入数据的维度为`（批大小，序列长度，隐藏层维度）`。通过指定GPU的数量，Colossal-AI会搜索所有可行的并行配置。用户可以通过查看`colossalai benchmark --help`来自定义相关的测试参数。
-
-```shell
-# 使用4个GPU
-colossalai benchmark --gpus 4
-
-# 使用8个GPU
-colossalai benchmark --gpus 8
-```
-
-:::caution
-
-目前仅支持单节点的基准测试。
-
-:::
+<!-- doc-test-command: echo  -->