[组合替代实现]fairscale.nn.model_parallel.initialize.model_parallel_is_initialized¶

fairscale.nn.model_parallel.initialize.model_parallel_is_initialized ¶

          fairscale.nn.model_parallel.initialize.model_parallel_is_initialized()

         

返回模型并行初始化设置是否完成; Paddle 无此 API，需要组合实现。

转写示例¶

          # Pytorch 写法
fairscale.nn.model_parallel.initialize.model_parallel_is_initialized()

# Paddle 写法
paddle.distributed.fleet.base.topology._HYBRID_PARALLEL_GROUP is not None