save_for_auto_inference save_for_auto_inference ( path_prefix, dist_model, cvt2cpu=False ) [source]

Save model parameters for auto parallel inference. Supporting dp + mp + pp + sharding(stage1), dp + sharding stage2-3. MoE not sdupported till MoE is supported in auto parallel mode.

System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/incubate/distributed/utils/io/ of, line 5)

Definition list ends without a blank line; unexpected unindent.

  • path_prefix

    path prefix to save If path_preifx ends with path sepreator,

    System Message: ERROR/3 (/usr/local/lib/python3.8/site-packages/paddle/incubate/distributed/utils/io/ of, line 7)

    Unexpected indentation.

    the path is processed as a directory and parameters will be saved in it, automatically named saved_parameters.

    System Message: WARNING/2 (/usr/local/lib/python3.8/site-packages/paddle/incubate/distributed/utils/io/ of, line 9)

    Block quote ends without a blank line; unexpected unindent.

    Otherwisw, the parameters will be saved with name

    path_preifx_dist{global_rank}.pdparams and path_preifx_dist{global_rank}.pdattrs

  • dist_model – model in distributed modeß

  • cvt2cpu – wheather to move parameters to CPU when using sharding stage 3. The var is invalid if not using sharding stage 3.




dist_model = build_distributed_model()

path_prefix = “path/to/save_infer”

save_for_auto_inference(path_prefix, dist_model=dist_model, original_model=single_model, cvt2cpu=False)


path/to/save_infer_dist0.pdparams path/to/save_infer_dist1.pdparams path/to/save_infer_dist2.pdparams … path/to/save_infer_dist0.pdattr path/to/save_infer_dist1.pdattr path/to/save_infer_dist2.pdattr …