\u200E
  • 开始使用
  • 特性
  • 文档
    • API
    • 使用指南
  • 工具平台
    • 工具
      • AutoDL
  • develop
  • 1.7
  • 1.6
  • 1.5
  • 1.4
  • 1.3
  • 1.2
  • 1.1
  • 1.0
  • 0.15.0
  • 0.14.0
  • 0.13.0
  • 0.12.0
  • 0.11.0
  • 0.10.0
  • 中文(简)
  • English(En)
  • Installation Manuals
    • Install on Ubuntu
    • Install on CentOS
    • Install on MacOS
    • Installation on Windows
    • Docker Installation
    • Compile From Source Code
      • Compile on Ubuntu from Source Code
      • Compile on CentOS from Source Code
      • Compile on MacOS from Source Code
      • Compile on Windows from Source Code
    • Appendix
  • Beginner’s Guide
    • Basic Concept
      • Guide to Fluid Programming
      • LoD-Tensor User Guide
    • Coding Practice
      • Set up Simple Model
      • Single-node training
        • Evaluate model while training
      • Evaluate model while training
      • Save, Load Models or Variables & Incremental Learning
  • User Guides
    • Simple Case
      • Linear Regression
      • Recognize Digits
      • Word Vector
    • Computer Vision
      • Image Classification
      • Generative Adversarial Network
    • Natural Language Processing
      • Sentiment Analysis
      • Label Semantic Roles
      • Machine Translation
    • Recommend
      • Recommender System
  • Advanced User Guides
    • Prepare Data
      • Prepare Steps
      • Python Reader
      • Asynchronous Data Reading
      • Take Numpy Array as Training Data
    • Distributed Training
      • Quick Start with Distributed Training
      • Manual for Distributed Training with Fluid
    • Deploy Inference Model
      • Server-side Deployment
        • Install and Compile C++ Inference Library on Linux
        • Install and Compile C++ Inference Library on Windows
        • Introduction to C++ Inference API
        • Performance Profiling for TensorRT Library
      • Model Compression
    • Practice Improving
      • Memory Allocation and Optimization
      • Best practices of distributed training on CPU
      • Recompute: Training with bigger batch size
      • Use Paddle-TensorRT Library for inference
      • Performance Profiling and Optimization
        • Tune CPU performance
        • Heap Memory Profiling and Optimization
        • How to use timeline tool to do profile
    • Model Evaluation and Debugging
      • Model Evaluation
      • VisualDL Tools
        • Introduction to VisualDL Toolset
        • VisualDL user guide
    • Addon Development
      • Design Principles of Fluid
      • Write New Operators
        • How to write a new operator
        • Notes on operator development
      • How to contribute codes to Paddle
        • Guide of local development
        • Guide of submitting PR to Github
    • FLAGS
      • cudnn
      • data processing
      • debug
        • check nan inf tool
      • device management
      • distributed
      • executor
      • memory management
      • others
  • API Reference
    • API Quick Search
      • Basic Concept
      • Neural Network Layer
        • Convolution
        • Pooling
        • Image Detection
        • Sequence
        • Mathematical operation
        • Activation Function
        • Loss function
        • Data input and output
        • Control Flow
        • Sparse update
        • Feed training/inference data with DataFeeder
        • Learning rate scheduler
        • Tensor
      • Complex Networks
      • Optimizer
      • Back Propagation
      • Metrics
      • Save and Load a Model
      • Inference Engine
      • Executor
      • Parallel Executor
      • CompiledProgram
      • Model Parameters
      • Distributed Training
        • Synchronous Distributed Training
        • Asynchronous Distributed Training
        • Training of Models with Large Scale Sparse Features
        • Preparing Data Reader for Distributed Training
    • fluid
      • BuildStrategy
      • CompiledProgram
      • cpu_places
      • CPUPlace
      • create_lod_tensor
      • create_random_int_lodtensor
      • cuda_pinned_places
      • cuda_places
      • CUDAPinnedPlace
      • CUDAPlace
      • data
      • DataFeedDesc
      • DataFeeder
      • default_main_program
      • default_startup_program
      • DistributeTranspiler
      • DistributeTranspilerConfig
      • embedding
      • ExecutionStrategy
      • Executor
      • global_scope
      • gradients
      • in_dygraph_mode
      • is_compiled_with_cuda
      • load
      • load_op_library
      • LoDTensor
      • LoDTensorArray
      • memory_optimize
      • name_scope
      • one_hot
      • ParallelExecutor
      • ParamAttr
      • Program
      • program_guard
      • release_memory
      • require_version
      • save
      • scope_guard
      • Tensor
      • Variable
      • WeightNormParamAttr
    • fluid.backward
      • append_backward
      • gradients
    • fluid.clip
      • ErrorClipByValue
      • GradientClipByGlobalNorm
      • GradientClipByNorm
      • GradientClipByValue
      • set_gradient_clip
    • Data Reader
      • Reader
    • dataset
      • cifar
      • conll05
      • imdb
      • imikolov
      • movielens
      • sentiment
      • uci_housing
      • wmt14
      • wmt16
    • fluid.dataset
      • DatasetFactory
      • InMemoryDataset
      • QueueDataset
    • fluid.dygraph
      • BackwardStrategy
      • BatchNorm
      • BilinearTensorProduct
      • Conv2D
      • Conv2DTranspose
      • Conv3D
      • Conv3DTranspose
      • CosineDecay
      • dygraph_to_static_output
      • Embedding
      • ExponentialDecay
      • GroupNorm
      • GRUUnit
      • guard
      • InverseTimeDecay
      • Layer
      • LayerList
      • LayerNorm
      • Linear
      • load_dygraph
      • NaturalExpDecay
      • NCE
      • no_grad
      • NoamDecay
      • ParameterList
      • PiecewiseDecay
      • PolynomialDecay
      • Pool2D
      • PRelu
      • prepare_context
      • save_dygraph
      • Sequential
      • SpectralNorm
      • to_variable
      • TracedLayer
      • TreeConv
    • fluid.executor
      • Executor
      • global_scope
      • scope_guard
    • fluid.initializer
      • Bilinear
      • BilinearInitializer
      • Constant
      • ConstantInitializer
      • force_init_on_cpu
      • init_on_cpu
      • MSRA
      • MSRAInitializer
      • Normal
      • NormalInitializer
      • NumpyArrayInitializer
      • TruncatedNormal
      • TruncatedNormalInitializer
      • Uniform
      • UniformInitializer
      • Xavier
      • XavierInitializer
    • fluid.io
      • batch
      • buffered
      • cache
      • chain
      • compose
      • ComposeNotAligned
      • DataLoader
      • firstn
      • get_program_parameter
      • get_program_persistable_vars
      • load
      • load_inference_model
      • load_params
      • load_persistables
      • load_program_state
      • load_vars
      • map_readers
      • multiprocess_reader
      • PyReader
      • save
      • save_inference_model
      • save_params
      • save_persistables
      • save_vars
      • set_program_state
      • shuffle
      • xmap_readers
    • fluid.layers
      • abs
      • accuracy
      • acos
      • adaptive_pool2d
      • adaptive_pool3d
      • add_position_encoding
      • affine_channel
      • affine_grid
      • anchor_generator
      • argmax
      • argmin
      • argsort
      • array_length
      • array_read
      • array_write
      • asin
      • assign
      • atan
      • auc
      • autoincreased_step_counter
      • batch_norm
      • beam_search
      • beam_search_decode
      • BeamSearchDecoder
      • bilinear_tensor_product
      • bipartite_match
      • box_clip
      • box_coder
      • box_decoder_and_assign
      • bpr_loss
      • brelu
      • case
      • cast
      • Categorical
      • ceil
      • center_loss
      • chunk_eval
      • clip
      • clip_by_norm
      • collect_fpn_proposals
      • concat
      • cond
      • continuous_value_model
      • conv2d
      • conv2d_transpose
      • conv3d
      • conv3d_transpose
      • cos
      • cos_sim
      • cosine_decay
      • create_array
      • create_global_var
      • create_parameter
      • create_py_reader_by_data
      • create_tensor
      • crf_decoding
      • crop
      • crop_tensor
      • cross_entropy
      • ctc_greedy_decoder
      • cumsum
      • data
      • data_norm
      • Decoder
      • deformable_conv
      • deformable_roi_pooling
      • density_prior_box
      • detection_output
      • diag
      • dice_loss
      • distribute_fpn_proposals
      • double_buffer
      • dropout
      • dynamic_decode
      • dynamic_gru
      • dynamic_lstm
      • dynamic_lstmp
      • DynamicRNN
      • edit_distance
      • elementwise_add
      • elementwise_div
      • elementwise_floordiv
      • elementwise_max
      • elementwise_min
      • elementwise_mod
      • elementwise_mul
      • elementwise_pow
      • elementwise_sub
      • elu
      • embedding
      • equal
      • erf
      • exp
      • expand
      • expand_as
      • exponential_decay
      • eye
      • fc
      • fill_constant
      • fill_constant_batch_size_like
      • filter_by_instag
      • flatten
      • floor
      • fsp_matrix
      • gather
      • gather_nd
      • gather_tree
      • gaussian_random
      • gaussian_random_batch_size_like
      • gelu
      • generate_mask_labels
      • generate_proposal_labels
      • generate_proposals
      • get_tensor_from_selected_rows
      • greater_equal
      • greater_than
      • grid_sampler
      • group_norm
      • gru_unit
      • GRUCell
      • hard_shrink
      • hard_sigmoid
      • hard_swish
      • has_inf
      • has_nan
      • hash
      • hsigmoid
      • huber_loss
      • IfElse
      • im2sequence
      • image_resize
      • image_resize_short
      • increment
      • instance_norm
      • inverse_time_decay
      • iou_similarity
      • is_empty
      • isfinite
      • kldiv_loss
      • l2_normalize
      • label_smooth
      • layer_norm
      • leaky_relu
      • less_equal
      • less_than
      • linear_chain_crf
      • linear_lr_warmup
      • linspace
      • load
      • locality_aware_nms
      • lod_append
      • lod_reset
      • log
      • log_loss
      • logical_and
      • logical_not
      • logical_or
      • logical_xor
      • logsigmoid
      • lrn
      • lstm
      • lstm_unit
      • LSTMCell
      • margin_rank_loss
      • matmul
      • maxout
      • mean
      • mean_iou
      • merge_selected_rows
      • mse_loss
      • mul
      • multi_box_head
      • multiclass_nms
      • multiplex
      • MultivariateNormalDiag
      • natural_exp_decay
      • nce
      • noam_decay
      • Normal
      • not_equal
      • npair_loss
      • one_hot
      • ones
      • ones_like
      • pad
      • pad2d
      • pad_constant_like
      • piecewise_decay
      • pixel_shuffle
      • polygon_box_transform
      • polynomial_decay
      • pool2d
      • pool3d
      • pow
      • prelu
      • Print
      • prior_box
      • prroi_pool
      • psroi_pool
      • py_func
      • py_reader
      • random_crop
      • range
      • rank
      • rank_loss
      • read_file
      • reciprocal
      • reduce_all
      • reduce_any
      • reduce_max
      • reduce_mean
      • reduce_min
      • reduce_prod
      • reduce_sum
      • relu
      • relu6
      • reorder_lod_tensor_by_rank
      • reshape
      • resize_bilinear
      • resize_nearest
      • resize_trilinear
      • retinanet_detection_output
      • retinanet_target_assign
      • reverse
      • rnn
      • RNNCell
      • roi_align
      • roi_perspective_transform
      • roi_pool
      • round
      • row_conv
      • rpn_target_assign
      • rsqrt
      • sampled_softmax_with_cross_entropy
      • sampling_id
      • scale
      • scatter
      • scatter_nd
      • scatter_nd_add
      • selu
      • sequence_concat
      • sequence_conv
      • sequence_enumerate
      • sequence_expand
      • sequence_expand_as
      • sequence_first_step
      • sequence_last_step
      • sequence_mask
      • sequence_pad
      • sequence_pool
      • sequence_reshape
      • sequence_reverse
      • sequence_scatter
      • sequence_slice
      • sequence_softmax
      • sequence_unpad
      • shape
      • shard_index
      • shuffle_channel
      • sigmoid
      • sigmoid_cross_entropy_with_logits
      • sigmoid_focal_loss
      • sign
      • similarity_focus
      • sin
      • size
      • slice
      • smooth_l1
      • soft_relu
      • softmax
      • softmax_with_cross_entropy
      • softplus
      • softshrink
      • softsign
      • space_to_depth
      • spectral_norm
      • split
      • sqrt
      • square
      • square_error_cost
      • squeeze
      • ssd_loss
      • stack
      • stanh
      • StaticRNN
      • strided_slice
      • sum
      • sums
      • swish
      • Switch
      • switch_case
      • tanh
      • tanh_shrink
      • target_assign
      • teacher_student_sigmoid_loss
      • temporal_shift
      • tensor_array_to_tensor
      • thresholded_relu
      • topk
      • transpose
      • unfold
      • Uniform
      • uniform_random
      • uniform_random_batch_size_like
      • unique
      • unique_with_counts
      • unsqueeze
      • unstack
      • warpctc
      • where
      • While
      • while_loop
      • yolo_box
      • yolov3_loss
      • zeros
      • zeros_like
    • fluid.metrics
      • Accuracy
      • Auc
      • ChunkEvaluator
      • CompositeMetric
      • DetectionMAP
      • EditDistance
      • MetricBase
      • Precision
      • Recall
    • fluid.nets
      • glu
      • img_conv_group
      • scaled_dot_product_attention
      • sequence_conv_pool
      • simple_img_conv_pool
    • fluid.optimizer
      • Adadelta
      • AdadeltaOptimizer
      • Adagrad
      • AdagradOptimizer
      • Adam
      • Adamax
      • AdamaxOptimizer
      • AdamOptimizer
      • DecayedAdagrad
      • DecayedAdagradOptimizer
      • DGCMomentumOptimizer
      • Dpsgd
      • DpsgdOptimizer
      • ExponentialMovingAverage
      • Ftrl
      • FtrlOptimizer
      • LambOptimizer
      • LarsMomentum
      • LarsMomentumOptimizer
      • LookaheadOptimizer
      • ModelAverage
      • Momentum
      • MomentumOptimizer
      • PipelineOptimizer
      • RecomputeOptimizer
      • RMSPropOptimizer
      • SGD
      • SGDOptimizer
    • fluid.profiler
      • cuda_profiler
      • profiler
      • reset_profiler
      • start_profiler
      • stop_profiler
    • fluid.regularizer
      • L1Decay
      • L1DecayRegularizer
      • L2Decay
      • L2DecayRegularizer
    • fluid.transpiler
      • DistributeTranspiler
      • DistributeTranspilerConfig
      • HashName
      • memory_optimize
      • release_memory
      • RoundRobin
    • fluid.unique_name
      • generate
      • guard
      • switch
  • Release Notes
  • Practice Improving
  • »
  • Advanced User Guides »
  • Practice Improving
  • View page source

Practice Improving¶

  • Memory Allocation and Optimization
  • Best practices of distributed training on CPU
  • Recompute: Training with bigger batch size
  • Use Paddle-TensorRT Library for inference
  • Performance Profiling and Optimization