Basic API¶

• Optimize the API visibility of paddle.distributed、paddle.device、paddle.vision , for more information, please see 2.1.0 Release Note. (#33420)

• Add paddle.is_compiled_with_rocm. (#33228)

• Add the paddle.strided_slice to support bool type.（#33373）

• Add paddle.equal_all、paddle.equal、paddle.greater_equal、paddle.greater_than、paddle.less_equal、paddle.less_than、paddle.not_equal to support bool type. （#33551）

• Fix paddle.utils.download does not perform Retry when ConnectionError is abnormal.（#33454）

• Fix the issue of infershape error when paddle.gather axis is not equal to 0.（#33553）

• Fix segment fault caused by paddle.io.DataLoader when num_workers=0 and Dataset returns GPU Tensor and sends it to DataLoader .（#33487, #33249）

• Fix the issue that when use slice result as an lvalue of inplace operation, the error message of backward is not related to the error. （#32981）

• Fix the issue of paddle.concat support uint8 in dynamic graph.(#33667)

• Fix the issue of paddle.grid_sample GPU memory overflow and abnormal output. （#33100、#33232）

• Fix bug of roi_align, when the input width or height of rois is 0, the output feature should be 0 .（#33446）

• Fixed in some corner cases, input was modified to ‘nan’ bug of log_softmax op. （#32937）

Dynamic Graphs to Static Graphs¶

• Add support for dynamic conversion of user code for sublayer in the paddle.nn.Sequential .（#33065）

• Fix the issue of subscript syntax errors in the phase of static type analysis of variables in control flow for statement conversions. （#32969）

• Refactor the dynamic to static param_guard logic code to comprehensively solve the dynamic to static graph Tensor type conversion problem.（#32985）

Distributed Training¶

• Fix the error in paddle.distributed.spawn when using the default nprocs argument.（#33249）

• Fix the hang issue of training start caused by the inconsistent creation of pipeline parallel communication group.（#32890、#33473）

• Fix the issue of failed to save parameters in mixed parallelism.（#33595、#33588）

• Fix the issue that Fleet API cannot run Program directly.（#33511）

• Fix the hang issue caused by the uneven sample bucketing in the pure GPU training mode of heterogeneous parameter server.（#32957）

Mixed Precision Training¶

• Add SyncBatchNorm support for AMP in Dynamic graph, to improve the performance of dynamic graph SyncBatchNorm layer in AMP mode, and improve the 8-card AMP mode speedup ratio by 19% on DeepLabV3P model of [PaddleSeg].(#33709)

Custom OP¶

• Remove the dependency on PADDLE_WITH_MKLDNN macro for custom OP compilation.（#32903）

• Default setting GLIBCXX_USE_CXX11_ABI=1 to resolve the issue of low GCC version that may cause compile-time errors.（#33185）

• Add support for c++14 syntax feature, and enable -std=c++14 compile option by default. （#33227）

Others¶

• Fix the random segment error of training when LoDTensorArray is input of Op under multi-threading.（#32984）

• Fix an issue where parameter regularization is executed twice when both the regularizer of paddle.ParamAttr and the weight_decay of paddle.optimize are specified as L2Decay.（#32881）

• Fix the issue of corrupted characters of warning information in windows system.（#33689）

Function Upgrades¶

• Add converter/plugin of gather_nd 和 reduce_sum in Paddle-TRT.（#33365）

• Add reshape in Paddle-TRT.（#33372）

Performance Optimization¶

• Add the dynamic shape plugin of TensorRT layer_norm to improve model dynamic shape inference performance.（#33448）

易用性优化¶

• Add Paddle Inference ROCm version of Prediction Example Document, so as to add C++ prediction library version.txt with ROCm related version information. (#33290)

• Update XPU compilation options. Please refer to #33581 for specific compilation options.

Bug Fixes¶

• Fix the calculation error of fused_fc_elementwise_layernorm caused by too large number of threads under DCU. (#33299)

• Fix the issue that yolov3 model fails to run after gpu is turned on on nano and TX2.(#33442)

• Fix the computation error when seq_len > 1024 in Paddle-TRT multihead_matmul plugin .（#33365）

• Fix the incorrect output error caused by inconsistent order of input when ERNIE model becomes longer.（#33622）

• Fix the reports error of OCR model in prediction on GPU.(#33431)

• Fix the issue that paddle.static.io.normalize_program failed to export paddle.static.normalize_program.（#33408）

• Fix the issue that conv with stride > 1 fails in TRT6.0 and below.(#33198 )

• Fix the out-of-bounds error of GPU memory access when batch predicting images. (#33370 )(#33531 )

• Fix the issue of cache size setting failure on X86 CPU. （#33571）

• Fix TRT conv2d_transpose op converter dimension error setting. Now the model of conv2d_transpose op can work normally on TRT.（#33242）

• Fix the error of prediction library compiled by sub-CUDA Arch on Jetson devices. This version will release the Jetson prediction library compiled by sub-Arch for users who have demand for shrinked prediction library binary size.（#33269）

• Fix the issue that when using PaddleSlim quantitative model to load prediction from memory, it still reports an error because the calibration table path is not set.（#33629）

• Fix the issue that BERT/ERNIE gives wrong cuda error 400 when using TRT prediction on non-0 card.（#33706）

• Fix a cmake syntax error caused by setting custom compilation parameters under Linux.（#33621）

• Optimize the calculation accuracy of layer_norm and fix the problem of outputting Nan when input is large data. (#33420)

support of Kunlun chips¶

• Fix the gather op, add support of logsumexp op. (#32931)

Basic API¶

• Add data types such as paddle.dtype and paddle.float32 as data types within the Paddle. (#32012)

• Add paddle.nn.functional.glu. (#32096)

• Add paddle.nn.utils.spectral_norm. (#32633)

• Add paddle.Tensor.register_hook API for registering the hook function for the gradient Tensor corresponding to the forward Tensor in dynamic graph scenes. (#31775)

• Add the Tensor.__array__ function to support numpy.array(Tensor) and numpy.asarray(Tensor) to convert paddle.Tensor type to numpy.ndarray type . (#32300)

• Add the Tensor API: Tensor.item(*args). It can convert the element at the specified position in Tensor to Python scalar value and return it. (#32634)

• Add the paddle.nn.LayerList support for negative indexing. (#31750)

• Add 12 dynamic graph inplace APIs: clip_, scale_, add_, subtract_, ceil_, floor_, exp_, reciprocal_, round_, sqrt_, rsqrt_, and flatten_. These inplace APIs cannot be called by using paddle.api_ and should be called by using Tensor.api_. (#32699)

• Add paddle.autograd.backward API for customizing the starting gradient. (#31540)

• Add paddle.nn.LayerDict class. (#31951)

• Add layer.to API. (#32040)

• Add paddle.autograd.PyLayer API for supporting custom backward calculation of dynamic graphs on Python side. (#32130)

• Add the support for paddle.optimizer to specify non-parametric Tensor as parameters for optimization in dynamic graphs. (#32362)

• Add several sequence* functions in paddle.static.nn. Add paddle.nn.functional in sequence_mask. (#32089)

• Add paddle.nn.CTCLoss parameters in norm_by_times. (#32490)

• paddle.fill_constant supports uint8_t. (#31911)

• paddle.clip supports int32 and int64. (#32373)

• Support the input data type to be int in Nearest neighbor mode in paddle.nn.functional.interpolate. (#32270)

• All parameters in API that support passing in list or tuple are upgraded to support passing in list and tuple. (#32344, #32528 #32360)

• Optimize softmax operator performance. (#31821)

• Optimize paddle.norm documentation description to clarify the functional differences between paddle.norm and numpy.linalg.norm API. (#32530)

• Optimize the printing form of data type (datatype) of Tensor, for example, the dtype of Tensor of float32 type is changed from VarType.FP32 to paddle.float32. (#30682)

• OneDNN Functional optimization

  • Upgraded oneDNN to 2.2.1 (#31067 #31473 #30295 32227)

  • Added more precise mkldnn kernel rules in GetExpectedKernelType based on kernel’s data type. (#29840)

  • Fused layer_norm subgraphs to single layer_norm op. (#32162, #30891, #30962)

  • Reduced unnecessary memory allocation during creation of elementwise_mul operator (#30203)

  • Improved memory consumption used in cache per thread (#30358)

  • Added oneDNN FP32 and INT8 support for vanilla LSTM (#30719 #31894)

  • Added OneDNN hardswish support (#30211)

  • Added bilinear_interp_v2 and nearest_interp_v2 oneDNN FP32 kernels (#32312)

• Updated Xbyak to v5.81 (#30809)

• Fix paddle.io.DataLoader to support data sets containing nested complex data formats such as list, dict and string, and fix the occasional error report and unreleased resources when the program exits during the iteration. (#31481)

• Fix the problem caused by modifying the root logger of logging library in paddle. (#32706)

• Fix the problem of L1Decay error report in backward dynamic graph mode. (#32718)

• Fix the problem that nan comes out in setting ignore_index and reduction='mean' in paddle.nn.functional.cross_entropy. (#32545)

• Fix the problem that the output type is bool during the summing of bool tensor and float tensor. (#32272)

• Fix the calculation error of comparison class API in broadcast. (#32470)

• Fix the gradient calculation error under broadcast where right input is large shape in addition, subtraction, multiplication and division. (#30818)

• Fix the problem of the calculation result of segment mean OP being incorrect when processing the large shape tensor input. (#32610)

• Fix the problem of the data type of optimizer variables not matching with the data type of model parameters. (#29917)

• Fix the error report in num worker>0 when the paddle.io.DataLoader pre-processing includes the paddle operation. (#31177)

• Fix the error report when printing empty tensor. (#32501)

• Adjust the initialization order of static graph parameters, and keep consistency with dynamic graphs after adjustment, so that the same model is set with the same random seed to get the same parameters initialized in dynamic graphs and static graphs. (#32177)

• Fix the bug that paddle.to_tensor does not support accepting dtype=Tensor.dtype. (#31931)

• Fix the bug that the gradient is nan when 2 inputs are equal in paddle.dist. (#32448)

• paddle.nn.functional.temporal_shift added data_format property to support to set to NCHW or NHWC. (#31642)

• Fix the problem of the calculation result being incorrect in adaptive_avg_pool2d when the input data type is float16. (#31887)

• paddle.nn.Layer.sublayers and paddle.nn.Layer.named_sublayers: Modify the include_sublayers = True parameter of original paddle.nn.Layer.sublayers to include_self = False, thus fixing the problem of returning null of the former include_sublayers = False. Now the default behavior is the same as that when no parameter is filled in, that is, return all recursive sublevels that don’t contain themselves. When include_self = True is the same as the literal meaning, return all recursive sublevels that contain themselves. The include_sublayers parameter in paddle.nn.Layer.named_sublayers is directly removed. Other behaviors remain unchanged. (#31824 )

>>> from paddle.vision.models import resnet18
>>> model = resnet18()
>>> 
>>> print(len(model.sublayers(include_sublayers=True)))
>>> print(len(model.sublayers(include_sublayers=False)))
67
0

>>> from paddle.vision.models import resnet18
>>> model = resnet18()
>>> 
>>> print(len(model.sublayers(include_self=True)))
>>> print(len(model.sublayers(include_self=False)))
68
67

High-level API¶

• Add the paddle.hub function. Provide help, list and load functions for viewing and loading third-party models, and support the loading of remote and local repository. (#31873)

• Support the mixed precision training. Provide O0, O1, O2 three modes, which correspond to FP32 training, automatic mixed precision training, pure FP16 training respectively. At present, pure FP16 training only supports static graphs. (#31417)

• Support the image transformation of the paddle.Tensor type, including operators such as normalize, to_grayscale, vflip, hflip, crop, center_crop, pad, rotate, resize. (#32705)

Dynamic Graphs to Static Graphs¶

Fix the bug of dynamic graphs converted to static graphs.

• The shape returned by the static graph arange、range API is not consistent with the dynamic graph.

• paddle.to_tensor supports the input as int，float，bool basic type in dynamic to static.

• Support the parsing of the dict derivative syntax in the for loop. (#32159)

• Fix the problem of undeclared variables errors in the nested control flow statements in some scenarios. (#32153)

• Fix the bug that the float16 type is missed in expand op. (#32238)

• Fix the bug of returning the gradient information as None when the shape dimension is 6 in the expand_v2、tile、expand、expand_as、expand_as_v2、meshgrid 6 OP backward gradient solution. (#32004)

• Fix the problem that the paddle.jit.TraceLayer.save_inference_model interface is inconsistent with paddle.static.load_inference_model because the network structure and parameters are not saved at the same time. (#31989)

Mixed Precision Training¶

• The op that does not support fp16 kernel is automatically kept as fp32 calculation in the dynamic graph mixed precision interface auto_cast. (#32543)

• Fix the unexpected error in the static graph mixed precision training caused by the incomplete statistics of the Op list (unsupported_fp16_list) which does not support FP16 calculation. The list of Op that currently does not support FP16 calculation can be generated automatically according to the runtime environment. (#32102)

• In the for loop in the update_loss_scaling, optimize the problem that multiple identical cuda kernel are fused into one cuda kernel. (#32554)

• Optimize the slow performance in slice multi-dimensional cases. (#32266)

• Optimize the redundant copy problem when elementwise_add_grad inputs and outputs are the same. (#32051)

• In the for loop in the check_finite_and_unscale, optimize the problem that multiple identical cuda kernel are fused into one cuda kernel. (#31954)

• Optimize the range parameter redundant copy problem. (#30811)

• Optimize the slow performance problem of top_k_v2 in input_width <= 1024. (#30403)

• Migrate where_index CPU calculation process to GPU for completion. (#30601)

BF16 Training¶

• Added initial bf16 amp integration that modify models by adding cast ops to BF16 enabled ops in the forward pass. #31093

• Added BF16 pure_mode, which means adding support for BF16 training based on BF16-enabled ops list and enable BF16 parameters, BF16 operators, BF16 decorator for optimizer during training. #32281 #32681

• Added CPU core flags verification for BF16 fast performance support. #30551

• Unification of BF16 enablement process #31034

• Added BF16 Constant Initializer and for other initializers, add cast op to convert other initializer output to be BF16 datatype. #31935

• Added BF16 uniform random initializer #32468

• Added mechanism that converts startup_program initializers to BF16 #32720

• Added BF16 support for sgd operator CPU kernel. #32162

• Added BF16 support for lookup_table operator. #31558

• Added Sum kernel for CPU supporting BF16 and SelectedRows #32755 #32631

• Added Conv Transpose BF16 support #30877

• Added elementwise_add bf16 grad #30925

• Added reshape op BWD grad bf16 #31035

• Added broadcasting support in elementwise_add grad bf16/fp32 #31385

• Added Elementwise Mul grad fp32/bf16 #31647

• Added LSTM BF16 and fixed GRU BF16 #31234

• Added oneDNN reduce_op fp32 and bf16 kernels #31816

• Added oneDNN reduce_op GRAD fp32 and bf16 kernels #32280 #32592

Distributed Training Optimization¶

• New graph-based retrieval engine for training distributed graph neural network over trillion edges(#31226).

• Added index-based data sampling class to support sampling from graph and TDM/OTM tree(#31696).

• Added paddle.distributed.send, paddle.distributed.recv, paddle.distributed.new_group, paddle.distributed.wait to improve the distributed communication API. (#32504, #31682)

• Support to initialize the sync_parameters_bufferin the distributed dynamic graph, which solved the issue that the buffer of the dynamic graph is not globally initialized. (#31625)

• Pipeline Parallelism supports 1F1B scheduling method to optimize the memory usage of GPU. Theoretically, it is constant(#31786).

• [Hybrid Parallel] Sharding strategy optimization: support Gradients aggregation, reducing the amount of parameter communication, and improving the speed of training; Could be used flexibly with other parallelism strategies. (#31884 #32486 #32485 #31996 #31939 #31796)

• [Hybrid Parallel] Added optimizer state offload in the Sharding strategy, to reduce the memory usage of GPU. (#32134)

• [Hybrid Parallel] Support the persistence of the broadcast ID’s socket service, reduced the conflicts of ports in the hybrid parallelism. (#31589)

• [Parameter Server] Optimize the output and printing of LOG, and remove invalid logs.

• [Parameter Server] Optimize the sparse parameter storage structure, with large memory reduction for small dimensions (below 64).

• [Parameter Server] Fix the bug of access policy taking effect in the distributed prediction.

• HeterPs supports multiple machines. (#31102)

Custom OP¶

• Add support for using custom OP function on Mac platform. (#31976)

• Support automatic search function of C++/v11 header file directory on Mac platform, compatible with the situation that multiple versions of clang may exist locally.

• Add support for Op forward/backward function Attribute parameter, inferShape, and InferDtype function input parameter using the const & type. (#31588)

• Add support for using three framework internal data types paddle::complex64, paddle::complex128, paddle::float16 in the implementation of custom Op. (#31602, #31657, #31669, #31725)

• Add support for using std::vector<paddle::Tensor> type parameters as input of forward/backward functions in custom Op. (#31535)

• Add support for the InferShape function using Attribute parameter as input. (#31713)

• Optimize the call stack of auto-generated Python API under dynamic graph to improve the execution efficiency. (#32209)

• Reduce the error reporting condition when checking the compiler cl.exe on Windows, and enhance the self-test robustness in Windows environment. (#32769)

• Fix a bug in compiler selection when installing multiple CUDA environments on Windows. (#31694)

• Fix a bug in Python encoding issue when installing Chinese version of VS on Windows. (#31493)

• Remove the dependency on separate dynamic library files and link only the framework core dynamic library files. (#32404、#32769)

• Remove the previous old custom OP scheme and clean up the redundant library files and header files in the whl package, reducing the whl package size by about 11M. (#31813), (#32463)

Model saving and loading¶

• paddle.save, paddle.load supports saving and loading of Tensor. (#31756)

• paddle.save, paddle.load supports saving and loading of list[Tensor]、dict[Tensor]、tuple[Tensor] and list、tuple、dict nested structures containing Tensor. (#32446)

• paddle.save, paddle.load supports saving and loading of Layer. (#32446)

• paddle.save, paddle.load supports saving and loading of Program. (#32336)

• paddle.save, paddle.load supports saving and loading of single Tensor in C++ binary format. (#32211)

• paddle.jit.save, paddle.jit.load supports saving and loading of Fucntion without parameters. (#32430)

Function Upgrade¶

• Release C API (experimental). The function of new C API is basically equal to that of C + +. (#32225)

• The prediction framework python interface access to the train custom operators. After loading a custom operator during training, users can execute the deployment of prediction models containing this custom operator directly through PaddlePredictor, just like the framework’s native operator. (#32533)

• The underlying implementation of Tensor has been refactored internally to decouple from the old ZeroCopyTensor data structure. This upgrade does not involve user API changes and is transparent to users. (#31402)

• Support TensorRT serialization and deserialization when loading models from memory. (#31342)

Performance Optimization¶

• Support quantilized ERNIE models to be inferred with the mixed precision using TensorRT, where Matmul is computed with Int8 precision and other parts are computed with FP16 precision. Compared with the pure FP16 inference, the inference performance of the standard ERNIE model on XNLI dataset is improved from 1898 seq/s to 2310 seq/s at batch size=40 on T4, improving by 17.8%. (#32232)

Ease-of-use optimization¶

• Add error messages when the user enables the TensorRT variable-length input settings, and the wrong input shape is provided. (#32155)

• Add runtime TensorRT version check. If the major version number of TensorRT at runtime and compile time differs, the warning is generated. (#32443)

• Add the TensorRT VERBOSE level log switch. Users can enable the TensorRT VERBOSE log by export GLOG_v=3 to print more debugging information. (#32459)

BugFix¶

• Fix the error of insufficient graphics card or video memory of unspecified usage at the end of prediction. (#32655)

• Fix the CPU performance issue caused by informal values of native inference in dynamic graphs. (#32350)

• Fix the problem of requiring the setting of the calibration table path in the data read from memory when TensorRT inference is enabled by using the PaddleSlim quantization model. (#32676)

• Upgrade the TensorRT quantization calibration table interface, fix the problem that TensorRT offline quantization is not supported on DLA. (#31060)

• Fix the problem of the number of header of crop Attention not being supported when using variable length method for ERNIE/BERT model inference (EnableTensorRtOSS). (#31497)

• Fix the occasional diff problem caused by the instable QK input sequence of the BERT model trained after version 2.0 (#32659)

• Fix the problem that ERNIE model reports an error or incorrect result due to the wrong order of input variable names when TensorRT varlen acceleration is enabled. (#32482)

• Fix the bug that plugin ElementwisePluginDynamic serialization of TensorRT fails. (#31587)

• Fix the problem of subsequent OP dimension error caused by FC layer dimension complement 1 under TensorRT dynamic shape. (#32458, #31803)

• Fix the problem of repeated_fc_relu_fuse_pass.cc error when FC uses Padding. (#32648)

• Fix the problem of the result of conv2d_transpose op being wrong when using TensorRT inference. (#32593)

• Fix the problem with OCR INT8 model oneDNN prediction errors caused by incorrect comparison of NAN. (#32227)

• Fix the problem of data contention when deploying multiple models for oneDNN prediction on multiple executors with multiple threads. (#32499, #32136 #32664)