Install and Compile C++ Inference Library

Direct Download and Installation

Table 4 c++ inference library list

version description

inference library(1.6.1 version)

inference library(develop version)

cpu_avx_mkl

fluid_inference.tgz

fluid_inference.tgz

cpu_avx_openblas

fluid_inference.tgz

fluid_inference.tgz

cpu_noavx_openblas

fluid_inference.tgz

fluid_inference.tgz

cuda9.0_cudnn7_avx_mkl

fluid_inference.tgz

fluid_inference.tgz

cuda10.0_cudnn7_avx_mkl

fluid_inference.tgz

fluid_inference.tgz

Build from Source Code

Users can also compile C++ inference libraries from the PaddlePaddle core code by specifying the following compile options at compile time:

Option

Value

CMAKE_BUILD_TYPE

Release

FLUID_INFERENCE_INSTALL_DIR

Path of installation

WITH_PYTHON

OFF(recommended)

ON_INFER

ON(recommended)

WITH_GPU

ON/OFF

WITH_MKL

ON/OFF

It is recommended to configure options according to the recommended values to avoid the link to unnecessary library. Other options can be set if it is necessary.

The following code snippet pulls the latest code from github and specifies the compiling options (you need to replace PADDLE_ROOT with the installation path of the PaddlePaddle inference library):

PADDLE_ROOT=/path/of/capi
git clone https://github.com/PaddlePaddle/Paddle.git
cd Paddle
mkdir build
cd build
cmake -DFLUID_INFERENCE_INSTALL_DIR=$PADDLE_ROOT \
      -DCMAKE_BUILD_TYPE=Release \
      -DWITH_PYTHON=OFF \
      -DWITH_MKL=OFF \
      -DWITH_GPU=OFF  \
      -DON_INFER=ON \
      ..
 make
 make inference_lib_dist

After successful compilation, dependencies required by the C++ inference library Will be stored in the PADDLE_ROOT directory. (dependencies including: (1) compiled PaddlePaddle inference library and header files; (2) third-party link libraries and header files; (3) version information and compilation option information)

The directory structure is:

PaddleRoot/
├── CMakeCache.txt
├── paddle
│   ├── include
│   │   ├── paddle_anakin_config.h
│   │   ├── paddle_analysis_config.h
│   │   ├── paddle_api.h
│   │   ├── paddle_inference_api.h
│   │   ├── paddle_mkldnn_quantizer_config.h
│   │   └── paddle_pass_builder.h
│   └── lib
│       ├── libpaddle_fluid.a
│       └── libpaddle_fluid.so
├── third_party
│   ├── boost
│   │   └── boost
│   ├── eigen3
│   │   ├── Eigen
│   │   └── unsupported
│   └── install
│       ├── gflags
│       ├── glog
│       ├── mkldnn
│       ├── mklml
│       ├── protobuf
│       ├── snappy
│       ├── snappystream
│       ├── xxhash
│       └── zlib
└── version.txt

The version information of the inference library is recorded in version.txt, including Git Commit ID, version of OpenBlas, MKL math library, or CUDA/CUDNN. For example:

GIT COMMIT ID: cc9028b90ef50a825a722c55e5fda4b7cd26b0d6
WITH_MKL: ON
WITH_MKLDNN: ON
WITH_GPU: ON
CUDA version: 8.0
CUDNN version: v7