Performance Profiling for TensorRT Library¶
Test Environment¶
- CPU:Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz GPU:Tesla P4
- TensorRT4.0, CUDA8.0, CUDNNV7
- Test model ResNet50, MobileNet, ResNet101, Inception V3.
Test Targets¶
PaddlePaddle, Pytorch, Tensorflow
- In test, PaddlePaddle adopts subgraph optimization to integrate TensorRT model .
- Native implementation is used in Pytorch. Model address 1 , address 2 .
- Test for TensorFlow contains test for native TF and TF—TRT. Test for TF—TRT hasn't reached expectation wihch will be complemented later. Model address .
ResNet50¶
| batch_size | PaddlePaddle(ms) | Pytorch(ms) | TensorFlow(ms) |
|---|---|---|---|
| 1 | 4.64117 | 16.3 | 10.878 |
| 5 | 6.90622 | 22.9 | 20.62 |
| 10 | 7.9758 | 40.6 | 34.36 |
MobileNet¶
|batch_size|PaddlePaddle(ms)|Pytorch(ms)|TensorFlow(ms)| |---|---|---|---| |1| 1.7541 | 7.8 |2.72| |5| 3.04666 | 7.8 |3.19| |10|4.19478 | 14.47 |4.25|
ResNet101¶
|batch_size|PaddlePaddle(ms)|Pytorch(ms)|TensorFlow(ms)| |---|---|---|---| |1|8.95767| 22.48 |18.78| |5|12.9811 | 33.88 |34.84| |10|14.1463| 61.97 |57.94|
Inception v3¶
|batch_size|PaddlePaddle(ms)|Pytorch(ms)|TensorFlow(ms)| |---|---|---|---| |1|15.1613 | 24.2 |19.1| |5|18.5373 | 34.8 |27.2| |10|19.2781| 54.8 |36.7|