Overview
TensorRT Python API Reference
Core Concepts¶
TensorRT Workflow¶
The general TensorRT workflow consists of 3 steps:
Populate a
tensorrt.INetworkDefinitioneither with a parser or by using the TensorRT Network API (seetensorrt.INetworkDefinitionfor more details). Thetensorrt.Buildercan be used to generate an emptytensorrt.INetworkDefinition.Use the
tensorrt.Builderto build atensorrt.ICudaEngineusing the populatedtensorrt.INetworkDefinition.Create a
tensorrt.IExecutionContextfrom thetensorrt.ICudaEngineand use it to perform optimized inference.
Classes Overview¶
Logger¶
Most other TensorRT classes use a logger to report errors, warnings and informative messages. TensorRT provides a basictensorrt.Logger implementation, but you can write your own implementation by deriving fromtensorrt.ILogger for more advanced functionality.
Parsers¶
Parsers are used to populate atensorrt.INetworkDefinition from a model trained in a Deep Learning framework.
Network¶
Thetensorrt.INetworkDefinition represents a computational graph. In order to populate the network, TensorRT provides a suite of parsers for a variety of Deep Learning frameworks. It is also possible to populate the network manually using the Network API.
Builder¶
Thetensorrt.Builder is used to build atensorrt.ICudaEngine . In order to do so, it must be provided a populatedtensorrt.INetworkDefinition .
Engine and Context¶
Thetensorrt.ICudaEngine is the output of the TensorRT optimizer. It is used to generate atensorrt.IExecutionContext that can perform inference.