INetworkDefinition¶

classtensorrt.INetworkDefinition¶

Represents a TensorRT Network from which the Builder can build an Engine

Variables:

• num_layers –int The number of layers in the network.

• num_inputs –int The number of inputs of the network.

• num_outputs –int The number of outputs of the network.

• num_ranks –int The number of ranks to use for multi-device execution.

• name –str The name of the network. This is used so that it can be associated with a built engine. The name must be at most 128 characters in length. TensorRT makes no use of this string except storing it as part of the engine so that it may be retrieved at runtime. A name unique to the builder will be generated by default.

• has_implicit_batch_dimension –bool [DEPRECATED] Deprecated in TensorRT 10.0. Always flase since the implicit batch dimensions support has been removed.

• error_recorder –IErrorRecorder Application-implemented error reporting interface for TensorRT objects.

Flags:

int:

A bitset of theNetworkDefinitionCreationFlag s set for this network.

__del__(self:tensorrt.tensorrt.INetworkDefinition)→None¶

__exit__(exc_type,exc_value,traceback)¶

Context managers are deprecated and have no effect. Objects are automatically freed whenthe reference count reaches 0.

__getitem__(self:tensorrt.tensorrt.INetworkDefinition,arg0:int)→tensorrt.tensorrt.ILayer¶

__init__(*args,**kwargs)¶

__len__(self:tensorrt.tensorrt.INetworkDefinition)→int¶

add_activation(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,type:tensorrt.tensorrt.ActivationType)→tensorrt.tensorrt.IActivationLayer¶

Add an activation layer to the network.SeeIActivationLayer for more information.

Parameters:

• input – The input tensor to the layer.

• type – The type of activation function to apply.

Returns:

The new activation layer, orNone if it could not be created.

add_assertion(self:tensorrt.tensorrt.INetworkDefinition,condition:tensorrt.tensorrt.ITensor,message:str)→tensorrt.tensorrt.IAssertionLayer¶

Add a assertion layer.SeeIAssertionLayer for more information.

Parameters:

• condition – The condition tensor to the layer.

• message – The message to print if the assertion fails.

Returns:

The new assertion layer, orNone if it could not be created.

add_attention(self:tensorrt.tensorrt.INetworkDefinition,query:tensorrt.tensorrt.ITensor,key:tensorrt.tensorrt.ITensor,value:tensorrt.tensorrt.ITensor,norm_op:tensorrt.tensorrt.AttentionNormalizationOp,causal:bool)→tensorrt.tensorrt.IAttention¶

Add an attention to the network.SeeIAttention for more information.

Parameters:

• query – The 4d query input tensor to the attention.

• key – The 4d key input tensor to the attention.

• value – The 4d value input tensor to the attention.

• normOp – The normalization operation to perform.

• causal – The boolean that specifies whether an attention will run casual inference.

Returns:

The new Attention, orNone if it could not be created.

add_cast(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,to_type:tensorrt.tensorrt.DataType)→tensorrt.tensorrt.ICastLayer¶

Add a cast layer.SeeICastLayer for more information.

Parameters:

• input – The input tensor to the layer.

• to_type – The data type the output tensor should be cast into.

Returns:

The new cast layer, orNone if it could not be created.

add_concatenation(self:tensorrt.tensorrt.INetworkDefinition,inputs:List[tensorrt.tensorrt.ITensor])→tensorrt.tensorrt.IConcatenationLayer¶

Add a concatenation layer to the network. Note that all tensors must have the same dimension except for the Channel dimension.SeeIConcatenationLayer for more information.

Parameters:

inputs – The input tensors to the layer.

Returns:

The new concatenation layer, orNone if it could not be created.

add_constant(self:tensorrt.tensorrt.INetworkDefinition,shape:tensorrt.tensorrt.Dims,weights:tensorrt.tensorrt.Weights)→tensorrt.tensorrt.IConstantLayer¶

Add a constant layer to the network.SeeIConstantLayer for more information.

Parameters:

• shape – The shape of the constant.

• weights – The constant value, represented as weights.

Returns:

The new constant layer, orNone if it could not be created.

add_convolution_nd(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,num_output_maps:int,kernel_shape:tensorrt.tensorrt.Dims,kernel:tensorrt.tensorrt.Weights,bias:tensorrt.tensorrt.Weights=None)→tensorrt.tensorrt.IConvolutionLayer¶

Add a multi-dimension convolution layer to the network.SeeIConvolutionLayer for more information.

Parameters:

• input – The input tensor to the convolution.

• num_output_maps – The number of output feature maps for the convolution.

• kernel_shape – The dimensions of the convolution kernel.

• kernel – The kernel weights for the convolution.

• bias – The optional bias weights for the convolution.

Returns:

The new convolution layer, orNone if it could not be created.

add_cumulative(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,axis:tensorrt.tensorrt.ITensor,op:tensorrt.tensorrt.CumulativeOperation,exclusive:bool,reverse:bool)→tensorrt.tensorrt.ICumulativeLayer¶

Add a cumulative layer to the network.SeeICumulativeLayer for more information.

Parameters:

• input – The input tensor to the layer.

• axis – The axis tensor to apply the cumulative operation on. Currently, it must be a build-time constant 0-D shape tensor.

• op – The reduction operation to perform.

• exclusive – The boolean that specifies whether it is an exclusive cumulative or inclusive cumulative.

• reverse – The boolean that specifies whether the cumulative should be applied backward.

Returns:

The new cumulative layer, orNone if it could not be created.

add_deconvolution_nd(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,num_output_maps:int,kernel_shape:tensorrt.tensorrt.Dims,kernel:tensorrt.tensorrt.Weights,bias:tensorrt.tensorrt.Weights=None)→tensorrt.tensorrt.IDeconvolutionLayer¶

Add a multi-dimension deconvolution layer to the network.SeeIDeconvolutionLayer for more information.

Parameters:

• input – The input tensor to the layer.

• num_output_maps – The number of output feature maps.

• kernel_shape – The dimensions of the convolution kernel.

• kernel – The kernel weights for the convolution.

• bias – The optional bias weights for the convolution.

Returns:

The new deconvolution layer, orNone if it could not be created.

add_dequantize(*args,**kwargs)¶

Overloaded function.

1. add_dequantize(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, scale: tensorrt.tensorrt.ITensor) -> tensorrt.tensorrt.IDequantizeLayer

   Add a dequantization layer to the network.SeeIDequantizeLayer for more information.

   arg input:

   A tensor to quantize.

   arg scale:

   A tensor with the scale coefficients.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.float32).

   returns:

   The new dequantization layer, orNone if it could not be created.

2. add_dequantize(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, scale: tensorrt.tensorrt.ITensor, output_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.IDequantizeLayer

   Add a dequantization layer to the network.SeeIDequantizeLayer for more information.

   arg input:

   A tensor to quantize.

   arg scale:

   A tensor with the scale coefficients.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.float32).

   returns:

   The new dequantization layer, orNone if it could not be created.

add_dynamic_quantize(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,axis:int,block_size:int,output_type:tensorrt.tensorrt.DataType,scale_type:tensorrt.tensorrt.DataType)→tensorrt.tensorrt.IDynamicQuantizeLayer¶

Add a dynamic quantization layer to the network.SeeIDynamicQuantizeLayer for more information.

Parameters:

• input – A tensor to quantize.

• axis – The axis that is sliced into blocks.

• block_size – The number of elements that are quantized using a shared scale factor.

• output_type – The data type of the quantized output tensor.

• scale_type – The data type of the scale factor used for quantizing the input data.

Returns:

The new DynamicQuantization layer, orNone if it could not be created.

add_einsum(self:tensorrt.tensorrt.INetworkDefinition,inputs:List[tensorrt.tensorrt.ITensor],equation:str)→tensorrt.tensorrt.IEinsumLayer¶

Adds an Einsum layer to the network.SeeIEinsumLayer for more information.

Parameters:

• inputs – The input tensors to the layer.

• equation – The Einsum equation of the layer.

Returns:

the new Einsum layer, orNone if it could not be created.

add_elementwise(self:tensorrt.tensorrt.INetworkDefinition,input1:tensorrt.tensorrt.ITensor,input2:tensorrt.tensorrt.ITensor,op:tensorrt.tensorrt.ElementWiseOperation)→tensorrt.tensorrt.IElementWiseLayer¶

Add an elementwise layer to the network.SeeIElementWiseLayer for more information.

Parameters:

• input1 – The first input tensor to the layer.

• input2 – The second input tensor to the layer.

• op – The binary operation that the layer applies.

The input tensors must have the same number of dimensions.For each dimension, their lengths must match, or one of them must be one.In the latter case, the tensor is broadcast along that axis.

The output tensor has the same number of dimensions as the inputs.For each dimension, its length is the maximum of the lengths of thecorresponding input dimension.

Returns:

The new element-wise layer, orNone if it could not be created.

add_fill(*args,**kwargs)¶

Overloaded function.

1. add_fill(self: tensorrt.tensorrt.INetworkDefinition, shape: tensorrt.tensorrt.Dims, op: tensorrt.tensorrt.FillOperation, output_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.IFillLayer

   Add a fill layer.SeeIFillLayer for more information.

   arg dimensions:

   The output tensor dimensions.

   arg op:

   The fill operation that the layer applies.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.float32).

   returns:

   The new fill layer, orNone if it could not be created.

2. add_fill(self: tensorrt.tensorrt.INetworkDefinition, shape: tensorrt.tensorrt.Dims, op: tensorrt.tensorrt.FillOperation) -> tensorrt.tensorrt.IFillLayer

   Add a fill layer.SeeIFillLayer for more information.

   arg dimensions:

   The output tensor dimensions.

   arg op:

   The fill operation that the layer applies.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.float32).

   returns:

   The new fill layer, orNone if it could not be created.

add_gather(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,indices:tensorrt.tensorrt.ITensor,axis:int)→tensorrt.tensorrt.IGatherLayer¶

Add a gather layer to the network.SeeIGatherLayer for more information.

Parameters:

• input – The tensor to gather values from.

• indices – The tensor to get indices from to populate the output tensor.

• axis – The non-batch dimension axis in the data tensor to gather on.

Returns:

The new gather layer, orNone if it could not be created.

add_gather_v2(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,indices:tensorrt.tensorrt.ITensor,mode:tensorrt.tensorrt.GatherMode)→tensorrt.tensorrt.IGatherLayer¶

Add a gather layer to the network.SeeIGatherLayer for more information.

Parameters:

• input – The tensor to gather values from.

• indices – The tensor to get indices from to populate the output tensor.

• mode – The gather mode.

Returns:

The new gather layer, orNone if it could not be created.

add_grid_sample(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,grid:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IGridSampleLayer¶

Creates a GridSample layer with a trt.InterpolationMode.LINEAR, unaligned corners, and trt.SampleMode.FILL for 4d-shape input tensors.SeeIGridSampleLayer for more information.

Parameters:

• input – The input tensor to the layer.

• grid – The grid tensor to the layer.

Variables:

• interpolation_mode – class:InterpolationMode The interpolation mode to use in the layer. Default is LINEAR.

• align_corners – class:bool the align mode to use in the layer. Default is False.

• padding_mode –SampleMode The padding mode to use in the layer. Default is FILL.

Returns:

The new grid sample layer, orNone if it could not be created.

add_identity(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IIdentityLayer¶

Add an identity layer.SeeIIdentityLayer for more information.

Parameters:

input – The input tensor to the layer.

Returns:

The new identity layer, orNone if it could not be created.

add_if_conditional(self:tensorrt.tensorrt.INetworkDefinition)→tensorrt.tensorrt.IIfConditional¶

Adds an if-conditional to the network, which provides a way to specify subgraphs that will be conditionally executed using lazy evaluation.SeeIIfConditional for more information.

Returns:

The new if-condtional, orNone if it could not be created.

add_input(self:tensorrt.tensorrt.INetworkDefinition,name:str,dtype:tensorrt.tensorrt.DataType,shape:tensorrt.tensorrt.Dims)→tensorrt.tensorrt.ITensor¶

Adds an input to the network.

Parameters:

• name – The name of the tensor. Each input and output tensor must have a unique name.

• dtype – The data type of the tensor.

• shape – The dimensions of the tensor.

Returns:

The newly added Tensor.

add_loop(self:tensorrt.tensorrt.INetworkDefinition)→tensorrt.tensorrt.ILoop¶

Adds a loop to the network, which provides a way to specify a recurrent subgraph.SeeILoop for more information.

Returns:

The new loop layer, orNone if it could not be created.

add_lrn(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,window:int,alpha:float,beta:float,k:float)→tensorrt.tensorrt.ILRNLayer¶

Add a LRN layer to the network.SeeILRNLayer for more information.

Parameters:

• input – The input tensor to the layer.

• window – The size of the window.

• alpha – The alpha value for the LRN computation.

• beta – The beta value for the LRN computation.

• k – The k value for the LRN computation.

Returns:

The new LRN layer, orNone if it could not be created.

add_matrix_multiply(self:tensorrt.tensorrt.INetworkDefinition,input0:tensorrt.tensorrt.ITensor,op0:tensorrt.tensorrt.MatrixOperation,input1:tensorrt.tensorrt.ITensor,op1:tensorrt.tensorrt.MatrixOperation)→tensorrt.tensorrt.IMatrixMultiplyLayer¶

Add a matrix multiply layer to the network.SeeIMatrixMultiplyLayer for more information.

Parameters:

• input0 – The first input tensor (commonly A).

• op0 – Whether to treat input0 as matrices, transposed matrices, or vectors.

• input1 – The second input tensor (commonly B).

• op1 – Whether to treat input1 as matrices, transposed matrices, or vectors.

Returns:

The new matrix multiply layer, orNone if it could not be created.

add_nms(*args,**kwargs)¶

Overloaded function.

1. add_nms(self: tensorrt.tensorrt.INetworkDefinition, boxes: tensorrt.tensorrt.ITensor, scores: tensorrt.tensorrt.ITensor, max_output_boxes_per_class: tensorrt.tensorrt.ITensor) -> tensorrt.tensorrt.INMSLayer

   Add a non-maximum suppression layer to the network.SeeINMSLayer for more information.

   arg boxes:

   The input boxes tensor to the layer.

   arg scores:

   The input scores tensor to the layer.

   arg max_output_boxes_per_class:

   The maxOutputBoxesPerClass tensor to the layer.

   ivar bounding_box_format:

   BoundingBoxFormat The bounding box format used by the layer. Default is CORNER_PAIRS.

   ivar topk_box_limit:

   int The maximum number of filtered boxes considered for selection per batch item. Default is 2000 for SM 5.3 and 6.2 devices, and 5000 otherwise. The TopK box limit must be less than or equal to {2000 for SM 5.3 and 6.2 devices, 5000 otherwise}.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   The new NMS layer, orNone if it could not be created.

2. add_nms(self: tensorrt.tensorrt.INetworkDefinition, boxes: tensorrt.tensorrt.ITensor, scores: tensorrt.tensorrt.ITensor, max_output_boxes_per_class: tensorrt.tensorrt.ITensor, indices_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.INMSLayer

   Add a non-maximum suppression layer to the network.SeeINMSLayer for more information.

   arg boxes:

   The input boxes tensor to the layer.

   arg scores:

   The input scores tensor to the layer.

   arg max_output_boxes_per_class:

   The maxOutputBoxesPerClass tensor to the layer.

   ivar bounding_box_format:

   BoundingBoxFormat The bounding box format used by the layer. Default is CORNER_PAIRS.

   ivar topk_box_limit:

   int The maximum number of filtered boxes considered for selection per batch item. Default is 2000 for SM 5.3 and 6.2 devices, and 5000 otherwise. The TopK box limit must be less than or equal to {2000 for SM 5.3 and 6.2 devices, 5000 otherwise}.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   The new NMS layer, orNone if it could not be created.

add_non_zero(*args,**kwargs)¶

Overloaded function.

1. add_non_zero(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor) -> tensorrt.tensorrt.INonZeroLayer

   Adds an NonZero layer to the network.SeeINonZeroLayer for more information.

   arg input:

   The input tensor to the layer.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   the new NonZero layer, orNone if it could not be created.

2. add_non_zero(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, indices_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.INonZeroLayer

   Adds an NonZero layer to the network.SeeINonZeroLayer for more information.

   arg input:

   The input tensor to the layer.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   the new NonZero layer, orNone if it could not be created.

add_normalization(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,scale:tensorrt.tensorrt.ITensor,bias:tensorrt.tensorrt.ITensor,axesMask:int)→tensorrt.tensorrt.INormalizationLayer¶

Adds a Normalization layer to the network.SeeNormalization for more information.

Parameters:

• input – The input tensor to the layer.

• scale – The scale tensor used to scale the normalized output.

• bias – The bias tensor used to scale the normalized output.

• axesMask – The axes on which to perform mean calculations.The bit in position i of bitmask axes corresponds to explicit dimension i of the result.E.g., the least significant bit corresponds to the first explicit dimension and the next to leastsignificant bit corresponds to the second explicit dimension.

Returns:

the new Normalization layer, orNone if it could not be created.

add_one_hot(self:tensorrt.tensorrt.INetworkDefinition,indices:tensorrt.tensorrt.ITensor,values:tensorrt.tensorrt.ITensor,depth:tensorrt.tensorrt.ITensor,axis:int)→tensorrt.tensorrt.IOneHotLayer¶

Add a OneHot layer to the network.SeeIOneHotLayer for more information.

Parameters:

• indices – The tensor to get indices from to populate the output tensor.

• values – The tensor to get off (cold) value and on (hot) value

• depth – The tensor to get depth (number of classes) of one-hot encoding

• axis – The axis to append the one-hot encoding to

Returns:

The new OneHot layer, orNone if it could not be created.

add_padding_nd(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,pre_padding:tensorrt.tensorrt.Dims,post_padding:tensorrt.tensorrt.Dims)→tensorrt.tensorrt.IPaddingLayer¶

Add a multi-dimensional padding layer to the network.SeeIPaddingLayer for more information.

Parameters:

• input – The input tensor to the layer.

• pre_padding – The padding to apply to the start of the tensor.

• post_padding – The padding to apply to the end of the tensor.

Returns:

The new padding layer, orNone if it could not be created.

add_parametric_relu(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,slopes:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IParametricReLULayer¶

Add a parametric ReLU layer.SeeIParametricReLULayer for more information.

Parameters:

• input – The input tensor to the layer.

• slopes – The slopes tensor (input elements are multiplied with the slopes where the input is negative).

Returns:

The new parametric ReLU layer, orNone if it could not be created.

add_plugin(*args,**kwargs)¶

Overloaded function.

1. add_plugin(self: tensorrt.tensorrt.INetworkDefinition, tuple: tuple) -> tensorrt.tensorrt.IPluginV3Layer

   Add a plugin layer to the network using anIPluginV3 interface.SeeIPluginV3 for more information.

   arg inputs:

   The input tensors to the layer.

   arg shape_inputs:

   The shape input tensors to the layer.

   arg plugin:

   The layer plugin.

   returns:

   The new plugin layer, orNone if it could not be created.

2. add_plugin(self: tensorrt.tensorrt.INetworkDefinition, func: function) -> tensorrt.tensorrt.IPluginV3Layer

3. add_plugin(self: tensorrt.tensorrt.INetworkDefinition, func: function, aot: bool) -> tensorrt.tensorrt.IPluginV3Layer

add_plugin_v2(self:tensorrt.tensorrt.INetworkDefinition,inputs:List[tensorrt.tensorrt.ITensor],plugin:tensorrt.tensorrt.IPluginV2)→tensorrt.tensorrt.IPluginV2Layer¶

Add a plugin layer to the network using anIPluginV2 interface.SeeIPluginV2 for more information.

Parameters:

• inputs – The input tensors to the layer.

• plugin – The layer plugin.

Returns:

The new plugin layer, orNone if it could not be created.

add_plugin_v3(self:tensorrt.tensorrt.INetworkDefinition,inputs:List[tensorrt.tensorrt.ITensor],shape_inputs:List[tensorrt.tensorrt.ITensor],plugin:tensorrt.tensorrt.IPluginV3)→tensorrt.tensorrt.IPluginV3Layer¶

Add a plugin layer to the network using anIPluginV3 interface.SeeIPluginV3 for more information.

Parameters:

• inputs – The input tensors to the layer.

• shape_inputs – The shape input tensors to the layer.

• plugin – The layer plugin.

Returns:

The new plugin layer, orNone if it could not be created.

add_pooling_nd(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,type:tensorrt.tensorrt.PoolingType,window_size:tensorrt.tensorrt.Dims)→tensorrt.tensorrt.IPoolingLayer¶

Add a multi-dimension pooling layer to the network.SeeIPoolingLayer for more information.

Parameters:

• input – The input tensor to the layer.

• type – The type of pooling to apply.

• window_size – The size of the pooling window.

Returns:

The new pooling layer, orNone if it could not be created.

add_quantize(*args,**kwargs)¶

Overloaded function.

1. add_quantize(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, scale: tensorrt.tensorrt.ITensor) -> tensorrt.tensorrt.IQuantizeLayer

   Add a quantization layer to the network.SeeIQuantizeLayer for more information.

   arg input:

   A tensor to quantize.

   arg scale:

   A tensor with the scale coefficients.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.int8).

   returns:

   The new quantization layer, orNone if it could not be created.

2. add_quantize(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, scale: tensorrt.tensorrt.ITensor, output_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.IQuantizeLayer

   Add a quantization layer to the network.SeeIQuantizeLayer for more information.

   arg input:

   A tensor to quantize.

   arg scale:

   A tensor with the scale coefficients.

   arg output_type:

   The datatype of the output tensor. Specifying output_type is optional (default value tensorrt.int8).

   returns:

   The new quantization layer, orNone if it could not be created.

add_ragged_softmax(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,bounds:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IRaggedSoftMaxLayer¶

Add a ragged softmax layer to the network.SeeIRaggedSoftMaxLayer for more information.

Parameters:

• input – The ZxS input tensor.

• bounds – The Zx1 bounds tensor.

Returns:

The new ragged softmax layer, orNone if it could not be created.

add_reduce(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,op:tensorrt.tensorrt.ReduceOperation,axes:int,keep_dims:bool)→tensorrt.tensorrt.IReduceLayer¶

Add a reduce layer to the network.SeeIReduceLayer for more information.

Parameters:

• input – The input tensor to the layer.

• op – The reduction operation to perform.

• axes – The reduction dimensions.The bit in position i of bitmask axes corresponds to explicit dimension i of the result.E.g., the least significant bit corresponds to the first explicit dimension and the next to leastsignificant bit corresponds to the second explicit dimension.

• keep_dims – The boolean that specifies whether or not to keep the reduced dimensions in the output of the layer.

Returns:

The new reduce layer, orNone if it could not be created.

add_resize(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IResizeLayer¶

Add a resize layer.SeeIResizeLayer for more information.

Parameters:

input – The input tensor to the layer.

Returns:

The new resize layer, orNone if it could not be created.

add_reverse_sequence(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,sequence_lens:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IReverseSequenceLayer¶

Adds a ReverseSequence layer to the network.SeeIReverseSequenceLayer for more information.

Parameters:

• input – The input tensor to the layer.

• sequence_lens – 1D tensor specifying lengths of sequences to reverse in a batch. The length ofsequence_lens must be equal to the size of the dimension ininput specified bybatch_axis.

Returns:

the new ReverseSequence layer, orNone if it could not be created.

add_scale(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,mode:tensorrt.tensorrt.ScaleMode,shift:tensorrt.tensorrt.Weights=None,scale:tensorrt.tensorrt.Weights=None,power:tensorrt.tensorrt.Weights=None)→tensorrt.tensorrt.IScaleLayer¶

Add a scale layer to the network.SeeIScaleLayer for more information.

Parameters:

• input – The input tensor to the layer. This tensor is required to have a minimum of 3 dimensions.

• mode – The scaling mode.

• shift – The shift value.

• scale – The scale value.

• power – The power value.

If the weights are available, then the size of weights are dependent on the ScaleMode.For UNIFORM, the number of weights is equal to 1.For CHANNEL, the number of weights is equal to the channel dimension.For ELEMENTWISE, the number of weights is equal to the volume of the input.

Returns:

The new scale layer, orNone if it could not be created.

add_scale_nd(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,mode:tensorrt.tensorrt.ScaleMode,shift:tensorrt.tensorrt.Weights=None,scale:tensorrt.tensorrt.Weights=None,power:tensorrt.tensorrt.Weights=None,channel_axis:int)→tensorrt.tensorrt.IScaleLayer¶

Add a multi-dimension scale layer to the network.SeeIScaleLayer for more information.

Parameters:

• input – The input tensor to the layer. This tensor is required to have a minimum of 3 dimensions.

• mode – The scaling mode.

• shift – The shift value.

• scale – The scale value.

• power – The power value.

• channel_axis – The channel dimension axis.

If the weights are available, then the size of weights are dependent on the ScaleMode.For UNIFORM, the number of weights is equal to 1.For CHANNEL, the number of weights is equal to the channel dimension.For ELEMENTWISE, the number of weights is equal to the volume of the input.

Returns:

The new scale layer, orNone if it could not be created.

add_scatter(self:tensorrt.tensorrt.INetworkDefinition,data:tensorrt.tensorrt.ITensor,indices:tensorrt.tensorrt.ITensor,updates:tensorrt.tensorrt.ITensor,mode:tensorrt.tensorrt.ScatterMode)→tensorrt.tensorrt.IScatterLayer¶

Add a scatter layer to the network.SeeIScatterLayer for more information.

Parameters:

• data – The tensor to get default values from.

• indices – The tensor to get indices from to populate the output tensor.

• updates – The tensor to get values from to populate the output tensor.

• mode – operation mode see IScatterLayer for more info

Returns:

The new Scatter layer, orNone if it could not be created.

add_select(self:tensorrt.tensorrt.INetworkDefinition,condition:tensorrt.tensorrt.ITensor,then_input:tensorrt.tensorrt.ITensor,else_input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.ISelectLayer¶

Add a select layer.SeeISelectLayer for more information.

Parameters:

• condition – The condition tensor to the layer.

• then_input – The then input tensor to the layer.

• else_input – The else input tensor to the layer.

Returns:

The new select layer, orNone if it could not be created.

add_shape(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IShapeLayer¶

Add a shape layer to the network.SeeIShapeLayer for more information.

Parameters:

input – The input tensor to the layer.

Returns:

The new shape layer, orNone if it could not be created.

add_shuffle(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IShuffleLayer¶

Add a shuffle layer to the network.SeeIShuffleLayer for more information.

Parameters:

input – The input tensor to the layer.

Returns:

The new shuffle layer, orNone if it could not be created.

add_slice(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,start:tensorrt.tensorrt.Dims,shape:tensorrt.tensorrt.Dims,stride:tensorrt.tensorrt.Dims)→tensorrt.tensorrt.ISliceLayer¶

Add a slice layer to the network.SeeISliceLayer for more information.

Parameters:

• input – The input tensor to the layer.

• start – The start offset.

• shape – The output shape.

• stride – The slicing stride. Positive, negative, zero stride values, and combinations of them in different dimensions are allowed.

Returns:

The new slice layer, orNone if it could not be created.

add_softmax(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.ISoftMaxLayer¶

Add a softmax layer to the network.SeeISoftMaxLayer for more information.

Parameters:

input – The input tensor to the layer.

Returns:

The new softmax layer, orNone if it could not be created.

add_squeeze(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,axes:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.ISqueezeLayer¶

Adds a Squeeze layer to the network.SeeISqueezeLayer for more information.

Parameters:

• input – The input tensor to the layer.

• axes – The tensor containing axes to remove. Must be resolvable to a constant Int32 or Int64 1D shape tensor.

Returns:

the new Squeeze layer, orNone if it could not be created.

add_topk(*args,**kwargs)¶

Overloaded function.

1. add_topk(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, op: tensorrt.tensorrt.TopKOperation, k: int, axes: int) -> tensorrt.tensorrt.ITopKLayer

   Add a TopK layer to the network.SeeITopKLayer for more information.

   The TopK layer has two outputs of the same dimensions. The first contains data values, the second contains index positions for the values. Output values are sorted, largest first for operationTopKOperation.MAX and smallest first for operationTopKOperation.MIN .

   Currently only values of K up to 3840 are supported.

   arg input:

   The input tensor to the layer.

   arg op:

   Operation to perform.

   arg k:

   Number of elements to keep.

   arg axes:

   The reduction dimensions.The bit in position i of bitmask axes corresponds to explicit dimension i of the result.E.g., the least significant bit corresponds to the first explicit dimension and the next to leastsignificant bit corresponds to the second explicit dimension.Currently axes must specify exactly one dimension, and it must be one of the last four dimensions.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   The new TopK layer, orNone if it could not be created.

2. add_topk(self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, op: tensorrt.tensorrt.TopKOperation, k: int, axes: int, indices_type: tensorrt.tensorrt.DataType) -> tensorrt.tensorrt.ITopKLayer

   Add a TopK layer to the network.SeeITopKLayer for more information.

   The TopK layer has two outputs of the same dimensions. The first contains data values, the second contains index positions for the values. Output values are sorted, largest first for operationTopKOperation.MAX and smallest first for operationTopKOperation.MIN .

   Currently only values of K up to 3840 are supported.

   arg input:

   The input tensor to the layer.

   arg op:

   Operation to perform.

   arg k:

   Number of elements to keep.

   arg axes:

   The reduction dimensions.The bit in position i of bitmask axes corresponds to explicit dimension i of the result.E.g., the least significant bit corresponds to the first explicit dimension and the next to leastsignificant bit corresponds to the second explicit dimension.Currently axes must specify exactly one dimension, and it must be one of the last four dimensions.

   arg indices_type:

   The datatype of the output indices tensor. Specifying indices_type is optional (default value tensorrt.int32).

   returns:

   The new TopK layer, orNone if it could not be created.

add_unary(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,op:tensorrt.tensorrt.UnaryOperation)→tensorrt.tensorrt.IUnaryLayer¶

Add a unary layer to the network.SeeIUnaryLayer for more information.

Parameters:

• input – The input tensor to the layer.

• op – The operation to apply.

Returns:

The new unary layer, orNone if it could not be created.

add_unsqueeze(self:tensorrt.tensorrt.INetworkDefinition,input:tensorrt.tensorrt.ITensor,axes:tensorrt.tensorrt.ITensor)→tensorrt.tensorrt.IUnsqueezeLayer¶

Adds an Unsqueeze layer to the network.SeeIUnsqueezeLayer for more information.

Parameters:

• input – The input tensor to the layer.

• axes – The tensor containing axes to add. Must be resolvable to a constant Int32 or Int64 1D shape tensor.

Returns:

the new Unsqueeze layer, orNone if it could not be created.

are_weights_marked_refittable(self:tensorrt.tensorrt.INetworkDefinition,name:str)→bool¶

Whether the weight has been marked as refittable.

Parameters:

name – The name of the weights to check.

propertybuilder¶

The builder from which this INetworkDefinition was created.

SeeIBuilder for more information.

get_flag(self:tensorrt.tensorrt.INetworkDefinition,flag:tensorrt.NetworkDefinitionCreationFlag)→bool¶

Returns true if the specifiedNetworkDefinitionCreationFlag is set.

Parameters:

flag – TheNetworkDefinitionCreationFlag .

Returns:

Whether the flag is set.

get_input(self:tensorrt.tensorrt.INetworkDefinition,index:int)→tensorrt.tensorrt.ITensor¶

Get the input tensor specified by the given index.

Parameters:

index – The index of the input tensor.

Returns:

The tensor, orNone if it is out of range.

get_layer(self:tensorrt.tensorrt.INetworkDefinition,index:int)→tensorrt.tensorrt.ILayer¶

Get the layer specified by the given index.

Parameters:

index – The index of the layer.

Returns:

The layer, orNone if it is out of range.

get_output(self:tensorrt.tensorrt.INetworkDefinition,index:int)→tensorrt.tensorrt.ITensor¶

Get the output tensor specified by the given index.

Parameters:

index – The index of the output tensor.

Returns:

The tensor, orNone if it is out of range.

is_debug_tensor(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→bool¶

Check if a tensor is marked as debug.

Parameters:

tensor – The tensor to be checked.

mark_debug(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→bool¶

Mark a tensor as a debug tensor in the network.

Parameters:

tensor – The tensor to be marked as debug tensor.

Returns:

True on success, False otherwise.

mark_output(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→None¶

Mark a tensor as an output.

Parameters:

tensor – The tensor to mark.

mark_output_for_shapes(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→bool¶

Enable tensor’s value to be computed byIExecutionContext.get_shape_binding().

Parameters:

tensor – The tensor to unmark as an output tensor. The tensor must be of typeint32 and have no more than one dimension.

Returns:

True if successful,False if tensor is already marked as an output.

mark_unfused_tensors_as_debug_tensors(self:tensorrt.tensorrt.INetworkDefinition)→bool¶

Mark unfused tensors as debug tensors.

Debug tensors can be optionally emitted at runtime.Tensors that are fused by the optimizer will not be emitted.Tensors marked this way will not prevent fusion like mark_debug() does, thus preserving performance.

Tensors marked this way cannot be detected by is_debug_tensor().DebugListener can only get internal tensor names instead of the original tensor names in the NetworkDefinition for tensors marked this way.But the names correspond to the names obtained by IEngineInspector.There is no guarantee that all unfused tensors are marked.

Returns:

True if tensors were successfully marked (or were already marked), false otherwise.

mark_weights_refittable(self:tensorrt.tensorrt.INetworkDefinition,name:str)→bool¶

Mark a weight as refittable.

Parameters:

name – The weight to mark.

remove_tensor(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→None¶

Remove a tensor from the network.

Parameters:

tensor – The tensor to remove

It is illegal to remove a tensor that is the input or output of a layer.if this method is called with such a tensor, a warning will be emitted on the logand the call will be ignored.

set_weights_name(self:tensorrt.tensorrt.INetworkDefinition,weights:tensorrt.tensorrt.Weights,name:str)→bool¶

Associate a name with all current uses of the given weights.

The name must be set after the Weights are used in the network.Lookup is associative. The name applies to all Weights with matchingtype, value pointer, and count. If Weights with a matching valuepointer, but different type or count exists in the network, anerror message is issued, the name is rejected, and return false.If the name has already been used for other weights,return false. None causes the weights to become unnamed,i.e. clears any previous name.

Parameters:

• weights – The weights to be named.

• name – The name to associate with the weights.

Returns:

true on success.

unmark_debug(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→bool¶

Unmark a tensor as a debug tensor in the network.

Parameters:

tensor – The tensor to be unmarked as debug tensor.

Returns:

True on success, False otherwise.

unmark_output(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→None¶

Unmark a tensor as a network output.

Parameters:

tensor – The tensor to unmark as an output tensor.

unmark_output_for_shapes(self:tensorrt.tensorrt.INetworkDefinition,tensor:tensorrt.tensorrt.ITensor)→bool¶

Undomark_output_for_shapes() .

Parameters:

tensor – The tensor to unmark as an output tensor.

Returns:

True if successful,False if tensor is not marked as an output.

unmark_unfused_tensors_as_debug_tensors(self:tensorrt.tensorrt.INetworkDefinition)→bool¶

Undo the marking of unfused tensor as debug tensors.

This has no effect on tensors marked by mark_debug().

Returns:

True if tensor successfully unmarked (or was already unmarked), false otherwise.

unmark_weights_refittable(self:tensorrt.tensorrt.INetworkDefinition,name:str)→bool¶

Unmark a weight as refittable.

Parameters:

name – The weight to unmark.