Efficient Access to Tensor Elements#

When using Tensor-wide operations, the relative cost of dynamic dispatch is verysmall. However, there are cases, especially in your own kernels, where efficientelement-wise access is needed, and the cost of dynamic dispatch inside theelement-wise loop is very high. ATen providesaccessors that are created witha single dynamic check that a Tensor is the type and number of dimensions.Accessors then expose an API for accessing the Tensor elements efficiently.

Accessors are temporary views of a Tensor. They are only valid for the lifetimeof the tensor that they view and hence should only be used locally in afunction, like iterators.

Note that accessors are not compatible with CUDA tensors inside kernel functions.Instead, you will have to use apacked accessor which behaves the same way butcopies tensor metadata instead of pointing to it.

It is thus recommended to useaccessors for CPU tensors andpacked accessorsfor CUDA tensors.

torch::Tensorfoo=torch::rand({12,12});// assert foo is 2-dimensional and holds floats.autofoo_a=foo.accessor<float,2>();floattrace=0;for(inti=0;i<foo_a.size(0);i++){// use the accessor foo_a to get tensor data.trace+=foo_a[i][i];}

__global__voidpacked_accessor_kernel(torch::PackedTensorAccessor64<float,2>foo,float*trace){inti=threadIdx.x;gpuAtomicAdd(trace,foo[i][i]);}torch::Tensorfoo=torch::rand({12,12});// assert foo is 2-dimensional and holds floats.autofoo_a=foo.packed_accessor64<float,2>();floattrace=0;packed_accessor_kernel<<<1,12>>>(foo_a,&trace);

Using Externally Created Data#

If you already have your tensor data allocated in memory (CPU or CUDA),you can view that memory as aTensor in ATen:

floatdata[]={1,2,3,4,5,6};torch::Tensorf=torch::from_blob(data,{2,3});

These tensors cannot be resized because ATen does not own the memory, butotherwise behave as normal tensors.

Scalars and zero-dimensional tensors#

In addition to theTensor objects, ATen also includesScalars thatrepresent a single number. Like a Tensor, Scalars are dynamically typed and canhold any one of ATen’s number types. Scalars can be implicitly constructed fromC++ number types. Scalars are needed because some functions likeaddmm takenumbers along with Tensors and expect these numbers to be the same dynamic typeas the tensor. They are also used in the API to indicate places where a functionwillalways return a Scalar value, likesum.

namespacetorch{Tensoraddmm(Scalarbeta,constTensor&self,Scalaralpha,constTensor&mat1,constTensor&mat2);Scalarsum(constTensor&self);}// namespace torch// Usage.torch::Tensora=...torch::Tensorb=...torch::Tensorc=...torch::Tensorr=torch::addmm(1.0,a,.5,b,c);

In addition toScalars, ATen also allowsTensor objects to bezero-dimensional. These Tensors hold a single value and they can be referencesto a single element in a largerTensor. They can be used anywhere aTensor is expected. They are normally created by operators likeselectwhich reduce the dimensions of aTensor.

torch::Tensortwo=torch::rand({10,20});two[1][2]=4;// ^^^^^^ <- zero-dimensional Tensor