Picking a Factory Function#

The following factory functions are available at the time of this writing (thehyperlinks lead to the corresponding Python functions, since they often havemore eloquent documentation – the options are the same in C++):

• arange: Returns a tensor with a sequence of integers,

• empty: Returns a tensor with uninitialized values,

• eye: Returns an identity matrix,

• full: Returns a tensor filled with a single value,

• linspace: Returns a tensor with values linearly spaced in some interval,

• logspace: Returns a tensor with values logarithmically spaced in some interval,

• ones: Returns a tensor filled with all ones,

• rand: Returns a tensor filled with values drawn from a uniform distribution on[0,1).

• randint: Returns a tensor with integers randomly drawn from an interval,

• randn: Returns a tensor filled with values drawn from a unit normal distribution,

• randperm: Returns a tensor filled with a random permutation of integers in some interval,

• zeros: Returns a tensor filled with all zeros.

Specifying a Size#

Functions that do not require specific arguments by nature of how they fill thetensor can be invoked with just a size. For example, the following line createsa vector with 5 components, initially all set to 1:

torch::Tensortensor=torch::ones(5);

What if we wanted to instead create a3x5 matrix, or a2x3x4tensor? In general, anIntArrayRef – the type of the size parameter of factoryfunctions – is constructed by specifying the size along each dimension incurly braces. For example,{2,3} for a tensor (in this case matrix) withtwo rows and three columns,{3,4,5} for a three-dimensional tensor, and{2} for a one-dimensional tensor with two components. In the onedimensional case, you can omit the curly braces and just pass the singleinteger like we did above. Note that the squiggly braces are just one way ofconstructing anIntArrayRef. You can also pass anstd::vector<int64_t> anda few other types. Either way, this means we can construct a three-dimensionaltensor filled with values from a unit normal distribution by writing:

torch::Tensortensor=torch::randn({3,4,5});assert(tensor.sizes()==std::vector<int64_t>{3,4,5});

tensor.sizes() returns anIntArrayRef which can be compared against anstd::vector<int64_t>, and we can see that it contains the sizes we passedto the tensor. You can also writetensor.size(i) to access a single dimension,which is equivalent to but preferred overtensor.sizes()[i].

Passing Function-Specific Parameters#

Neitherones norrandn accept any additional parameters to change theirbehavior. One function which does require further configuration israndint,which takes an upper bound on the value for the integers it generates, as wellas an optional lower bound, which defaults to zero. Here we create a5x5square matrix with integers between 0 and 10:

torch::Tensortensor=torch::randint(/*high=*/10,{5,5});

And here we raise the lower bound to 3:

torch::Tensortensor=torch::randint(/*low=*/3,/*high=*/10,{5,5});

The inline comments/*low=*/ and/*high=*/ are not required of course,but aid readability just like keyword arguments in Python.

Configuring Properties of the Tensor#

The previous section discussed function-specific arguments. Function-specificarguments can only change the values with which tensors are filled, andsometimes the size of the tensor. They never change things like the data type(e.g.float32 orint64) of the tensor being created, or whether itlives in CPU or GPU memory. The specification of these properties is left tothe very last argument to every factory function: aTensorOptions object,discussed below.

TensorOptions is a class that encapsulates the construction axes of aTensor. Withconstruction axis we mean a particular property of a Tensor thatcan be configured before its construction (and sometimes changed afterwards).These construction axes are:

• Thedtype (previously “scalar type”), which controls the data type of theelements stored in the tensor,

• Thelayout, which is either strided (dense) or sparse,

• Thedevice, which represents a compute device on which a tensor is stored (like a CPU or CUDA GPU),

• Therequires_grad boolean to enable or disable gradient recording for a tensor,

If you are used to PyTorch in Python, these axes will sound very familiar. Theallowed values for these axes at the moment are:

• Fordtype:kUInt8,kInt8,kInt16,kInt32,kInt64,kFloat32 andkFloat64,

• Forlayout:kStrided andkSparse,

• Fordevice: EitherkCPU, orkCUDA (which accepts an optional device index),

• Forrequires_grad: eithertrue orfalse.

An instance ofTensorOptions stores a concrete value for each of theseaxes. Here is an example of creating aTensorOptions object that representsa 64-bit float, strided tensor that requires a gradient, and lives on CUDAdevice 1:

autooptions=torch::TensorOptions().dtype(torch::kFloat32).layout(torch::kStrided).device(torch::kCUDA,1).requires_grad(true);

Notice how we use the ‘“builder”-style methods ofTensorOptions toconstruct the object piece by piece. If we pass this object as the lastargument to a factory function, the newly created tensor will have theseproperties:

torch::Tensortensor=torch::full({3,4},/*value=*/123,options);assert(tensor.dtype()==torch::kFloat32);assert(tensor.layout()==torch::kStrided);assert(tensor.device().type()==torch::kCUDA);// or device().is_cuda()assert(tensor.device().index()==1);assert(tensor.requires_grad());

Now, you may be thinking: do I really need to specify each axis for every newtensor I create? Fortunately, the answer is “no”, asevery axis has a defaultvalue. These defaults are:

• kFloat32 for the dtype,

• kStrided for the layout,

• kCPU for the device,

• false forrequires_grad.

What this means is that any axis you omit during the construction of aTensorOptions object will take on its default value. For example, this isour previousTensorOptions object, but with thedtype andlayoutdefaulted:

autooptions=torch::TensorOptions().device(torch::kCUDA,1).requires_grad(true);

In fact, we can even omit all axes to get an entirely defaultedTensorOptions object:

autooptions=torch::TensorOptions();// or `torch::TensorOptions options;`

A nice consequence of this is that theTensorOptions object we just spokeso much about can be entirely omitted from any tensor factory call:

// A 32-bit float, strided, CPU tensor that does not require a gradient.torch::Tensortensor=torch::randn({3,4});torch::Tensorrange=torch::arange(5,10);

But the sugar gets sweeter: In the API presented here so far, you may havenoticed that the initialtorch::TensorOptions() is quite a mouthful towrite. The good news is that for every construction axis (dtype, layout, deviceandrequires_grad), there is onefree function in thetorch::namespace which you can pass a value for that axis. Each function then returnsaTensorOptions object preconfigured with that axis, but allowing evenfurther modification via the builder-style methods shown above. For example,

torch::ones(10,torch::TensorOptions().dtype(torch::kFloat32))

is equivalent to

torch::ones(10,torch::dtype(torch::kFloat32))

and further instead of

torch::ones(10,torch::TensorOptions().dtype(torch::kFloat32).layout(torch::kStrided))

we can just write

torch::ones(10,torch::dtype(torch::kFloat32).layout(torch::kStrided))

which saves us quite a bit of typing. What this means is that in practice, youshould barely, if ever, have to write outtorch::TensorOptions. Instead usethetorch::dtype(),torch::device(),torch::layout() andtorch::requires_grad() functions.

A final bit of convenience is thatTensorOptions is implicitlyconstructible from individual values. This means that whenever a function has aparameter of typeTensorOptions, like all factory functions do, we candirectly pass a value liketorch::kFloat32 ortorch::kStrided in placeof the full object. Therefore, when there is only a single axis we would liketo change compared to its default value, we can pass only that value. As such,what was

torch::ones(10,torch::TensorOptions().dtype(torch::kFloat32))

became

torch::ones(10,torch::dtype(torch::kFloat32))

and can finally be shortened to

torch::ones(10,torch::kFloat32)

Of course, it is not possible to modify further properties of theTensorOptions instance with this short syntax, but if all we needed was tochange one property, this is quite practical.

In conclusion, we can now compare howTensorOptions defaults, together withthe abbreviated API for creatingTensorOptions using free functions, allowtensor creation in C++ with the same convenience as in Python. Compare thiscall in Python:

torch.randn(3,4,dtype=torch.float32,device=torch.device('cuda',1),requires_grad=True)

with the equivalent call in C++:

torch::randn({3,4},torch::dtype(torch::kFloat32).device(torch::kCUDA,1).requires_grad(true))

Pretty close!