Initializing a Tensor#

Tensors can be initialized in various ways. Take a look at the following examples:

Directly from data

Tensors can be created directly from data. The data type is automatically inferred.

data=[[1,2],[3,4]]x_data=torch.tensor(data)

From a NumPy array

Tensors can be created from NumPy arrays (and vice versa - seeBridge with NumPy).

np_array=np.array(data)x_np=torch.from_numpy(np_array)

From another tensor:

The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.

x_ones=torch.ones_like(x_data)# retains the properties of x_dataprint(f"Ones Tensor:\n{x_ones}\n")x_rand=torch.rand_like(x_data,dtype=torch.float)# overrides the datatype of x_dataprint(f"Random Tensor:\n{x_rand}\n")

Ones Tensor: tensor([[1, 1],        [1, 1]])Random Tensor: tensor([[0.6379, 0.1982],        [0.2667, 0.1378]])

With random or constant values:

shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.

shape=(2,3,)rand_tensor=torch.rand(shape)ones_tensor=torch.ones(shape)zeros_tensor=torch.zeros(shape)print(f"Random Tensor:\n{rand_tensor}\n")print(f"Ones Tensor:\n{ones_tensor}\n")print(f"Zeros Tensor:\n{zeros_tensor}")

Random Tensor: tensor([[0.4122, 0.1048, 0.6723],        [0.0083, 0.1560, 0.6437]])Ones Tensor: tensor([[1., 1., 1.],        [1., 1., 1.]])Zeros Tensor: tensor([[0., 0., 0.],        [0., 0., 0.]])

Attributes of a Tensor#

Tensor attributes describe their shape, datatype, and the device on which they are stored.

tensor=torch.rand(3,4)print(f"Shape of tensor:{tensor.shape}")print(f"Datatype of tensor:{tensor.dtype}")print(f"Device tensor is stored on:{tensor.device}")

Shape of tensor: torch.Size([3, 4])Datatype of tensor: torch.float32Device tensor is stored on: cpu

Operations on Tensors#

Over 1200 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing,indexing, slicing), sampling and more arecomprehensively describedhere.

Each of these operations can be run on the CPU andAcceleratorsuch as CUDA, MPS, MTIA, or XPU. If you’re using Colab, allocate an accelerator by going to Runtime > Change runtime type > GPU.

By default, tensors are created on the CPU. We need to explicitly move tensors to the accelerator using.to method (after checking for accelerator availability). Keep in mind that copying large tensorsacross devices can be expensive in terms of time and memory!

# We move our tensor to the current accelerator if availableiftorch.accelerator.is_available():tensor=tensor.to(torch.accelerator.current_accelerator())

Try out some of the operations from the list.If you’re familiar with the NumPy API, you’ll find the Tensor API a breeze to use.

Standard numpy-like indexing and slicing:

tensor=torch.ones(4,4)print(f"First row:{tensor[0]}")print(f"First column:{tensor[:,0]}")print(f"Last column:{tensor[...,-1]}")tensor[:,1]=0print(tensor)

First row: tensor([1., 1., 1., 1.])First column: tensor([1., 1., 1., 1.])Last column: tensor([1., 1., 1., 1.])tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])

Joining tensors You can usetorch.cat to concatenate a sequence of tensors along a given dimension.See alsotorch.stack,another tensor joining operator that is subtly different fromtorch.cat.

t1=torch.cat([tensor,tensor,tensor],dim=1)print(t1)

tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])

Arithmetic operations

# This computes the matrix multiplication between two tensors. y1, y2, y3 will have the same value# ``tensor.T`` returns the transpose of a tensory1=tensor@tensor.Ty2=tensor.matmul(tensor.T)y3=torch.rand_like(y1)torch.matmul(tensor,tensor.T,out=y3)# This computes the element-wise product. z1, z2, z3 will have the same valuez1=tensor*tensorz2=tensor.mul(tensor)z3=torch.rand_like(tensor)torch.mul(tensor,tensor,out=z3)

tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])

Single-element tensors If you have a one-element tensor, for example by aggregating allvalues of a tensor into one value, you can convert it to a Pythonnumerical value usingitem():

agg=tensor.sum()agg_item=agg.item()print(agg_item,type(agg_item))

12.0 <class 'float'>

In-place operationsOperations that store the result into the operand are called in-place. They are denoted by a_ suffix.For example:x.copy_(y),x.t_(), will changex.

print(f"{tensor}\n")tensor.add_(5)print(tensor)

tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])tensor([[6., 5., 6., 6.],        [6., 5., 6., 6.],        [6., 5., 6., 6.],        [6., 5., 6., 6.]])

Bridge with NumPy#

Tensors on the CPU and NumPy arrays can share their underlying memorylocations, and changing one will change the other.

t=torch.ones(5)print(f"t:{t}")n=t.numpy()print(f"n:{n}")

t: tensor([1., 1., 1., 1., 1.])n: [1. 1. 1. 1. 1.]

t.add_(1)print(f"t:{t}")print(f"n:{n}")

t: tensor([2., 2., 2., 2., 2.])n: [2. 2. 2. 2. 2.]

n=np.ones(5)t=torch.from_numpy(n)

np.add(n,1,out=n)print(f"t:{t}")print(f"n:{n}")

t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)n: [2. 2. 2. 2. 2.]