Rate this Page
★★★★★

Note

Go to the endto download the full example code.

Tensors#

Created On: Mar 24, 2017 | Last Updated: Jan 16, 2024 | Last Verified: Nov 05, 2024

Tensors are a specialized data structure that are very similar to arraysand matrices. In PyTorch, we use tensors to encode the inputs andoutputs of a model, as well as the model’s parameters.

Tensors are similar to NumPy’s ndarrays, except that tensors can run onGPUs or other specialized hardware to accelerate computing. If you’re familiar with ndarrays, you’llbe right at home with the Tensor API. If not, follow along in this quickAPI walkthrough.

importtorchimportnumpyasnp

Tensor Initialization#

Tensors can be initialized in various ways. Take a look at the following examples:

Directly from data

Tensors can be created directly from data. The data type is automatically inferred.

data=[[1,2],[3,4]]x_data=torch.tensor(data)

From a NumPy array

Tensors can be created from NumPy arrays (and vice versa - seeBridge with NumPy).

np_array=np.array(data)x_np=torch.from_numpy(np_array)

From another tensor:

The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.

x_ones=torch.ones_like(x_data)# retains the properties of x_dataprint(f"Ones Tensor:\n{x_ones}\n")x_rand=torch.rand_like(x_data,dtype=torch.float)# overrides the datatype of x_dataprint(f"Random Tensor:\n{x_rand}\n")
Ones Tensor: tensor([[1, 1],        [1, 1]])Random Tensor: tensor([[0.1205, 0.6931],        [0.7261, 0.6252]])

With random or constant values:

shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.

shape=(2,3,)rand_tensor=torch.rand(shape)ones_tensor=torch.ones(shape)zeros_tensor=torch.zeros(shape)print(f"Random Tensor:\n{rand_tensor}\n")print(f"Ones Tensor:\n{ones_tensor}\n")print(f"Zeros Tensor:\n{zeros_tensor}")
Random Tensor: tensor([[0.2630, 0.6093, 0.4630],        [0.7375, 0.8007, 0.3654]])Ones Tensor: tensor([[1., 1., 1.],        [1., 1., 1.]])Zeros Tensor: tensor([[0., 0., 0.],        [0., 0., 0.]])

Tensor Attributes#

Tensor attributes describe their shape, datatype, and the device on which they are stored.

tensor=torch.rand(3,4)print(f"Shape of tensor:{tensor.shape}")print(f"Datatype of tensor:{tensor.dtype}")print(f"Device tensor is stored on:{tensor.device}")
Shape of tensor: torch.Size([3, 4])Datatype of tensor: torch.float32Device tensor is stored on: cpu

Tensor Operations#

Over 100 tensor operations, including transposing, indexing, slicing,mathematical operations, linear algebra, random sampling, and more arecomprehensively describedhere.

Each of them can be run on the GPU (at typically higher speeds than on aCPU). If you’re using Colab, allocate a GPU by going to Edit > NotebookSettings.

# We move our tensor to the GPU if availableiftorch.cuda.is_available():tensor=tensor.to('cuda')print(f"Device tensor is stored on:{tensor.device}")
Device tensor is stored on: cuda:0

Try out some of the operations from the list.If you’re familiar with the NumPy API, you’ll find the Tensor API a breeze to use.

Standard numpy-like indexing and slicing:

tensor=torch.ones(4,4)tensor[:,1]=0print(tensor)
tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])

Joining tensors You can usetorch.cat to concatenate a sequence of tensors along a given dimension.See alsotorch.stack,another tensor joining op that is subtly different fromtorch.cat.

tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],        [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])

Multiplying tensors

# This computes the element-wise productprint(f"tensor.mul(tensor)\n{tensor.mul(tensor)}\n")# Alternative syntax:print(f"tensor * tensor\n{tensor*tensor}")
tensor.mul(tensor) tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])tensor * tensor tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])

This computes the matrix multiplication between two tensors

print(f"tensor.matmul(tensor.T)\n{tensor.matmul(tensor.T)}\n")# Alternative syntax:print(f"tensor @ tensor.T\n{tensor@tensor.T}")
tensor.matmul(tensor.T) tensor([[3., 3., 3., 3.],        [3., 3., 3., 3.],        [3., 3., 3., 3.],        [3., 3., 3., 3.]])tensor @ tensor.T tensor([[3., 3., 3., 3.],        [3., 3., 3., 3.],        [3., 3., 3., 3.],        [3., 3., 3., 3.]])

In-place operationsOperations that have a_ suffix are in-place. For example:x.copy_(y),x.t_(), will changex.

print(tensor,"\n")tensor.add_(5)print(tensor)
tensor([[1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.],        [1., 0., 1., 1.]])tensor([[6., 5., 6., 6.],        [6., 5., 6., 6.],        [6., 5., 6., 6.],        [6., 5., 6., 6.]])

Note

In-place operations save some memory, but can be problematic when computing derivatives because of an immediate lossof history. Hence, their use is discouraged.

Bridge with NumPy#

Tensors on the CPU and NumPy arrays can share their underlying memorylocations, and changing one will change the other.

Tensor to NumPy array#

t=torch.ones(5)print(f"t:{t}")n=t.numpy()print(f"n:{n}")
t: tensor([1., 1., 1., 1., 1.])n: [1. 1. 1. 1. 1.]

A change in the tensor reflects in the NumPy array.

t.add_(1)print(f"t:{t}")print(f"n:{n}")
t: tensor([2., 2., 2., 2., 2.])n: [2. 2. 2. 2. 2.]

NumPy array to Tensor#

n=np.ones(5)t=torch.from_numpy(n)

Changes in the NumPy array reflects in the tensor.

np.add(n,1,out=n)print(f"t:{t}")print(f"n:{n}")
t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)n: [2. 2. 2. 2. 2.]

Total running time of the script: (0 minutes 0.465 seconds)