Tensor Views#

Created On: Feb 28, 2020 | Last Updated On: Feb 26, 2025

PyTorch allows a tensor to be aView of an existing tensor. View tensor shares the same underlying datawith its base tensor. SupportingView avoids explicit data copy, thus allows us to do fast and memory efficientreshaping, slicing and element-wise operations.

For example, to get a view of an existing tensort, you can callt.view(...).

>>>t=torch.rand(4,4)>>>b=t.view(2,8)>>>t.storage().data_ptr()==b.storage().data_ptr()# `t` and `b` share the same underlying data.True# Modifying view tensor changes base tensor as well.>>>b[0][0]=3.14>>>t[0][0]tensor(3.14)

Since views share underlying data with its base tensor, if you edit the datain the view, it will be reflected in the base tensor as well.

Typically a PyTorch op returns a new tensor as output, e.g.add().But in case of view ops, outputs are views of input tensors to avoid unnecessary data copy.No data movement occurs when creating a view, view tensor just changes the wayit interprets the same data. Taking a view of contiguous tensor could potentially produce a non-contiguous tensor.Users should pay additional attention as contiguity might have implicit performance impact.transpose() is a common example.

>>>base=torch.tensor([[0,1],[2,3]])>>>base.is_contiguous()True>>>t=base.transpose(0,1)# `t` is a view of `base`. No data movement happened here.# View tensors might be non-contiguous.>>>t.is_contiguous()False# To get a contiguous tensor, call `.contiguous()` to enforce# copying data when `t` is not contiguous.>>>c=t.contiguous()

For reference, here’s a full list of view ops in PyTorch:

• Basic slicing and indexing op, e.g.tensor[0,2:,1:7:2] returns a view of basetensor, see note below.

• adjoint()

• as_strided()

• detach()

• diagonal()

• expand()

• expand_as()

• movedim()

• narrow()

• permute()

• select()

• squeeze()

• transpose()

• t()

• T

• H

• mT

• mH

• real

• imag

• view_as_real()

• unflatten()

• unfold()

• unsqueeze()

• view()

• view_as()

• unbind()

• split()

• hsplit()

• vsplit()

• tensor_split()

• split_with_sizes()

• swapaxes()

• swapdims()

• chunk()

• indices() (sparse tensor only)

• values() (sparse tensor only)

It’s also worth mentioning a few ops with special behaviors:

• reshape(),reshape_as() andflatten() can return either a view or new tensor, user code shouldn’t rely on whether it’s view or not.

• contiguous() returnsitself if input tensor is already contiguous, otherwise it returns a new contiguous tensor by copying data.

For a more detailed walk-through of PyTorch internal implementation,please refer toezyang’s blogpost about PyTorch Internals.