Untyped Storage API#

Atorch.UntypedStorage is a contiguous, one-dimensional array of elements. Its length is equal to the number ofbytes of the tensor. The storage serves as the underlying data container for tensors.In general, a tensor created in PyTorch using regular constructors such aszeros(),zeros_like()ornew_zeros() will produce tensors where there is a one-to-one correspondence between the tensorstorage and the tensor itself.

However, a storage is allowed to be shared by multiple tensors.For instance, any view of a tensor (obtained throughview() or some, but not all, kinds of indexinglike integers and slices) will point to the same underlying storage as the original tensor.When serializing and deserializing tensors that share a common storage, the relationship is preserved, and the tensorscontinue to point to the same storage. Interestingly, deserializing multiple tensors that point to a single storagecan be faster than deserializing multiple independent tensors.

A tensor storage can be accessed through theuntyped_storage() method. This will return an object oftypetorch.UntypedStorage.Fortunately, storages have a unique identifier accessed through thetorch.UntypedStorage.data_ptr() method.In regular settings, two tensors with the same data storage will have the same storagedata_ptr.However, tensors themselves can point to two separate storages, one for its data attribute and another for its gradattribute. Each will require adata_ptr() of its own. In general, there is no guarantee that atorch.Tensor.data_ptr() andtorch.UntypedStorage.data_ptr() match and this should not be assumed to be true.

Untyped storages are somewhat independent of the tensors that are built on them. Practically, this means that tensorswith different dtypes or shape can point to the same storage.It also implies that a tensor storage can be changed, as the following example shows:

>>>t=torch.ones(3)>>>s0=t.untyped_storage()>>>s0 0 0 128 63 0 0 128 63 0 0 128 63[torch.storage.UntypedStorage(device=cpu) of size 12]>>>s1=s0.clone()>>>s1.fill_(0) 0 0 0 0 0 0 0 0 0 0 0 0[torch.storage.UntypedStorage(device=cpu) of size 12]>>># Fill the tensor with a zeroed storage>>>t.set_(s1,storage_offset=t.storage_offset(),stride=t.stride(),size=t.size())tensor([0., 0., 0.])

Other thandata_ptr, untyped storage also have other attributes such asfilename(in case the storage points to a file on disk),device oris_cuda for device checks. A storage can also be manipulated in-place orout-of-place with methods likecopy_,fill_ orpin_memory. FOr more information, check the APIreference below. Keep in mind that modifying storages is a low-level API and comes with risks!Most of these APIs also exist on the tensor level: if present, they should be prioritized over their storagecounterparts.

Special cases#

We mentioned that a tensor that has a non-Nonegrad attribute has actually two pieces of data within it.In this case,untyped_storage() will return the storage of thedata attribute,whereas the storage of the gradient can be obtained throughtensor.grad.untyped_storage().

>>>t=torch.zeros(3,requires_grad=True)>>>t.sum().backward()>>>assertlist(t.untyped_storage())==[0]*12# the storage of the tensor is just 0s>>>assertlist(t.grad.untyped_storage())!=[0]*12# the storage of the gradient isn't

There are also special cases where tensors do not have a typical storage, or no storage at all:

• Tensors on"meta" device: Tensors on the"meta" device are used for shape inferenceand do not hold actual data.

• Fake Tensors: Another internal tool used by PyTorch’s compiler isFakeTensor which is based on a similar idea.

Tensor subclasses or tensor-like objects can also display unusual behaviours. In general, we do notexpect many use cases to require operating at the Storage level!

classtorch.UntypedStorage(*args,**kwargs)[source]#

bfloat16()[source]#

Casts this storage to bfloat16 type.

bool()[source]#

Casts this storage to bool type.

byte()[source]#

Casts this storage to byte type.

byteswap(dtype)[source]#

Swap bytes in underlying data.

char()[source]#

Casts this storage to char type.

clone()[source]#

Return a copy of this storage.

complex_double()[source]#

Casts this storage to complex double type.

complex_float()[source]#

Casts this storage to complex float type.

copy_()#

cpu()[source]#

Return a CPU copy of this storage if it’s not already on the CPU.

cuda(device=None,non_blocking=False)[source]#

Returns a copy of this object in CUDA memory.

If this object is already in CUDA memory and on the correct device, thenno copy is performed and the original object is returned.

Parameters:

• device (int) – The destination GPU id. Defaults to the current device.

• non_blocking (bool) – IfTrue and the source is in pinned memory,the copy will be asynchronous with respect to the host. Otherwise,the argument has no effect.

Return type:

_StorageBase |TypedStorage

data_ptr()#

device:device#

double()[source]#

Casts this storage to double type.

element_size()#

propertyfilename:str|None#

Returns the file name associated with this storage.

The file name will be a string if the storage is on CPU and was created viafrom_file() withshared asTrue. This attribute isNone otherwise.

fill_()#

float()[source]#

Casts this storage to float type.

float8_e4m3fn()[source]#

Casts this storage to float8_e4m3fn type

float8_e4m3fnuz()[source]#

Casts this storage to float8_e4m3fnuz type

float8_e5m2()[source]#

Casts this storage to float8_e5m2 type

float8_e5m2fnuz()[source]#

Casts this storage to float8_e5m2fnuz type

staticfrom_buffer()#

staticfrom_file(filename,shared=False,nbytes=0)→Storage#

Creates a CPU storage backed by a memory-mapped file.

Ifshared isTrue, then memory is shared between all processes.All changes are written to the file. Ifshared isFalse, then the changes onthe storage do not affect the file.

nbytes is the number of bytes of storage. Ifshared isFalse,then the file must contain at leastnbytes bytes. Ifshared isTrue the file will be created if needed. (Note that forUntypedStoragethis argument differs from that ofTypedStorage.from_file)

Parameters:

• filename (str) – file name to map

• shared (bool) – whether to share memory (whetherMAP_SHARED orMAP_PRIVATE is passed to theunderlyingmmap(2) call)

• nbytes (int) – number of bytes of storage

get_device()[source]#

Return type:

int

half()[source]#

Casts this storage to half type.

hpu(device=None,non_blocking=False)[source]#

Returns a copy of this object in HPU memory.

If this object is already in HPU memory and on the correct device, thenno copy is performed and the original object is returned.

Parameters:

• device (int) – The destination HPU id. Defaults to the current device.

• non_blocking (bool) – IfTrue and the source is in pinned memory,the copy will be asynchronous with respect to the host. Otherwise,the argument has no effect.

Return type:

_StorageBase |TypedStorage

int()[source]#

Casts this storage to int type.

propertyis_cuda#

propertyis_hpu#

is_pinned(device='cuda')[source]#

Determine whether the CPU storage is already pinned on device.

Parameters:

device (str ortorch.device) – The device to pin memory on (default:'cuda').This argument is discouraged and subject to deprecated.

Returns:

A boolean variable.

is_shared()#

is_sparse:bool=False#

is_sparse_csr:bool=False#

long()[source]#

Casts this storage to long type.

mps()[source]#

Return a MPS copy of this storage if it’s not already on the MPS.

nbytes()#

new()#

pin_memory(device='cuda')[source]#

Copy the CPU storage to pinned memory, if it’s not already pinned.

Parameters:

device (str ortorch.device) – The device to pin memory on (default:'cuda').This argument is discouraged and subject to deprecated.

Returns:

A pinned CPU storage.

resizable()#

resize_()#

share_memory_(*args,**kwargs)[source]#

Moves the storage to shared memory.

This is a no-op for storages already in shared memory and for CUDAstorages, which do not need to be moved for sharing across processes.Storages in shared memory cannot be resized.

Note that to mitigate issues likethisit is thread safe to call this function from multiple threads on the same object.It is NOT thread safe though to call any other function on self without propersynchronization. Please seeMultiprocessing best practices for more details.

Note

When all references to a storage in shared memory are deleted, the associated shared memoryobject will also be deleted. PyTorch has a special cleanup process to ensure that this happenseven if the current process exits unexpectedly.

It is worth noting the difference betweenshare_memory_() andfrom_file() withshared=True

1. share_memory_ usesshm_open(3) to create aPOSIX shared memory object whilefrom_file() usesopen(2) to open the filename passed by the user.

2. Both use anmmap(2) call withMAP_SHAREDto map the file/object into the current virtual address space

3. share_memory_ will callshm_unlink(3) on the object after mapping it to make sure the shared memoryobject is freed when no process has the object open.torch.from_file(shared=True) does not unlink thefile. This file is persistent and will remain until it is deleted by the user.

Returns:

self

short()[source]#

Casts this storage to short type.

size()[source]#

Return type:

int

to(*,device,non_blocking=False)[source]#

tolist()[source]#

Return a list containing the elements of this storage.

type(dtype=None,non_blocking=False)[source]#

Return type:

_StorageBase |TypedStorage

untyped()[source]#

Legacy Typed Storage#

torch.Storage is an alias for the storage class that corresponds withthe default data type (torch.get_default_dtype()). For example, if thedefault data type istorch.float,torch.Storage resolves totorch.FloatStorage.

Thetorch.<type>Storage andtorch.cuda.<type>Storage classes,liketorch.FloatStorage,torch.IntStorage, etc., are notactually ever instantiated. Calling their constructors createsatorch.TypedStorage with the appropriatetorch.dtype andtorch.device.torch.<type>Storage classes have all of thesame class methods thattorch.TypedStorage has.

Atorch.TypedStorage is a contiguous, one-dimensional array ofelements of a particulartorch.dtype. It can be given anytorch.dtype, and the internal data will be interpreted appropriately.torch.TypedStorage contains atorch.UntypedStorage whichholds the data as an untyped array of bytes.

Every stridedtorch.Tensor contains atorch.TypedStorage,which stores all of the data that thetorch.Tensor views.

classtorch.TypedStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

bfloat16()[source]#

Casts this storage to bfloat16 type.

bool()[source]#

Casts this storage to bool type.

byte()[source]#

Casts this storage to byte type.

char()[source]#

Casts this storage to char type.

clone()[source]#

Return a copy of this storage.

complex_double()[source]#

Casts this storage to complex double type.

complex_float()[source]#

Casts this storage to complex float type.

copy_(source,non_blocking=None)[source]#

cpu()[source]#

Return a CPU copy of this storage if it’s not already on the CPU.

cuda(device=None,non_blocking=False)[source]#

Returns a copy of this object in CUDA memory.

If this object is already in CUDA memory and on the correct device, thenno copy is performed and the original object is returned.

Parameters:

• device (int) – The destination GPU id. Defaults to the current device.

• non_blocking (bool) – IfTrue and the source is in pinned memory,the copy will be asynchronous with respect to the host. Otherwise,the argument has no effect.

Return type:

Self

data_ptr()[source]#

propertydevice#

double()[source]#

Casts this storage to double type.

dtype:dtype#

element_size()[source]#

propertyfilename:str|None#

Returns the file name associated with this storage if the storage was memory mapped from a file.orNone if the storage was not created by memory mapping a file.

fill_(value)[source]#

float()[source]#

Casts this storage to float type.

float8_e4m3fn()[source]#

Casts this storage to float8_e4m3fn type

float8_e4m3fnuz()[source]#

Casts this storage to float8_e4m3fnuz type

float8_e5m2()[source]#

Casts this storage to float8_e5m2 type

float8_e5m2fnuz()[source]#

Casts this storage to float8_e5m2fnuz type

classmethodfrom_buffer(*args,**kwargs)[source]#

classmethodfrom_file(filename,shared=False,size=0)→Storage[source]#

Creates a CPU storage backed by a memory-mapped file.

Ifshared isTrue, then memory is shared between all processes.All changes are written to the file. Ifshared isFalse, then the changes onthe storage do not affect the file.

size is the number of elements in the storage. Ifshared isFalse,then the file must contain at leastsize*sizeof(Type) bytes(Type is the type of storage). Ifshared isTrue the file will be created if needed.

Parameters:

• filename (str) – file name to map

• shared (bool) –

  whether to share memory (whetherMAP_SHARED orMAP_PRIVATE is passed to theunderlyingmmap(2) call)

• size (int) – number of elements in the storage

get_device()[source]#

Return type:

int

half()[source]#

Casts this storage to half type.

hpu(device=None,non_blocking=False)[source]#

Returns a copy of this object in HPU memory.

If this object is already in HPU memory and on the correct device, thenno copy is performed and the original object is returned.

Parameters:

• device (int) – The destination HPU id. Defaults to the current device.

• non_blocking (bool) – IfTrue and the source is in pinned memory,the copy will be asynchronous with respect to the host. Otherwise,the argument has no effect.

Return type:

Self

int()[source]#

Casts this storage to int type.

propertyis_cuda#

propertyis_hpu#

is_pinned(device='cuda')[source]#

Determine whether the CPU TypedStorage is already pinned on device.

Parameters:

device (str ortorch.device) – The device to pin memory on (default:'cuda').This argument is discouraged and subject to deprecated.

Returns:

A boolean variable.

is_shared()[source]#

is_sparse:bool=False#

long()[source]#

Casts this storage to long type.

nbytes()[source]#

pickle_storage_type()[source]#

pin_memory(device='cuda')[source]#

Copy the CPU TypedStorage to pinned memory, if it’s not already pinned.

Parameters:

device (str ortorch.device) – The device to pin memory on (default:'cuda').This argument is discouraged and subject to deprecated.

Returns:

A pinned CPU storage.

resizable()[source]#

resize_(size)[source]#

share_memory_()[source]#

Seetorch.UntypedStorage.share_memory_()

short()[source]#

Casts this storage to short type.

size()[source]#

to(*,device,non_blocking=False)[source]#

Returns a copy of this object in device memory.

If this object is already on the correct device, then no copy is performedand the original object is returned.

Parameters:

• device (int) – The destination device.

• non_blocking (bool) – IfTrue and the source is in pinned memory,the copy will be asynchronous with respect to the host. Otherwise,the argument has no effect.

Return type:

Self

tolist()[source]#

Return a list containing the elements of this storage.

type(dtype=None,non_blocking=False)[source]#

Returns the type ifdtype is not provided, else casts this object tothe specified type.

If this is already of the correct type, no copy is performed and theoriginal object is returned.

Parameters:

• dtype (type orstring) – The desired type

• non_blocking (bool) – IfTrue, and the source is in pinned memoryand destination is on the GPU or vice versa, the copy is performedasynchronously with respect to the host. Otherwise, the argumenthas no effect.

• **kwargs – For compatibility, may contain the keyasync in place ofthenon_blocking argument. Theasync arg is deprecated.

Return type:

_StorageBase |TypedStorage |str

untyped()[source]#

Return the internaltorch.UntypedStorage.

classtorch.DoubleStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.float64#

classtorch.FloatStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.float32#

classtorch.HalfStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.float16#

classtorch.LongStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.int64#

classtorch.IntStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.int32#

classtorch.ShortStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.int16#

classtorch.CharStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.int8#

classtorch.ByteStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.uint8#

classtorch.BoolStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.bool#

classtorch.BFloat16Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.bfloat16#

classtorch.ComplexDoubleStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.complex128#

classtorch.ComplexFloatStorage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.complex64#

classtorch.QUInt8Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.quint8#

classtorch.QInt8Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.qint8#

classtorch.QInt32Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.qint32#

classtorch.QUInt4x2Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.quint4x2#

classtorch.QUInt2x4Storage(*args,wrap_storage=None,dtype=None,device=None,_internal=False)[source]#

dtype:torch.dtype=torch.quint2x4#