Memory Limits in R

Description

R holds objects it is using in virtual memory. This help filedocuments the current design limitations on large objects: thesediffer between 32-bit and 64-bit builds ofR.

Details

CurrentlyR runs on 32- and 64-bit operating systems, and most 64-bitOSes (including Linux, Solaris, Windows and macOS) can run either32- or 64-bit builds ofR. The memory limits depends mainly on thebuild, but for a 32-bit build ofR on Windows they also depend on theunderlying OS version.

R holds all objects in virtual memory, and there are limits based on theamount of memory that can be used by all objects:

• There may be limits on the size of the heap and the number ofcons cells allowed – seeMemory – but these areusually not imposed.

• There is a limit on the (user) address space of a singleprocess such as theR executable. This is system-specific, and candepend on the executable.

• The environment may impose limitations on the resourcesavailable to a single process: Windows' versions ofR do so directly.

Error messages beginningcannot allocate vector of sizeindicate a failure to obtain memory, either because the size exceededthe address-space limit for a process or, more likely, because thesystem was unable to provide the memory. Note that on a 32-bit buildthere may well be enough free memory available, but not a large enoughcontiguous block of address space into which to map it.

There are also limits on individual objects. The storage spacecannot exceed the address limit, and if you try to exceed that limit,the error message beginscannot allocate vector of length.The number of bytes in a character string is limited to2^31 - 1 ~ 2*10^9,which is also the limit on each dimension of an array.

Unix

The address-space limit is system-specific: 32-bit OSesimposes a limit of no more than 4Gb: it is often 3Gb. Running32-bit executables on a 64-bit OS will have similar limits: 64-bitexecutables will have an essentially infinite system-specific limit(e.g., 128Tb for Linux on x86_64 cpus).

See the OS/shell's help on commands such aslimit orulimit for how to impose limitations on the resources availableto a single process. For example abash user could use

ulimit -t 600 -v 4000000

whereas acsh user might use

limit cputime 10mlimit vmemoryuse 4096m

to limit a process to 10 minutes of CPU time and (around) 4Gb ofvirtual memory. (There are other options to set the RAM in use, but theyare not generally honoured.)

Windows

The address-space limit is 2Gb under 32-bit Windows unless the OS'sdefault has been changed to allow more (up to 3Gb). Seehttps://docs.microsoft.com/en-gb/windows/desktop/Memory/physical-address-extensionandhttps://docs.microsoft.com/en-gb/windows/desktop/Memory/4-gigabyte-tuning.Under most 64-bit versions of Windows the limit for a 32-bit buildofR is 4Gb: for the oldest ones it is 2Gb. The limit for a 64-bitbuild ofR (imposed by the OS) is 8Tb.

It is not normally possible to allocate as much as 2Gb to a singlevector in a 32-bit build ofR even on 64-bit Windows because ofpreallocations by Windows in the middle of the address space.

Under Windows,R imposes limits on the total memory allocationavailable to a single session as the OS provides no way to do so: seememory.size andmemory.limit.

See Also

object.size(a) for the (approximate) size ofR objecta.