CPU vs. I/O#

In order to minimize the overhead of context switches our default thread poolfor CPU-intensive tasks has a fixed size, defaulting to the process CPU affinity (on Linux) orstd::thread::hardware_concurrency.This means that CPU tasks should never block for long periods of time because thiswill result in under-utilization of the CPU. To achieve this we have a separatethread pool which should be used for tasks that need to block. Since these tasksare usually associated with I/O operations we call this the I/O thread pool. Thismodel is often associated with asynchronous computation.

The size of the I/O thread pool currently defaults to 8 threads and shouldbe sized according to the parallel capabilities of the I/O hardware. For example,if most reads and writes occur on a typical HDD then the default of 8 will probablybe sufficient. On the other hand, when most reads and writes occur on a remotefilesystem such as S3, it is often possible to benefit from many concurrent readsand it may be possible to increase I/O performance by increasing the size of theI/O thread pool. The size of the default I/O thread pool can be managed withtheARROW_IO_THREADS environment variable orwith thearrow::io::SetIOThreadPoolCapacity() function.

Increasing the size of the CPU thread pool is not likely to have any benefit. Insome cases it may make sense to decrease the size of the CPU thread pool in orderto reduce the impact that Arrow C++ has on hardware shared with other processes or userthreads. The size of the default CPU thread pool can be managed with theOMP_NUM_THREADS environment variable or with thearrow::SetCpuThreadPoolCapacity() function.

Serial Execution#

Operations in Arrow C++ that may use threads can usually be configured to run seriallyvia some kind of parameter. In this case we typically replace the CPU executor withan event loop operated by the calling thread. However, many operations will continueto use the I/O thread pool. This means that some parallelism may still occur even whenserial execution is requested.

Jemalloc Background Threads#

When using thejemalloc allocator a small number ofbackground threads will be created by jemalloc to manage the pool. These threadsshould have minimal impact but can show up as a memory leak when running analysistools like Valgrind. This is harmless and can be safely suppressed or Arrow C++ canbe compiled without jemalloc.

Future#

Arrow C++ usesarrow::Future to communicate results between threads. Typicallyanarrow::Future will be created when an operation needs to perform some kindof long running task that will block for some period of time.arrow::Futureobjects are mainly meant for internal use and any method that returns anarrow::Future will usually have a synchronous variant as well.