CTL

Contents

CTL#

UMF’s CTL is a mechanism for advanced configuration and control of UMF poolsand providers. It allows programmatic access to provider- or pool-specificconfiguration options, statistics and auxiliary APIs. CTL entries can also beset through environment variables or a configuration file, allowing adjustmentof UMF behavior without modifying the program.

Note

The CTL API is experimental and may change in future releases.

Main concepts#

The core concept is apath. A path is a string of nodes separated by periods.You can imagine nodes as directories where the last element is a file that canbe read, written or executed (similar tosysfs but with periods instead ofslashes). Example pathumf.logger.level controls the log level. You canaccess it with:

intlevel;umf_result_tret=umfCtlGet("umf.logger.level",&level,sizeof(level));

To change the level programmatically use:

intlevel=LOG_WARNING;umf_result_tret=umfCtlSet("umf.logger.level",&level,sizeof(level));

Accessing pool or provider paths is slightly more involved. For example:

size_talloc_count;umf_memory_pool_handle_thPool=createPool();umf_result_tret=umfCtlGet("umf.pool.by_handle.{}.stats.alloc_count",&alloc_count,sizeof(alloc_count),hPool);

Theumf.pool.by_handle prefix selects a pool addressed by its handle.Every{} in the path is replaced with an extra argument passed to the CTLfunction. Alternative addressing methods are described below.

Pool / Provider addressing#

Two addressing schemes are provided:by_handle andby_name. Each pooland provider has a unique handle and an optional user-defined name that can bequeried withumfMemoryProviderGetName() orumfMemoryPoolGetName().When usingby_name the name appears in the path, e.g.:

umfCtlGet("umf.pool.by_name.myPool.stats.alloc_count",&alloc_count,sizeof(alloc_count));

If multiple pools share a name, read operations must disambiguate the target byappending an index after the name:

umfCtlGet("umf.pool.by_name.myPool.0.stats.alloc_count",&alloc_count,sizeof(alloc_count));

The number of pools with a given name can be obtained with thecount node.

Wildcards#

A{} in the path acts as a wildcard and is replaced with successivearguments ofumfCtlGet,umfCtlSet orumfCtlExec. Wildcards canreplace any node, not only handles. For example:

size_tpool_count;constchar*name="myPool";umfCtlGet("umf.pool.by_name.{}.count",&pool_count,sizeof(pool_count),name);for(size_ti=0;i<pool_count;i++){umfCtlGet("umf.pool.by_name.{}.{}.stats.alloc_count",&alloc_count,sizeof(alloc_count),name,i);}

Ensure that the types of wildcard arguments match the expected node types.

Default addressing#

umf.provider.default andumf.pool.default store default values appliedto providers or pools created after the defaults are set. For example:

size_tcapacity=16;umfCtlSet("umf.pool.default.disjoint.params.capacity",&capacity,sizeof(capacity));

Every subsequently created disjoint pool will use16 as its startingcapacity overriding its creation parameters. Defaults are keyed by thename returned from the provider or poolget_name callback, so if pool/providerhas custom name it must be addressed explicitly. Defaults may be supplied programmaticallyor via environment variable and are saved internally and applied during initialization of amatching provider or pool.

Environment variables#

CTL entries may also be specified in theUMF_CONF environment variable ora configuration file specified in theUMF_CONF_FILE.Multiple entries are separated with semicolons, e.g.:

UMF_CONF="umf.logger.output=stdout;umf.logger.level=0"

CTL options available through environment variables are limited — you can onlytarget default nodes when addressing pools. This means that configurationstrings can influence values consumed during pool creation but cannot alterruntime-only parameters.

CTL nodes#

The CTL hierarchy is rooted atumf. The next component selects one of themajor subsystems:

  • umf.logger – logging configuration and diagnostics.

  • umf.provider – provider-specific parameters, statistics and commands.

  • umf.pool – pool-specific parameters, statistics and inspection helpers.

Within each subsystem the path continues with an addressing scheme followed bythe module or leaf of interest.

Reading below sections#

Parameter annotations describe the values stored in the node rather than thepointer types passed toumfCtlGet/umfCtlSet/umfCtlExec. TheAccess field indicates whether the node can be read, written, or executed.TheDefaults / Env field notes whether the entry can be controlled throughdefaults written underumf.provider.default.<name> orumf.pool.default.<name> and viaUMF_CONF/UMF_CONF_FILE. Nodes that donot accept either configuration source are marked as not supported.

Logger nodes#

umf.logger.timestamp(enabled)#
Parameters:

enabled (int) – Receives or supplies0 when timestamps are disabled and1 when they are emitted.

Access: read-write.Defaults / Env: supported.

Toggle timestamp prefixes in future log records. Logging starts withtimestamps disabled, and the flag affects only messages emitted after thechange.

umf.logger.pid(enabled)#
Parameters:

enabled (int) – Receives or supplies0 to omit the process identifier and1 to include it in every message header.

Access: read-write.Defaults / Env: supported.

Controls whether each log line is annotated with the current process id.Logging omits the pid by default. Setting non-boolean values results incoercion to zero/non-zero; the change applies to subsequent messages only.

umf.logger.level(level)#
Parameters:

level (int (0 ..4)) – Receives or supplies the minimum severity that will be written.

Access: read-write.Defaults / Env: supported.

Sets the filtering threshold for the logger. Records below the configuredlevel are dropped. Writes that fall outside the enumerated range arerejected. 0 means debug logs, 1 means info logs, 2 means warnings, 3 meanserrors, and 4 means fatal logs. Until an output is selected the loggerignores the level because logging is disabled.

umf.logger.flush_level(level)#
Parameters:

level (int (0 ..4)) – Receives or supplies the severity at which the logger forces aflush of the output stream.

Access: read-write.Defaults / Env: supported.

Adjusts when buffered log data is synchronously flushed. Writes outside thevalid severity range fail, and lowering the level can incur additional flushoverhead for future messages. With logging disabled no flushing occurs.

umf.logger.output(path)#
Parameters:

path (char* when reading,constchar* when writing) – Receives the currently selected sink on reads. On writes, pass"stdout" or"stderr" to redirect to standard streams, aNULL-terminated file path to append to a file, orNULL to disablelogging altogether.

Access: read-write.Defaults / Env: supported.

Controls the destination for log messages. The logger closes any previouslyopened file when switching targets. Providing a path longer than 256 bytes orpointing to a file that cannot be opened causes the write to fail. Specialvalues"stdout" and"stderr" redirect output to the correspondingstreams. PassingNULL disables logging entirely, which is also theinitial state until a path is provided.

Provider nodes#

Provider entries are organized beneathumf.provider. Useumf.provider.by_handle.{provider} with aumf_memory_provider_handle_t argument to reach a specific provider.Providers can also be addressed by name throughumf.provider.by_name.{provider};append.{index} to address specific provider when multiple providers share the same label.Defaults for future providers reside underumf.provider.default.{provider} where{provider} isa name returned by each provider’sget_name implementation. Providers have theirdefault names (OS,FILE,DEVDAX,FIXED,CUDA orLEVEL_ZERO),unless their name was changed during creation, those renamed providers must be addressed explicitly.Defaults can be written viaumf.provider.default.<name> either programmatically or throughconfiguration strings. The entries below list only the suffix of each node;prefix them with the appropriateumf.provider path.

Common provider statistics#

.stats.allocated_memory(bytes)#

Accessible through bothumf.provider.by_handle.{provider} andumf.provider.by_name.{name}. Supply the provider handle or name (with anoptional.{index} suffix for duplicates) as the first wildcard argument.

Parameters:

bytes (size_t) – Receives the total number of bytes currently outstanding.

Access: read-only.Defaults / Env: not supported.

Returns the amount of memory the provider has allocated but not yet freed.The counter updates atomically as the provider serves requests and is notresettable.

.stats.peak_memory(bytes)#

Available viaumf.provider.by_handle.{provider} orumf.provider.by_name.{name}. Pass the provider selector as the firstwildcard argument.

Parameters:

bytes (size_t) – Receives the highest observed outstanding allocation size sincethe last reset.

Access: read-only.Defaults / Env: not supported.

Reports the historical maximum allocation footprint of the provider. Combinewithstats.peak_memory.reset() to discard stale peaks whendesired.

.stats.peak_memory.reset()#

Invoke throughumf.provider.by_handle.{provider} orumf.provider.by_name.{name} after supplying the provider selector as thefirst wildcard argument.

Access: execute.Defaults / Env: not supported.

Resets the peak allocation counter to the provider’s current outstandingusage. The operation does not affect other statistics and can be invoked atany time.

OS memory provider (OS)#

The OS provider supports the common statistics nodes described above and addsthe following parameter entry.

.params.ipc_enabled(enabled)#
Parameters:

enabled (int) – Receives0 when inter-process sharing is disabled and anon-zero value when it is active.

Access: read-only.Defaults / Env: not supported.

Indicates whether the OS memory provider has been initialized with IPCsupport. The value is fixed at provider creation time and cannot be modifiedafterwards.

Fixed memory provider (FIXED)#

The fixed-size allocation provider currently exposes only the common statisticsnodes.

DevDAX memory provider (DEVDAX)#

The DevDAX provider exposes the common statistics nodes described earlier.

File memory provider (FILE)#

The file-backed provider exposes the common statistics nodes.

CUDA memory provider (CUDA)#

The CUDA provider currently exposes only the common statistics nodes.

Level Zero memory provider (LEVEL_ZERO)#

The Level Zero provider supports the common statistics nodes described above andadds the following parameter entry.

.params.use_import_export_for_IPC(policy)#
Parameters:

policy (int) – Receives or supplies0 to use IPC API for memory sharingand1 to use import/export mechanism for memory sharing.

Access: read-write.Defaults / Env: Supported.

Controls the memory exchange policy for inter-process communicationoperations. When set to0 (default), the provider uses the IPC APIfor memory sharing between processes. When set to1, the provider usesthe import/export mechanism for memory sharing. This option is supportedonly on Windows with the Level Zero provider, where the default IPC mechanismdoes not work. Note that enabling import/export adds overhead duringallocation and deallocation for all allocations on the current provider.

Pool nodes#

Pool entries mirror the provider layout.umf.pool.by_handle.{pool} accepts aumf_memory_pool_handle_t, whileumf.pool.by_name.{pool} addressespools by name with an optional.{index} suffix when names are reused.Defaults for future pools reside underumf.pool.default.{pool} and track thename returned by each pool’sget_name implementation. Pools that keep theirdefault names (disjoint,scalable andjemalloc) continue to matchthose entries, while renamed pools must be addressed explicitly. Defaults can bewritten viaumf.pool.default.<pool> either programmatically or throughconfiguration strings. The entries below list only the suffix of each node;prefix them with the appropriateumf.pool path.

Common pool statistics#

.stats.alloc_count(count)#
Parameters:

count (size_t) – Receives the number of live allocations tracked by the pool.

Access: read-only.Defaults / Env: not supported.

Counts the allocations currently outstanding according to the pool’s publicallocation API. The value increments on successful allocations anddecrements when memory is released.

Disjoint pool (disjoint)#

.params.slab_min_size(bytes)#
Parameters:

bytes (size_t) – Receives or supplies the minimum slab size requested from theprovider.

Access: read-write. (write is only available through defaults)Defaults / Env: supported.

Governs how much memory the pool grabs in each slab. Lower values reduceper-allocation slack while higher values amortize provider overhead. Writesare accepted only before the pool completes itspost_initialize phase.

.params.max_poolable_size(bytes)#
Parameters:

bytes (size_t) – Receives or supplies the largest allocation size that is stillcached by the pool.

Access: read-write. (write is only available through defaults)Defaults / Env: supported.

Sets the cut-off for pooling allocations. Requests larger than this value aredelegated directly to the provider. Updates must occur beforepost_initialize completes.

.params.capacity(count)#
Parameters:

count (size_t) – Receives or supplies the maximum number of slabs each bucketmay retain.

Access: read-write. (write is only available through defaults)Defaults / Env: supported.

Caps the pool’s cached slabs per bucket to limit memory retention. Shrinkingthe capacity may cause future frees to return slabs to the provider. Writesare rejected afterpost_initialize.

.params.min_bucket_size(bytes)#
Parameters:

bytes (size_t) – Receives or supplies the minimal allocation size a bucket mayserve.

Access: read-write. (write is only available through defaults)Defaults / Env: supported.

Controls the smallest chunk size kept in the pool, which in turn affects thenumber of buckets. Writes are validated for size correctness and disallowedafterpost_initialize.

.params.pool_trace(level)#
Parameters:

level (int (0 disables tracing)) – Receives or supplies the tracing level for the pool.

Access: read-write. (write is only available through defaults)Defaults / Env: supported.

Controls the disjoint pool’s tracing features.0 disables tracing.1 records slab usage totals exposed through the.stats.curr_slabs_*and.stats.max_slabs_* nodes.2 additionally tracks allocation andfree counters and prints a usage summary when the pool is destroyed. Valuesgreater than2 also emit debug logs for every allocation and free.Tracing must be activated beforepost_initialize; attempting to change itlater fails withUMF_RESULT_ERROR_NOT_SUPPORTED.

.stats.used_memory(bytes)#

Available underumf.pool.by_handle.disjoint andumf.pool.by_name.disjoint. Provide the pool selector as the first wildcardargument.

Parameters:

bytes (size_t) – Receives the amount of memory that is presently allocated bythe pool’s clients.

Access: read-only.Defaults / Env: not supported.

Reports the memory currently in use across all slabs by active allocations.Available even whenpool_trace is disabled.

.stats.reserved_memory(bytes)#
Parameters:

bytes (size_t) – Receives the total number of bytes reserved in slabs that thepool owns.

Access: read-only.Defaults / Env: not supported.

Returns the total slab capacity reserved by the pool, including cached freespace. Available even whenpool_trace is disabled.

.stats.alloc_num(count)#
Parameters:

count (size_t) – Receives the number of allocations the pool has issued.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Counts everyallocation handed out by the pool since it was created.

.stats.alloc_pool_num(count)#
Parameters:

count (size_t) – Receives the number of allocations served directly from cachedslabs.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Countsallocations served from cached slabs without visiting the provider.

.stats.free_num(count)#
Parameters:

count (size_t) – Receives the total number of frees processed by the pool.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Tracks thenumber of frees observed by the pool since its creation.

.stats.curr_slabs_in_use(count)#
Parameters:

count (size_t) – Receives the current number of slabs actively servingallocations.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Returns the number ofslabs that currently have live allocations.

.stats.curr_slabs_in_pool(count)#
Parameters:

count (size_t) – Receives how many slabs are cached and ready for reuse.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Reports the slabsretained in the pool for future reuse.

.stats.max_slabs_in_use(count)#
Parameters:

count (size_t) – Receives the historical maximum of simultaneously used slabs.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Provides the peaknumber of slabs that were in use at the same time.

.stats.max_slabs_in_pool(count)#
Parameters:

count (size_t) – Receives the largest number of slabs retained in the cache.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Returns the highestnumber of slabs ever retained in the cache simultaneously.

.buckets.count(count)#
Parameters:

count (size_t) – Receives the number of distinct bucket sizes.

Access: read-only.Defaults / Env: not supported.

Returns the total number of buckets in the pool.

.buckets.{id}.size(bytes)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

bytes (size_t) – Receives the allocation size that the bucket serves.

Access: read-only.Defaults / Env: not supported.

Reports the allocation size serviced by the selected bucket. This value isavailable even when tracing is disabled.

.buckets.{id}.stats.alloc_num(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the number of allocations performed by this bucket.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Counts everyallocation that passed through the specified bucket.

.buckets.{id}.stats.alloc_pool_num(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the number of allocations satisfied from cached slabsin this bucket.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Counts how manyallocations were served entirely from the bucket’s cached slabs.

.buckets.{id}.stats.free_num(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the number of frees recorded for this bucket.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace set to2 or higher. Tracks thenumber of frees observed for the bucket.

.buckets.{id}.stats.curr_slabs_in_use(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives how many slabs for this bucket currently serveallocations.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Returns the currentslab utilization for the bucket.

.buckets.{id}.stats.curr_slabs_in_pool(count)

Available throughumf.pool.by_handle.disjoint andumf.pool.by_name.disjoint. Provide the pool selector and bucket index asthe first two wildcard arguments.{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the number of slabs cached and immediately availablefor this bucket.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Reports cached slabsthat the bucket can reuse without a provider call.

.buckets.{id}.stats.max_slabs_in_use(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the peak number of slabs in use for this bucket.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Provides thehistorical maximum of slabs simultaneously in use for the bucket.

.buckets.{id}.stats.max_slabs_in_pool(count)

{id} denotes a bucket index of typesize_t. Valid indices range from0 to.buckets.count-1.

Parameters:

count (size_t) – Receives the largest number of slabs retained in the bucket’scache.

Access: read-only.Defaults / Env: not supported.

Requires tracing withpool_trace of at least1. Returns the maximumnumber of slabs cached for later use by the bucket.

Scalable pool (scalable)#

The scalable pool currently exposes only the common statistics nodes.

Jemalloc pool (jemalloc)#

The jemalloc-backed pool currently exposes only the common statistics nodes.

Adding CTL support to custom providers and pools#

Theexamples/ctl/custom_ctl.c source demonstrates how a minimalprovider can expose configuration entries, statistics and runnables through theCTL API. To add similar support to your own provider or pool you must implementanext_ctl callback – parse incoming CTL paths and handleCTL_QUERY_READ,CTL_QUERY_WRITE andCTL_QUERY_RUNNABLE requests.The callback receives aumf_ctl_query_source_t indicating whether thequery came from the application or a configuration source. Programmaticcalls pass typed binary data, while configuration sources deliver stringsthat must be parsed. Wildcards ({}) may appear in paths and are suppliedas additional arguments.

During initialization UMF will executepost_initialize on the callback afterapplying any queued defaults, allowing the provider or pool to finalize itsstate before it is used by the application. The example converts wildcardedpaths intoprintf-style format strings with%s and usesvsnprintf toresolve the extra arguments. It also shows a helper that accepts integers fromeither source, printing the final values frompost_initialize.

Building and running the example:

cmake-Bbuildcmake--buildbuild./build/examples/umf_example_ctl

An optional modulus can be supplied via the environment:

UMF_CONF="umf.provider.default.ctl.m=10"./build/examples/umf_example_ctl
Contents