Repository files navigation

• A memory allocator enables applications to transparently allocate data into a file system.

  • From applications, Metall looks like a normal heap allocator.

  • Applications can keep their data beyond single process life cycles and reattach the data in succeeding runs.

  • Leverages the memory-mapped file mechanism (i.e.,mmap(2)) tomap application data in files to the main memory.

• Employs the API developed byBoost.Interprocess.

  • Useful for allocating C++ data structures (including STL containers).

• Incorporates state-of-the-art allocation algorithms.

• Provides a space-efficient snapshot/versioning, leveraging the reflink copy mechanism in file systems.

• See details:Metall overview slides.

Metall consists of only header files and requires some header files in Boost C++ Libraries.

All core files exist undermetall/include/metall/.

• Boost C++ Libraries 1.80 or more.
  • Build is not required; needs only their header files.
• C++17 compiler
  • Tested with GCC 8.1 or more; however, 8.3 or more is recommended due to early implementation of the C++ Filesystem library.

To build your program with Metall, all you have to do is just setting include paths such as '-I' or CPLUS_INCLUDE_PATH.

For example,

# Download Boost (Boost C++ Libraries 1.80 or more is required)# One can skip this step if Boost is already available.wget https://boostorg.jfrog.io/artifactory/main/release/1.80.0/source/boost_1_80_0.tar.gztar xvf boost_1_80_0.tar.gzexport BOOST_ROOT=$PWD/boost_1_80_0git clone https://github.com/LLNL/metallexport METALL_INCLUDE=$PWD/metall/includeg++ -std=c++17 your_program.cpp -lstdc++fs -I${BOOST_ROOT} -I${METALL_INCLUDE}

Clang can be used instead of GCC to build Metall.However, we haven't tested it intensively.

Metall package is also available onSpack.

As Metall depends on Boost C++ Libraries,Spack also installs a proper version of Boost C++ Libraries automatically, if needed.

# Install Metall and Boost C++ Librariesspack install metall# Sets environment variables: BOOST_ROOT and METALL_ROOT.# Boost C++ Libraries and Metall are installed at the locations, respectively.spack load metall# Build a program that uses Metall# Please note that one has to put 'include' at the end of BOOST_ROOT and METALL_ROOTg++ -std=c++17 your_program.cpp -lstdc++fs -I${BOOST_ROOT}/include -I${METALL_ROOT}/include

Metall is also available onConan, thanks to theDICE research group.Conan Metall package information ishere.

To download and/or link Metall package from a CMake project,see example CMake files placedhere.

Metall repository contains some example programs underexample directory.One can use CMake to build the examples.For more details, see a pagehere.

• Full documentation
• API documentation

A Doxygen configuration file ishere.

To generate API document:

cd metallmkdir build_doccd build_docdoxygen ../docs/Doxyfile.in

Keita Iwabuchi, Karim Youssef, Kaushik Velusamy, Maya Gokhale, Roger Pearce,Metall: A persistent memory allocator for data-centric analytics,Parallel Computing, 2022, 102905, ISSN 0167-8191, https://doi.org/10.1016/j.parco.2022.102905.

• Parallel Computing (journal)

• arXiv (preprint)

• GitHub Issues is open.

• Primary contact:Keita Iwabuchi (LLNL).

Metall is distributed under the terms of both the MIT license and theApache License (Version 2.0). Users may choose either license, at theiroption.

All new contributions must be made under both the MIT and Apache-2.0licenses.

SeeLICENSE-MIT,LICENSE-APACHE,NOTICE, andCOPYRIGHT for details.

SPDX-License-Identifier: (Apache-2.0 OR MIT)

LLNL-CODE-768617