Repository files navigation

Author: Petros Koutoupis (petros@petroskoutoupis.com)

RapidDisk contains a set of advanced Linux RAM Drive and Caching kernelmodules. The user space utilities allow you to dynamically allocate RAMas block devices to either use them as stand alone drives or even mapthem as caching nodes to slower local (or remote) disk drives. The sameutilities provide users with the capability to export the same volumesacross an NVMe Target network.

Leverage a high speed RAM drive to add speed to a slower volume byutilizing the (Linux native) Device-Mapper framework. Enable a Write /Read-through or Write Around Least Recently Used or LRU (FIFO) cache.

This is where an application treats cache as the main data store andreads data from it and writes data to it. The cache is responsible forreading and writing this data to the permanent storage volume, therebyrelieving the application of this responsibility.

In this mode, all writes are cached to a RapidDisk RAM drive but are alsowritten to disk immediately. All disk reads are cached. Cache is not persistentover device removal, reboots, or after you remove the Device-Mapper mapping.This module does not store any cache metadata on RapidDisk volumes but insteadin memory outside of RapidDisk. You can map and unmap a cache drive to anyvolume at any time and it will not affect the integrity of the data on thepersistent storage drive.

Write Around caching shares some similarities with the Write-Throughimplementation. However, in this method, only read operations are cachedand not write operations. This way, all read data considered hot canremain in cache a bit longer before being evicted.

The RapidDisk Daemon (rapiddiskd) enabled remote management of RapidDiskvolumes. The management commands are simplified into a set of GET and POSTcommands. It operates over port 9118 by default. This can be changed wheninvoking the daemon with the use of a parameter. Either way, pleaseensure that the port is open for TCP within your firewall rules.

An example of a GET command:

#curl -s --output - 127.0.0.1:9118/v1/listRapidDiskVolumes|jq.{  "volumes": [    {      "rapiddisk": [        {          "device": "rd1",          "size": 67108864        },        {          "device": "rd0",          "size": 67108864        }      ]    },    {      "rapiddisk_cache": [        {          "device": "rc-wa_loop7",          "cache": "rd0",          "source": "loop7",          "mode": "write-around"        }      ]    }  ]}

An example of a POST command:

#curl -X POST -s 127.0.0.1:9118/v1/createRapidDisk/128|jq.{  "status": "Success"}

Change into the project's parent directory path.

To build the rapiddisk management utility, you will need to have thelibjansson,libpcre2,libdevmapper andlibmicrohttpddevelopment library files installed on your host system.

You are required to having either the full kernel source or the kernelheaders installed for your current kernel revision.

To build rapiddisk from source, you would type the following on the commandline:

#make

To install rapiddisk (must execute with superuser rights:sudo):

#make install

To uninstall rapiddisk (must execute with superuser rights:sudo):

#make uninstall

The rapiddisk utility will install in/sbin/

For utility information please reference the rapiddisk manual page:

#man 1 rapiddisk

Both modules are required to be loaded for the rapiddisk daemon to start.

To insert the rapiddisk module:

#modprobe rapiddisk

To remove the rapiddisk module:

#modprobe -r rapiddisk

To insert the rapiddisk-cache module:

#modprobe rapiddisk-cache

To remove the rapiddisk-cache module:

#modprobe -r rapiddisk-cache

Installing:

#make tools-install

Uninstalling:

#make tools-uninstall

#make dkms-install

Note - In dkms package versions 3.x and later, the REMAKE_INITRD option hasbeen deprecated. As a result, the operating system's initrd is not remade on everydkms rebuild of the module. If this is desired, then it will need to be manuallyrebuilt per your operating system's procedures. Otherwise, you can install and enablethe rapiddisk-on-boot functions which is supported on both Ubuntu and RHEL basedLinux distributions.

#make dkms-uninstall

Note - Please refer to the above note on initrd remakes during dkms rebuildsof the rapiddisk modules.

After installation, to start the service via systemd:

#systemctl start rapiddiskd.service

To check the status of the service via systemd:

#systemctl status rapiddiskd.service

To stop the service via systemd:

#systemctl stop rapiddiskd.service

To start the service at boot via systemd:

#systemctlenable rapiddiskd.service

There are a few things that need to be known when using the NVMe Targetfeatures of the RapidDisk suite.

In order to map any RapidDisk device and export it in the NVMe Targetframework, the nvmet and the nvmet-tcp or nvmet-rdma (or nvme-loop) kernelmodules must be inserted.

#modprobe nvmet nvmet-tcp

At least one Ethernet interface will need to be configured as a targetport to export the RapidDisk volume from.

#rapiddisk -i eth -P 1 -t tcp

When exporting a volume, a RapidDisk volume and a target port must bedefined. If a host NQN is not defined, the administration utility willprovide access to any host NQN. Note - a target can be exported across morethan one target port.

#rapiddisk -e -b rd3 -P 1

If a host NQN is defined, access is restricted to only those host NQNs.Note - the following command example can be repeated multiple times to addadditional host NQNs for the specified target export.

#rapiddisk -e -b rd3 -P 1 -H nqn.host1

Unexporting RapidDisk volumes looks a bit different than exporting. If ahost NQN is defined for a specified target, only that NQN will be removed fromaccessing the exported target.

#rapiddisk -x -b rd3 -H nqn.host1

Removing all allowed host NQNs will revert access to any and all host NQNsrequesting access to the target.

If a target port is defined, the exported target will not be exported from theinterface if one condition is met: the target has no defined allowed host NQNs.

#rapiddisk -x -b rd3 -P 1 -H nqn.host1

OR

#rapiddisk -x -b rd3 -P 1

And if there are no defined allowed host NQNs and the target is not being exportedacross any target ports, the entire target is removed from the subsystem.