Device-mapper “unstriped” target

Introduction

The device-mapper “unstriped” target provides a transparent mechanism tounstripe a device-mapper “striped” target to access the underlying diskswithout having to touch the true backing block-device. It can also beused to unstripe a hardware RAID-0 to access backing disks.

Parameters:<number of stripes> <chunk size> <stripe #> <dev_path> <offset>

<number of stripes>
The number of stripes in the RAID 0.
<chunk size>
The amount of 512B sectors in the chunk striping.
<dev_path>
The block device you wish to unstripe.
<stripe #>
The stripe number within the device that corresponds to physicaldrive you wish to unstripe. This must be 0 indexed.

Why use this module?

An example of undoing an existing dm-stripe

This small bash script will setup 4 loop devices and use the existingstriped target to combine the 4 devices into one. It then will usethe unstriped target ontop of the striped device to access theindividual backing loop devices. We write data to the newly exposedunstriped devices and verify the data written matches the correctunderlying device on the striped array:

#!/bin/bashMEMBER_SIZE=$((128 * 1024 * 1024))NUM=4SEQ_END=$((${NUM}-1))CHUNK=256BS=4096RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512))DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}"COUNT=$((${MEMBER_SIZE} / ${BS}))for i in $(seq 0 ${SEQ_END}); do  dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct  losetup /dev/loop${i} member-${i}  DM_PARMS+=" /dev/loop${i} 0"doneecho $DM_PARMS | dmsetup create raid0for i in $(seq 0 ${SEQ_END}); do  echo "0 1 unstriped ${NUM} ${CHUNK} ${i} /dev/mapper/raid0 0" | dmsetup create set-${i}done;for i in $(seq 0 ${SEQ_END}); do  dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct  diff /dev/mapper/set-${i} member-${i}done;for i in $(seq 0 ${SEQ_END}); do  dmsetup remove set-${i}donedmsetup remove raid0for i in $(seq 0 ${SEQ_END}); do  losetup -d /dev/loop${i}  rm -f member-${i}done

Another example

Intel NVMe drives contain two cores on the physical device.Each core of the drive has segregated access to its LBA range.The current LBA model has a RAID 0 128k chunk on each core, resultingin a 256k stripe across the two cores:

 Core 0:       Core 1:__________    __________| LBA 512|    | LBA 768|| LBA 0  |    | LBA 256|----------    ----------

The purpose of this unstriping is to provide better QoS in noisyneighbor environments. When two partitions are created on theaggregate drive without this unstriping, reads on one partitioncan affect writes on another partition. This is because the partitionsare striped across the two cores. When we unstripe this hardware RAID 0and make partitions on each new exposed device the two partitions are nowphysically separated.

With the dm-unstriped target we’re able to segregate an fio script thathas read and write jobs that are independent of each other. Compared towhen we run the test on a combined drive with partitions, we were ableto get a 92% reduction in read latency using this device mapper target.

Example dmsetup usage

unstriped ontop of Intel NVMe device that has 2 cores

dmsetup create nvmset0 --table '0 512 unstriped 2 256 0 /dev/nvme0n1 0'dmsetup create nvmset1 --table '0 512 unstriped 2 256 1 /dev/nvme0n1 0'

There will now be two devices that expose Intel NVMe core 0 and 1respectively:

/dev/mapper/nvmset0/dev/mapper/nvmset1

unstriped ontop of striped with 4 drives using 128K chunk size

dmsetup create raid_disk0 --table '0 512 unstriped 4 256 0 /dev/mapper/striped 0'dmsetup create raid_disk1 --table '0 512 unstriped 4 256 1 /dev/mapper/striped 0'dmsetup create raid_disk2 --table '0 512 unstriped 4 256 2 /dev/mapper/striped 0'dmsetup create raid_disk3 --table '0 512 unstriped 4 256 3 /dev/mapper/striped 0'