#6 Linux LVM, RAID, caching 2015

2015-12-04, by René Rebe

In the initial LVM/RAID notes a very long time ago (in a galaxy far far away) we used the classic Linux Multiple Device software RAID setup.

However, in the meantime it is possible to setup some RAID levels directly within the LVM management to simplify and streamline the experience:


Nowadays, we can intead let LVM manage all this by simply adding all single devices as physical volumes and combine them to a volume-group:

pvcreate /dev/sd*9
vgcreate vg0 /dev/sd*9

What is new is, that we can now set up the RAID directly inside the LVM:

lvcreate --type raid1 -L 200G -n lv-mirror vg0

Of course various RAID levels are avilable too (but may require additional parametners). This not only simplifies the setup, it also makes the whole setup easier to manage, and to easily expand in the future.

Additional you can combine this with all the latest and greatest thin-pools, snapshots, and whatnot that may be great options for virutalization.

Adding SSD caching

One of the greatest and latest options is adding a cache to the logical volume, usually to improve the performance of large spinning disks. Especially seek times and thus operations per second are much higher for solid state storage. So by adding small, high performance SSD you can now enable caching for your LVM setup:

vgextend vg0 /dev/sdc9
lvcreate --type cache -L 20G -n lv-mirror-cache vg0/lv-mirror /dev/sdc9

and you should get a confirmation like:

Logical volume vg0/lv-mirror is now cached.

Behind the scene this command will create two volumes for the cache- and meta-data. However, latest lvm tools automatically manage this complexicty for you.

You can also ask for some basic statics about the utilization and hits vs. misses, like:

lvs -o lv_name,cache_read_hits,cache_read_misses,cache_write_hits,cache_write_misses

Added bonus: Crypt the whole thing!1!

There are many reasons why you may want to add some crpyto into the mix. May it be simply your accouning and tax backup, your trade secret, running on an external root server, or all your business and prive emails and documents. With Linux, device mapper and all it's infrastrucutre it is as easy as:

cryptsetup luksFormat /dev/sda9
cryptsetup luksOpen /dev/sda9 crypt0
pvcreate /dev/mapper/crypto

and then setting everything up on top of /dev/mapper/crypto.

More notes

Create RAID5 with specifc devices, e.g. no cache ssd:

lvcreate --type raid5 -L 1T -i 2 -n bigdata vg0 /dev/sda2 /dev/sdb2 /dev/sdd2

Extend just created RAID5 volume to 2TB:

lvextend vg0/bigdata -L 2T

Followed by resizing the underlying file-system by something like:

btrfs filesystem resize max /mnt/bigdata

Uncaching (removing) a cache

lvconvert --uncache vg0/lv
# or: lvremove vg0/lv-cache

External links

The Author

René Rebe studied computer science and digital media science at the University of Applied Sciences of Berlin, Germany. He is the founder of the T2 Linux SDE (System Development Environment), and contributer to various projects in the open source landscape for more than 10 years, now. He also founded the Berlin-based software company ExactCODE GmbH. A company dedicated to exact software solutions that just work, everyday.