How to add a VDO disk to a virtual machine to add storage while taking advantage of transparent compression and deduplication

Shit, one of the VMs in my homelab has just crashed because of a full root filesystem. Apparently Nessus Essentials is generating reports much faster than I expected :)

To fix it I will add a new disk to move the /opt directory (or part of it). But I will use the new Red Hat Virtual Data Optimizer (VDO) to take advantage of transparent compression and deduplication.

So first of all, create a new disk and connect it to the VM. The way to do this will of course depend on the virtualization product you are using.

The important thing is you will end up with a new disk on your vm:

# dmesg | tail
[...]
[89575.706450] scsi 0:0:0:1: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[89575.710622] sd 0:0:0:1: Power-on or device reset occurred
[89575.710950] sd 0:0:0:1: Attached scsi generic sg2 type 0
[89575.713614] sd 0:0:0:1: [sdb] 33554432 512-byte logical blocks: (17.2 GB/16.0 GiB)
[89575.713719] sd 0:0:0:1: [sdb] Write Protect is off
[89575.713720] sd 0:0:0:1: [sdb] Mode Sense: 63 00 00 08
[89575.713869] sd 0:0:0:1: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[89575.723197] sd 0:0:0:1: [sdb] Attached SCSI disk

vdo is not installed by default in a minimal CentOS 8 installation, so we must run:

yum -y install vdo kmod-kvdo

Now we can create the vdo volume where /opt will live:

# vdo create --vdoLogicalSize=32G -n vdo_opt --device=/dev/sdb
Creating VDO vdo_opt
Starting VDO vdo_opt
Starting compression on VDO vdo_opt
VDO instance 0 volume is ready at /dev/mapper/vdo_opt

Create an XFS filesystem on it (don’t forget to add the -K argument):

mkfs.xfs -K /dev/mapper/vdo_opt

So far so good. Now we have to move the data. Before we start, we have to make sure that no files in the directory we want to move are in use. In this case we just have to stop nessusd.service.

So we will start renaming the /opt directory to /opt2 or whatever and create a new /opt where the vdo volume will be mounted:

cd /
mv /opt /opt2
mkdir /opt
mount /dev/mapper/vdo_opt /opt
mv /opt2/* /opt/

Depending on the data to be moved, it will take some time… If everything goes as expected, once the command finishes we will be able to delete the old /opt2 directory:

rmdir /opt2

In case of error, there are probably some hidden files or files in use. You will have to troubleshoot on your own.

# Mounting the VDO disk

You probably noted that we mounted the vdo manually without creating an /etc/fstab entry. And we won’t.

We will let systemd take care of the mount. As you probably know, systemd units are not only traditional services, but can be mounts as well.

When we installed the requiered vdo packages, some examples were added to /usr/share/doc/vdo/examples. We will use one of them:

cp /usr/share/doc/vdo/examples/systemd/VDO.mount.example /etc/systemd/system/opt.mount

Edit vdo-opt.mount to our needs:

[Unit]
Description = Mount filesystem that lives on VDO
name = opt.mount
Requires = vdo.service systemd-remount-fs.service
After = multi-user.target
Conflicts = umount.target
 
[Mount]
What = /dev/mapper/vdo_opt
Where = /opt
Type = xfs
Options = discard

[Install]
WantedBy = multi-user.target

WARNING: to be able to do this, mount unit name (opt) must match mount point name (/opt).

Enable and start:

# systemctl enable --now opt.mount
# df
Filesystem                 1K-blocks    Used Available Use% Mounted on
devtmpfs                      920060       0    920060   0% /dev
tmpfs                         936732       0    936732   0% /dev/shm
tmpfs                         936732    8692    928040   1% /run
tmpfs                         936732       0    936732   0% /sys/fs/cgroup
/dev/mapper/vg_system-root  10475520 1633256   8842264  16% /
/dev/mapper/vg_system-home   1038336  126544    911792  13% /home
/dev/mapper/vg_system-tmp    2086912   47780   2039132   3% /tmp
/dev/sda1                     999320  265636    664872  29% /boot
tmpfs                         187344       0    187344   0% /run/user/1000
/dev/mapper/vdo_opt         33538048 8992684  24545364  27% /opt

How long will it take until the filesystem is full again? Don’t forget to watch the stats carefully:

vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo_opt      16.0G      8.8G      7.2G  55%           42%

## References

A look at VDO, the new Linux compression layer