Persistent Naming for Linux OS Block Devices

Issues and Causes

Phenomenon: On virtual machines with multiple disks, the order of volume labels may change after a reboot, potentially causing inconsistencies in file systems across various mount targets.

Reason: The Linux disk scanning mechanism does not guarantee a consistent mapping between volume labels and block devices. This mapping depends on factors like the disk driver loading order, the host’s PCI slot detection sequence, and the order of physical disk insertion. For example, a disk mounted before boot may take an earlier volume label if its driver loads and its PCI signal is detected first.

Solution

This test environment uses Baidu AI Cloud BCC running CentOS Linux 7.5.1804 and Cloud Disk Server storage.

Simple but not recommended UUID method

The challenge of using volume labels is that the mapping between disks and labels can change. To resolve this, we recommend bypassing volume labels and using UUIDs, which have a stable mapping to specific disks. This ensures the correct file system mounts after a reboot. It can be implemented in just one step:

Change the volume label entries in /etc/fstab from

1/dev/your-target-label   /your-mnt-dir   ext4    defaults    1 1

Replace with

1UUID=the-uuid-of-the-device  /your-mnt-dir   ext4    defaults    1 1

The UUID of the corresponding device can be obtained through the lsblk -f or blkid Command.

File system mounting works this way because a UUID serves as a link to the corresponding /dev/vd* device. This link is automatically maintained, ensuring the UUID consistently points to a fixed device even if volume labels change.

1# UUID is a link
2[root@instance-iqil7m0l ~] ls -l /dev/disk/by-uuid/
3 lrwxrwxrwx 1 root root  9 Dec. 18 11:33 49454dd5-1891-4d68-ba0a-e0be35e3c6cd -> ../../vdc
4 lrwxrwxrwx 1 root root  9 December 18 11:20 9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da -> ../../vdb
5 lrwxrwxrwx 1 root root 10 December 17 19:59 ce4f29f2-f1a9-4c31-95e6-5bc32a698d2b -> ../../vda1
6# After Reboot, we reversed the disk mount order, causing the previous vdc to shift forward as vdb, while the UUID still maintains correct referencing
7[root@instance-iqil7m0l ~] ls -l /dev/disk/by-uuid/
8 lrwxrwxrwx 1 root root  9 December 18 11:35 49454dd5-1891-4d68-ba0a-e0be35e3c6cd -> ../../vdb
9 lrwxrwxrwx 1 root root  9 December 18 12:24 9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da -> ../../vdc
10 lrwxrwxrwx 1 root root 10 December 18 11:36 AM ce4f29f2-f1a9-4c31-95e6-5bc32a698d2b -> ../../vda1

The UUID method is simple and effective but has two shortcomings:

• When a disk is created from a snapshot, its UUID remains identical to that of the source disk. If both disks are mounted to the host, the UUID points to the most recently attached disk, overriding the earlier one. For this reason, it is not possible to use UUIDs to simultaneously mount the file systems of both the source disk and its snapshot.

1# Use the current vdc to create a Snapshot, then generate a disk from this Snapshot and mount it to the host as vdd. The UUIDs of both are consistent.
2[root@instance-iqil7m0l ~] blkid
3/dev/vda1: UUID="ce4f29f2-f1a9-4c31-95e6-5bc32a698d2b" TYPE="ext4"
4/dev/vdb: UUID="49454dd5-1891-4d68-ba0a-e0be35e3c6cd" TYPE="ext4"
5/dev/vdc: UUID="9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da" TYPE="ext4"
6/dev/vdd: UUID="9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da" TYPE="ext4"
7# And the UUID link points to the later-mounted vdd (vdc is overwritten)
8[root@instance-iqil7m0l ~] ls -l /dev/disk/by-uuid/
9 lrwxrwxrwx 1 root root  9 December 18 11:35 49454dd5-1891-4d68-ba0a-e0be35e3c6cd -> ../../vdb
10 lrwxrwxrwx 1 root root  9 December 18 12:31 9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da -> ../../vdd
11 lrwxrwxrwx 1 root root 10 December 18 11:36 AM ce4f29f2-f1a9-4c31-95e6-5bc32a698d2b -> ../../vda1

• UUIDs can only be used with file systems, not raw devices. Since UUIDs are attributes of file systems, unformatted block devices do not possess UUIDs.

  Shell Copy

  1# After erasing vdc's file system, vdc loses its UUID.
  2[root@instance-iqil7m0l ~] lsblk -f
  3NAME   FSTYPE LABEL UUID                        MOUNTPOINT
  4vda
  5└─vda1 ext4         ce4f29f2-f1a9-4c31-95e6-5bc32a698d2b /
  6vdb    ext4         49454dd5-1891-4d68-ba0a-e0be35e3c6cd
  7vdc
  8vdd    ext4         9a97ecfb-98e1-4bc7-9d05-9ccbfa9a11da

The Disk ID method resolves most issues

Disk IDs perfectly address UUID-related issues. Disks derived from snapshots have unique Disk IDs distinct from their source disks, avoiding any conflict during mounting. For raw device operations, simply use the links within the /dev/disk/by-id/ directory. File system mount usage:

First, identify the Disk ID corresponding to the volume label:

1ls -l /dev/disk/by-id/ | grep your-target-label | cut -d " " -f 10

Change the volume label entries in /etc/fstab from

1/dev/your-target-label   /your-mnt-dir   ext4    defaults    1 1

Replace with

1/dev/disk/your-disk-id  /your-mnt-dir   ext4    defaults    1 1

Don’t want to remember long Disk IDs or UUIDs? No worries! Check out the udev solution below to bind a custom "volume label.\

The powerful tool udev

Using udev is more complex than the methods mentioned earlier and is not recommended unless you need advanced operations. udev can resolve almost every related issue. In fact, UUIDs and Disk IDs rely on the udev mechanism. By leveraging udev, you can establish a persistent naming system for block devices. The example below demonstrates how to bind a device to a custom "volume label.\

First, we need to identify the disk device to be bound. Here, the unique disk serial number is used, which can be obtained through the following Command:

1udevadm info --query=all --name=/dev/vdd | grep ID_SERIAL | cut -d "=" -f 2

Add a new rule under /etc/udev/rules.d/90-myblk.rules (the number indicates the execution order, with higher numbers being executed later). To ensure the system creates corresponding device files before our rule is applied, we chose 90 as a relatively high number. Rule content:

1ENV{ID_SERIAL}=="your-serial-number", SYMLINK="myblk_a" #If serial number matches, create corresponding link

After rebooting the host, we can reference the specified disk via /dev/myblk_a to perform raw disk operations, create a File System, or mount the File System, etc. This method ensures the persistent correspondence between disks and our custom volume labels.

1# Before reboot: vdc <-> 046b...  &   vdd <->  da0a...
2[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/vdc | grep ID_SERIAL
3E: ID_SERIAL=046b4230-760d-42db-8
4[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/vdd | grep ID_SERIAL
5E: ID_SERIAL=da0a787e-fb65-4e0b-b
6# Add rule files: myblk_a <-> 046b...  &  myblk_b <-> da0a...
7ENV{ID_SERIAL}=="046b4230-760d-42db-8", SYMLINK="myblk_a"
8ENV{ID_SERIAL}=="da0a787e-fb65-4e0b-b", SYMLINK="myblk_b"
9# After reboot, the correspondence between the disk and the system volume label changes:  vdc <-> da0a... & vdd <-> 046b...
10[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/vdc | grep ID_SERIAL
11E: ID_SERIAL=da0a787e-fb65-4e0b-b
12[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/vdd | grep ID_SERIAL
13E: ID_SERIAL=046b4230-760d-42db-8
14# However, the correspondence between disks and custom volume labels is correct:  myblk_a <-> 046b...  &  myblk_b <-> da0a...
15[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/myblk_a | grep ID_SERIAL
16E: ID_SERIAL=046b4230-760d-42db-8
17[root@instance-iqil7m0l ~] udevadm info --query=all --name=/dev/myblk_b | grep ID_SERIAL
18E: ID_SERIAL=da0a787e-fb65-4e0b-b

udev is a service provided jointly by the kernel and user-mode programs. Its mechanism works as follows: when a device connects to the system, the hot-plug device driver sends a notification to the user-mode program udevd through the kernel, carrying relevant information (e.g., the disk serial number). The user-mode program processes the notification by matching it against predefined rules (e.g., our 90-myblk.rules file) and executes actions on target messages (e.g., creating a “myblock_a” link, as defined).

Summary

Instabilities in the mapping between volume labels and devices are the root cause of these issues. To achieve persistent mappings, we suggest bypassing volume labels and using Disk IDs instead of UUIDs in a cloud environment. Finally, we

introduced the underlying mechanisms for UUIDs and Disk IDs—based on udev. While udev supports flexible and customizable persistent naming, its configuration is more complex than other methods.