Wednesday, March 11, 2015

Hot migrate LVM volume to new LUN(s)

This example hot-migrates an existing LVM volume spanned over 3 disks to a new LVM volume spanned over 3 disks.


  • lvm2 (apt-get install lvm2)
  • 3 disks to start with
  • 3 new disks to be added. Disks in this example are 100% identical!

Current LVM

This first part you probably already have, since you want to migrate this volume. But i’m going to create it anyway as part of the whole documentation.
I’m not going to work with partitions and just use the whole disks.

Create the Pysical Volumes

root@lvmtest:~# pvcreate /dev/sdb /dev/sdc /dev/sdd
  Physical volume "/dev/sdb" successfully created
  Physical volume "/dev/sdc" successfully created
  Physical volume "/dev/sdd" successfully created

Create the Volume Group

root@lvmtest:~# vgcreate MAIN /dev/sdb /dev/sdc /dev/sdd

Create the Logical Volume

root@lvmtest:~# lvcreate -n LVMAIN -l 100%FREE MAIN
  Logical volume "LVMAIN" created

Create the filesystem, mount it

root@lvmtest:~# mkfs.xfs /dev/MAIN/LVMAIN

root@lvmtest:~# mkdir /mnt/mylvmvolume

root@lvmtest:~# mount /dev/MAIN/LVMAIN /mnt/mylvmvolume

root@lvmtest:~# df -h | grep MAIN
/dev/mapper/MAIN-LVMAIN   24G   33M   24G   1% /mnt/mylvmvolume

Put some data on it

root@lvmtest:/mnt/mylvmvolume# dd if=/dev/zero of=blabla.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 5.93346 s, 177 MB/s

Add new disks and create the mirror

Add new disks to the machine.

Prepare the new disks:

root@lvmtest:~# pvcreate /dev/sde /dev/sdf /dev/sdg
  Physical volume "/dev/sde" successfully created
  Physical volume "/dev/sdf" successfully created
  Physical volume "/dev/sdg" successfully created

Add the disks to the existing Volume Group

root@lvmtest:~# vgextend MAIN /dev/sde /dev/sdf /dev/sdg

Create a mirror (-m1) of the current data to the recently added space.
Do this in a screen. This can take days, depending on the size!

root@lvmtest:~# lvconvert -m1 --corelog MAIN/LVMAIN /dev/sde /dev/sdf /dev/sdg
  MAIN/LVMAIN: Converted: 0.0%
  MAIN/LVMAIN: Converted: 2.8%
  MAIN/LVMAIN: Converted: 10.6%
  MAIN/LVMAIN: Converted: 20.2%
  MAIN/LVMAIN: Converted: 29.9%
  MAIN/LVMAIN: Converted: 39.1%
  MAIN/LVMAIN: Converted: 48.8%
  MAIN/LVMAIN: Converted: 58.3%
  MAIN/LVMAIN: Converted: 67.8%
  MAIN/LVMAIN: Converted: 77.5%
  MAIN/LVMAIN: Converted: 87.1%
  MAIN/LVMAIN: Converted: 96.8%
  MAIN/LVMAIN: Converted: 100.0%

The mirror is live.

During the conversion, you might see some nice figures using iostat

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
sdb             126.00         0.00        63.00          0        126
sdc               0.00         0.00         0.00          0          0
sdd               0.00         0.00         0.00          0          0
sde             126.00        63.00         0.00        126          0
sdg               0.00         0.00         0.00          0          0
sdf               0.00         0.00         0.00          0          0
sda               0.00         0.00         0.00          0          0
dm-0              0.00         0.00         0.00          0          0
dm-1           1004.00        62.75         0.00        125          0
dm-2           1008.00         0.00        63.00          0        126

Break the mirror and go live with the new disks

Create 0 copies (-m0) for the devices that will be removed, a.k.a. breaking the mirror.

root@lvmtest:~# lvconvert -m0 MAIN/LVMAIN /dev/sdb /dev/sdc /dev/sdd

Remove the devices from the Volume Group

root@lvmtest:~# vgreduce MAIN /dev/sdb /dev/sdc /dev/sdd
  Removed "/dev/sdb" from volume group "MAIN"
  Removed "/dev/sdc" from volume group "MAIN"
  Removed "/dev/sdd" from volume group "MAIN"

Remove the Physical Volumes

root@lvmtest:~# pvremove /dev/sdb /dev/sdc /dev/sdd
  Labels on physical volume "/dev/sdb" successfully wiped
  Labels on physical volume "/dev/sdc" successfully wiped
  Labels on physical volume "/dev/sdd" successfully wiped

That’s it.. Hot migrated!

root@lvmtest:~# df -h | grep MAIN
/dev/mapper/MAIN-LVMAIN   24G   11G   14G  42% /mnt/mylvmvolume