Handige Knakker

When creating full images from one of my rootdisks …

dd if=/dev/sda | bzip2 >/opt/backup/sda.img.bzip2

… i noticed the backups were growing, but the amount of data on the device was not.

Since dd is a full blocklevel- and not a filebased backup, there must be some free space containing old bits and bytes.
The sfill utility can overwrite the freespace with zeroes, giving me better compressed images.

sfill -f -l -l -z /mnt/mountpoint

My script to clean up some stuff.
Seems that those kernel header packages are eating up all inodes on small ext volumes.

#!/bin/sh
nr_of_removed_packages=`dpkg -l | egrep "^rc" | cut -d" " -f3 | wc -l`
nr_of_active_kernels=`ls /boot/vmlinuz* | wc -l`
active_kernels=`ls /boot/vmlinuz* | cut -d" " -f9 | sed -r 's/\/boot\/vmlinuz-//' | sed -r 's/-generic//'`
nr_of_headers_to_be_cleaned=`dpkg -l | grep linux-headers | grep -v headers-generic | cut -d" " -f3 | grep -v "$active_kernels" | wc -l`

if [ "$nr_of_removed_packages" -gt "0" ]; then
  echo "Purge configuration files for removed packages ($nr_of_removed_packages)"
  dpkg --purge `dpkg -l | egrep "^rc" | cut -d" " -f3`
else
  echo "No removed packages"
fi

if [ "$nr_of_headers_to_be_cleaned" -gt "0" ]; then
  echo "Cleaning old kernel headers, but skipping active kernels:"
  echo "$active_kernels"
  echo ""
  echo "Going to clean:"
  dpkg -l | grep linux-headers | grep -v headers-generic | cut -d" " -f3 | grep -v "$active_kernels"
  echo "Wait 5 seconds or break now!!"
  sleep 5
  dpkg --purge `dpkg -l | grep linux-headers | grep -v headers-generic | cut -d" " -f3 | grep -v "$active_kernels"`
else
  echo "No kernel headers to be cleaned"
fi

echo "Done!"

Build/format/mount encrypted LVM volume

cryptsetup -y -v luksFormat /dev/sdc1
cryptsetup luksOpen /dev/sdc1 encrypted-sdc1
pvcreate /dev/mapper/encrypted-sdc1
vgcreate MAIN /dev/mapper/encrypted-sdc1
lvcreate -n LVMAIN -l 100%FREE MAIN
mkfs.ext4 /dev/MAIN/LVMAIN
mount /dev/MAIN/LVMAIN /mnt/lvmtest

Build expansion volume

cryptsetup -y -v luksFormat /dev/sdc2
cryptsetup luksOpen /dev/sdc2 encrypted-sdc2
pvcreate /dev/mapper/encrypted-sdc2

Extend the volumegroup and Logical Volume

vgextend MAIN /dev/mapper/encrypted-sdc2
vgdisplay (and look for the free PE's)
lvextend -l +1830 /dev/MAIN/LVMAIN

(Online) grow the filesystem

resize2fs /dev/MAIN/LVMAIN

Shutdown procedure

umount /mnt/lvmtest
vgchange -an MAIN
cryptsetup luksClose /dev/mapper/encrypted-sdc2
cryptsetup luksClose /dev/mapper/encrypted-sdc1

Start again

cryptsetup luksOpen /dev/sdc1 encrypted-sdc1
cryptsetup luksOpen /dev/sdc2 encrypted-sdc2
vgscan --mknodes
vgchange -ay
mount /dev/MAIN/LVMAIN /mnt/lvmtest

This example hot-migrates an existing LVM volume spanned over 3 disks to a new LVM volume spanned over 3 disks.

Prerequisites:

• lvm2 (apt-get install lvm2)
• 3 disks to start with
• 3 new disks to be added. Disks in this example are 100% identical!

Current LVM

This first part you probably already have, since you want to migrate this volume. But i’m going to create it anyway as part of the whole documentation.
I’m not going to work with partitions and just use the whole disks.

Create the Pysical Volumes

root@lvmtest:~# pvcreate /dev/sdb /dev/sdc /dev/sdd
  Physical volume "/dev/sdb" successfully created
  Physical volume "/dev/sdc" successfully created
  Physical volume "/dev/sdd" successfully created

Create the Volume Group

root@lvmtest:~# vgcreate MAIN /dev/sdb /dev/sdc /dev/sdd

Create the Logical Volume

root@lvmtest:~# lvcreate -n LVMAIN -l 100%FREE MAIN
  Logical volume "LVMAIN" created

Create the filesystem, mount it

root@lvmtest:~# mkfs.xfs /dev/MAIN/LVMAIN

root@lvmtest:~# mkdir /mnt/mylvmvolume

root@lvmtest:~# mount /dev/MAIN/LVMAIN /mnt/mylvmvolume

root@lvmtest:~# df -h | grep MAIN
/dev/mapper/MAIN-LVMAIN   24G   33M   24G   1% /mnt/mylvmvolume

Put some data on it

root@lvmtest:/mnt/mylvmvolume# dd if=/dev/zero of=blabla.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 5.93346 s, 177 MB/s

Add new disks and create the mirror

Add new disks to the machine.

Prepare the new disks:

root@lvmtest:~# pvcreate /dev/sde /dev/sdf /dev/sdg
  Physical volume "/dev/sde" successfully created
  Physical volume "/dev/sdf" successfully created
  Physical volume "/dev/sdg" successfully created

Add the disks to the existing Volume Group

root@lvmtest:~# vgextend MAIN /dev/sde /dev/sdf /dev/sdg

Create a mirror (-m1) of the current data to the recently added space.
Do this in a screen. This can take days, depending on the size!

root@lvmtest:~# lvconvert -m1 --corelog MAIN/LVMAIN /dev/sde /dev/sdf /dev/sdg
  MAIN/LVMAIN: Converted: 0.0%
  MAIN/LVMAIN: Converted: 2.8%
  MAIN/LVMAIN: Converted: 10.6%
  MAIN/LVMAIN: Converted: 20.2%
  MAIN/LVMAIN: Converted: 29.9%
  MAIN/LVMAIN: Converted: 39.1%
  MAIN/LVMAIN: Converted: 48.8%
  MAIN/LVMAIN: Converted: 58.3%
  MAIN/LVMAIN: Converted: 67.8%
  MAIN/LVMAIN: Converted: 77.5%
  MAIN/LVMAIN: Converted: 87.1%
  MAIN/LVMAIN: Converted: 96.8%
  MAIN/LVMAIN: Converted: 100.0%

The mirror is live.

During the conversion, you might see some nice figures using iostat

Device:            tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
sdb             126.00         0.00        63.00          0        126
sdc               0.00         0.00         0.00          0          0
sdd               0.00         0.00         0.00          0          0
sde             126.00        63.00         0.00        126          0
sdg               0.00         0.00         0.00          0          0
sdf               0.00         0.00         0.00          0          0
sda               0.00         0.00         0.00          0          0
dm-0              0.00         0.00         0.00          0          0
dm-1           1004.00        62.75         0.00        125          0
dm-2           1008.00         0.00        63.00          0        126

Break the mirror and go live with the new disks

Create 0 copies (-m0) for the devices that will be removed, a.k.a. breaking the mirror.

root@lvmtest:~# lvconvert -m0 MAIN/LVMAIN /dev/sdb /dev/sdc /dev/sdd

Remove the devices from the Volume Group

root@lvmtest:~# vgreduce MAIN /dev/sdb /dev/sdc /dev/sdd
  Removed "/dev/sdb" from volume group "MAIN"
  Removed "/dev/sdc" from volume group "MAIN"
  Removed "/dev/sdd" from volume group "MAIN"

Remove the Physical Volumes

root@lvmtest:~# pvremove /dev/sdb /dev/sdc /dev/sdd
  Labels on physical volume "/dev/sdb" successfully wiped
  Labels on physical volume "/dev/sdc" successfully wiped
  Labels on physical volume "/dev/sdd" successfully wiped

That’s it.. Hot migrated!

root@lvmtest:~# df -h | grep MAIN
/dev/mapper/MAIN-LVMAIN   24G   11G   14G  42% /mnt/mylvmvolume

HP’s website didn’t seem to work today. Just when i needed an offline ACU iso.

Seems they can be found here:

ftp://ftp.hp.com/ftp1/pub/softlib2/software1/pubsw-windows/p1067040366/