Moving disks from Xen to KVM

Sunday, 04. 8. 2012  –  Category: stash

Overview

  • Moving virtual machines from Xen to KVM
  • The storage for both is under LVM
  • Under Xen the storage for each VM isn’t partitioned as a disk, it’s just a filesystem. GRUB under KVM will need to see a boot sector and a partition table.
  • Under Xen the storage for each VM doesn’t contain a kernel. GRUB under KVM will need one to boot the VM.

Sequence

  • Measure disks on Xen server, create on KVM server. Create the new root disk 10MB larger than its counterpart. (( Later we align the original root filesystem at cylinder 1 on this new disk. A LVM cylinder is roughly 8MB ))
    
kyero@xen03 ~ $ sudo lvscan
  ACTIVE            '/dev/vg4/mail-root' [20.00 GB] inherit
  ACTIVE            '/dev/vg4/mail-swap' [1.00 GB] inherit


    
[kyero@vm02 ~]$ sudo lvcreate -L20490M -n mail-root vg4 
[kyero@vm02 ~]$ sudo lvcreate -L2G -n mail-swap vg4
  • Create a boot disk on the KVM server. We make it 4 cylinders long, roughly 32MB. (( File backed disk is used rather than LVM owing to grief with GRUB and LVM (Error 22: No such partition) ))
    
[kyero@vm02 ~]$ sudo dd if=/dev/zero of=/var/boot-images/mail-boot bs=32901120 count=1
  • Partition the boot disk with one partition starting at cylinder zero
    
[kyero@vm02 ~]$ sudo sfdisk /var/boot-images/mail-boot
last_lba(): I don't know how to handle files with mode 81a4
Warning: /var/boot-images/mail-boot is not a block device
Disk /var/boot-images/mail-boot: cannot get geometry

Disk /var/boot-images/mail-boot: 4 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /var/boot-images/mail-boot: unrecognized partition table type
Old situation:
No partitions found
Input in the following format; absent fields get a default value.
<start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>\nUsually you only need to specify <start> and <size> (and perhaps <type>).

/var/boot-images/mail-boot1 :0
/var/boot-images/mail-boot1          0+      3       4-     32129+  83  Linux
/var/boot-images/mail-boot2 :
/var/boot-images/mail-boot2          0       -       0          0    0  Empty
/var/boot-images/mail-boot3 :
/var/boot-images/mail-boot3          0       -       0          0    0  Empty
/var/boot-images/mail-boot4 :
/var/boot-images/mail-boot4          0       -       0          0    0  Empty
New situation:
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/var/boot-images/mail-boot1          0+      3       4-     32129+  83  Linux
/var/boot-images/mail-boot2          0       -       0          0    0  Empty
/var/boot-images/mail-boot3          0       -       0          0    0  Empty
/var/boot-images/mail-boot4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Do you want to write this to disk? [ynq] y
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Inappropriate ioctl for device

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
  • Format and mount the partition
    
[kyero@vm02 ~]$ sudo kpartx -av /var/boot-images/mail-boot 
add map loop1p1 : 0 64259 linear /dev/loop1 1

[kyero@vm02 ~]$ sudo mke2fs /dev/mapper/loop1p1 
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
8032 inodes, 32128 blocks
1606 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=33030144
4 block groups
8192 blocks per group, 8192 fragments per group
2008 inodes per group
Superblock backups stored on blocks: 
        8193, 24577

Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

[kyero@vm02 ~]$ sudo mount /dev/mapper/loop1p1 /mnt/boot/

[kyero@vm02 ~]$ df -h /mnt/boot
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/loop1p1    31M  389K   29M   2% /mnt/boot
  • Copy a kernel to it, configure GRUB’s notion of the root device and install the boot sector.
    
[kyero@vm02 ~]$ sudo rsync -a /boot/ /mnt/boot/
[kyero@vm02 ~]$ sudo vi /mnt/boot/grub/menu.lst
[kyero@vm02 ~]$ sudo touch /mnt/boot/kvm-boot-image
[kyero@vm02 ~]$ sudo grub --device-map=/dev/null

    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

 [ Minimal BASH-like line editing is supported.  For the first word, TAB
   lists possible command completions.  Anywhere else TAB lists the possible
   completions of a device/filename.]
grub> device (hd0) /var/boot-images/mail-boot
device (hd0) /var/boot-images/mail-boot
grub> find /kvm-boot-image
find /kvm-boot-image
 (hd0,0)
grub> root (hd0,0)
root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
Done.
grub> quit

[kyero@vm02 ~]$ sudo umount /mnt/boot
[kyero@vm02 ~]$ sudo kpartx -d /var/boot-images/mail-boot 
loop deleted : /dev/loop1
  • Populate the new root disk with one cylinder of zeros and a copy of the old root disk.
    The first cylinder will contain the MBR and not much else. The padding means it’s easy to align the old root disk at cylinder one.
    Use pv so we can see the copy progress.
    The dd-lvm-out script is mostly just a sudo-blessed wrapper.
    [kyero@vm02 ~]$ ( dd if=/dev/zero bs=8225280 count=1 ; ssh xen03.local -l kyero sudo /data/system/bin/dd-lvm-out.sh mail-root ) | pv -e -r -s 21474836480 | sudo dd bs=1048577 of=/dev/vg4/mail-root
1+0 records in
1+0 records out
8225280 bytes (8.2 MB) copied, 0.03395 seconds, 242 MB/s
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 611.215 s, 35.1 MB/s
0+662630 records in
0+662630 records out
21483061795 bytes (21 GB) copied, 615.249 seconds, 34.9 MB/s
  • Partition the new root disk with a single partition starting at cylinder one.
    [kyero@vm02 ~]$ sudo sfdisk /dev/vg4/mail-root
Checking that no-one is using this disk right now ...
BLKRRPART: Invalid argument
OK

Disk /dev/vg4/mail-root: 2614 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
 /dev/vg4/mail-root: unrecognized partition table type
Old situation:
No partitions found
Input in the following format; absent fields get a default value.
<start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>\nUsually you only need to specify <start> and <size> (and perhaps <type>).

/dev/vg4/mail-root1 :1
/dev/vg4/mail-root1          1    2613    2613   20988922+  83  Linux
/dev/vg4/mail-root2 :
/dev/vg4/mail-root2          0+      0       1-      8032   83  Linux
/dev/vg4/mail-root3 :
/dev/vg4/mail-root3          0       -       0          0    0  Empty
/dev/vg4/mail-root4 :
/dev/vg4/mail-root4          0       -       0          0    0  Empty
New situation:
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/vg4/mail-root1          1    2613    2613   20988922+  83  Linux
/dev/vg4/mail-root2          0+      0       1-      8032   83  Linux
/dev/vg4/mail-root3          0       -       0          0    0  Empty
/dev/vg4/mail-root4          0       -       0          0    0  Empty
Warning: no primary partition is marked bootable (active)
This does not matter for LILO, but the DOS MBR will not boot this disk.
Do you want to write this to disk? [ynq] y
Successfully wrote the new partition table

Re-reading the partition table ...
BLKRRPART: Invalid argument

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
  • Mount the new root filesystem
    [kyero@vm02 ~]$ sudo kpartx -av /dev/vg4/mail-root
add map mail-root1 : 0 41977845 linear /dev/vg4/mail-root 16065
add map mail-root2 : 0 16064 linear /dev/vg4/mail-root 1
[kyero@vm02 ~]$ sudo e2fsck /dev/mapper/mail-root1
e2fsck 1.39 (29-May-2006)
/dev/mapper/mail-root1: clean, 435213/1310720 files, 4086961/5242880 blocks

[kyero@vm02 ~]$ sudo mount /dev/mapper/mail-root1 /mnt/tmp
  • Add the new kernel’s modules, tweak filesystem mounts, disable swap, check console.
    
[kyero@vm02 ~]$ sudo rsync -a /lib/modules/ /mnt/tmp/lib/modules/

[kyero@vm02 ~]$ sudo vi /mnt/tmp/etc/fstab

[kyero@vm02 ~]$ sudo vi /mnt/tmp/etc/inittab
  • Create the KVM guest, assigning the boot, root and swap storage
    [kyero@vm02 ~]$ sudo virt-install -n mail -r 1024 --vcpus 1 --os-type linux --os-variant generic26 --accelerate --disk path=/var/boot-images/mail-boot --disk path=/dev/mapper/vg4-mail--root,bus=ide,format=raw --disk path=/dev/mapper/vg4-mail--swap,bus=ide,format=raw --network bridge:br0 --network bridge:br1 --import --noreboot
Starting install...
Guest installation complete... you can restart your domain
by running 'virsh start mail'

[kyero@vm02 ~]$ sudo virsh dumpxml mail
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/boot-images/mail-boot'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/mapper/vg4-mail--root'/>
      <target dev='hdb' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='1'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/mapper/vg4-mail--swap'/>
      <target dev='hdd' bus='ide'/>
      <address type='drive' controller='0' bus='1' unit='1'/>
    </disk>
...
  • Boot the KVM guest
    
[kyero@vm02 ~]$ sudo virsh start mail
[kyero@vm02 ~]$ sudo virsh autostart mail
[kyero@vm02 ~]$ sudo virsh console mail
  • Log into KVM guest, partition /dev/hdd and configure it as swap, update /etc/fstab.

Comments are closed.