Issue


When trying to hot migrate a virtual server on a CloudBoot compute resource, the migration fails with an "error: internal error Attempt to migrate guest to the same host" message.


Environment


CloudBoot KVM compute resources 


Troubleshooting


Before a migration starts, virsh check the UUID of the system by executing virsh sysinfo. Additionally, it executes the same command on the host the virtual server will be migrated to. This is to avoid migrating to the same server in case of a cluster of compute resources.

  1. Check what virsh sysinfo returns for your compute resources:

    [root@10.2.2.106 ~]# dmidecode -s system-uuid
    4C4C4544-0053-4C10-8038-B2C04F334831
    
    [root@10.2.2.107 ~]# dmidecode -s system-uuid
    4C4C4544-0053-4C10-8038-B2C04F334831
    
    [root@10.2.2.108 ~]# dmidecode -s system-uuid
    4C4C4544-0053-4C10-8038-B2C04F334831
    
    [root@10.2.2.109 ~]# dmidecode -s system-uuid
    4C4C4544-0053-4C10-8038-B2C04F334831
    CODE
  2. As we can see, they all have the same UUID. In this case, it was Dell all in one 4x servers in one enclosure.

Resolution


In a non-–°loudBoot environment, we can simply specify the UUID in /etc/libvirt/libvirtd.conf and restart libvirtd. Another option is to apply the wrapper only (see how to download the wrapper in step 8). virsh searches for dmidecode in /usr/sbin/dmidecode, but /usr in our CloudBoot images is a cramfs file system and mounts read-only.

The solution is to recreate initrd with the wrapper and specify the UUID in /etc/libvirt/libvirtd.conf.

  1. Go to the images directory:

    cd /tftpboot/images/centos6/ramdisk-kvm/
    CODE
  2. Back up initrd:

    cp initrd.img initrd.img.backup
    CODE
  3. Create the directory where the original initrd will be unpacked:

    mkdir -v /tftpboot/images/centos6/ramdisk-kvm/initrd.unpacked && cd /tftpboot/images/centos6/ramdisk-
    
    kvm/initrd.unpacked
    CODE
  4. Decompress intrd:

    zcat ../initrd.img | cpio -id
    CODE
  5. Mount the usr directory (it will be read-only):

    mount -o loop usr.cramfs usr
    CODE
  6. Copy the information of usr to another location to be able edit and apply the changes (it is recommended to use the /tftpboot/images/centos6/ramdisk-kvm/initrd.unpacked directory):

    rsync -av usr/ /tmp/cramfs-usr/
    CODE
  7. Copy the real binary demidecode:

    cp usr/sbin/dmidecode sbin/dmidecode.orig
    CODE
  8. Download the wrapper, copy it to the temporary directory and give permissions to execute:

    wget http://downloads.repo.onapp.com/dmidecode
    
    cp demidecode (This should be the wrapper) /tmp/cramfs-usr/sbin/dmidecode
    
    chmod +x /tmp/cramfs-usr/sbin/dmidecode
    CODE
  9. Again, from within the /tftpboot/images/centos6/ramdisk-kvm/initrd.unpacked directory, umount the usr.cramfs image and delete it:

    umount usr.cramfs
    
    rm -rf usr.cramfs
    CODE
  10. Create usr.cramfs with the wrapper applied. It will create a ro cramfs image:

    mkfs.cramfs /tmp/cramfs-usr/ usr.cramfs
    CODE
  11. Recreate initrd and clean the dirs created:

    find . | cpio -H newc --quiet -o | gzip -9 > ../initrd.img.new && cd .. && mv -fv initrd.img.new initrd.img
    CODE
  12. Clean the files:

    rm -rf /tmp/cramfs-usr/
    CODE
  13. Add the following custom config to all compute resources. Create a UUID with uuidgen so that the UUIDs are different:

    ##uuid for libvirtd.conf##
    echo 'host_uuid = "BA7CC229-D375-410F-88E9-0A918DAA7120" ' >> /etc/libvirt/libvirtd.conf
    service libvirtd restart
    #####
    CODE
  14. Reboot the compute resources in order to load the new initrd and custom config. 
  15. Check that the new UUID is being used by libvirtd.

As you can see, the UUID in libvirtd.conf is the same as the one we get when executing virsh sysinfo:

[root@10.2.2.108 ~]# tail /etc/libvirt/libvirtd.conf
# sending any keepalive messages.
#
#keepalive_interval = 5
#keepalive_count = 5
#
# If set to 1, libvirtd will refuse to talk to clients that do not
# support keepalive protocol. Defaults to 0.
#
#keepalive_required = 1
host_uuid = "BA7CC229-D375-410F-88E9-0A918DAA7120"

[root@10.2.2.108 ~]# virsh sysinfo | grep uuid
<entry name='uuid'>BA7CC229-D375-410F-88E9-0A918DAA7120</entry> 
CODE


Cause


The same UUIDS on the compute resources. Before the migration starts, virsh checks the UUID of the system by executing virsh sysinfo. Additionally, it executes the same command on the host that the VS will be migrated to. This is to avoid migrating to the same server in case of a cluster of compute resources.