Template/Image migration with reduced file size

In any migration or moving the data from one platform to another, the most difficult challenge is the file size of the image. When we convert the VM instance as a template/image/snapshot it is nothing but a complete clone of the entire disk along with free sectors. Hence even the data is only 10GB the image will be around 30GB or more when the disk size is around 40GB. Moving or uploading a 30GB file over ethernet (even in fast ethernet), it is still a difficult time consuming job and it must work without interruption.  

We faced a similar challenges in moving the templates from Openstack platform to VMware Esxi.

In this blog, we will explain how we move the Openstack Virtual machine to VMware Esxi host with minimal effort and 100% working. I'm actually covering only QCOW2 image to VMDK image conversion in this section, however using the same methods we can do the conversion between any formats like QCOW2 to VHD, QCOW2 to OVA and vice versa.  Let us go step by step.

Requirements

• Must have root admin access to openstack platform, VMware Vcenter and Esxi host terminals.
• Enough free space in the server to do the image conversion actions.

Image Download

As usual, first take a snapshot of the openstack instance that we want to move to VMware Esxi.

This operation will take time depends on the performance on our openstack platform.

Once snapshot is ready, the Image will be stored in Glance image repository. We can use glance client or login to the Openstack controller to download the image in a QCOW2 format. Below is the command syntax. Replace the with our own name.

$ glance image-download --file .qcow2

Image Conversion

Once we have downloaded the file in our local disk, the size of the image will be more than the data inside the image. For example, the 'df -h' in the VM instance will show 5GB occupied on a 20GB disk, however the image size will be around 12GB. To remove the unwanted space in the template, we will use 'virt-sparsify' to do the work along with the format conversion to migrate only the data.

    

Virt-sparsify is a tool which can make a virtual machine disk (or any disk image) sparse a.k.a. thin-provisioned. This means that free space within the disk image can be converted back to free space on the host.

Below is the command to convert and sparse the original image.

$ virt-sparsify --convert

In the area, we have to specify which format that we are going to convert the image to and the extension of the file name should change accordingly. In the below example, we are converting the QCOW2 image to VMDK template.

$ virt-sparsify .qcow2 --convert vmdk .vmdk

We can still reduce this size by zipping it like below.

$ tar -zcvf .vmdk

So, now our image/template will be lot lesser in size.

Deploy the template in Vcenter

In VMware Vcenter, we can not directly upload the VMDK file to launch the instance. Please refer the below steps.

First, we have to create an Virtual machine in Vcenter without a hard disk. In the “customize hardware” page, we can remove the ‘new hard disk’ from the list and create the VM instance.

Now, we have to login to the Esxi host, where the VM instance without hard disk is launched. Unzip the image that we have bundled in the previous step inside this machine. We have to run the VMKFStools to clone the VMDK file into VMware compatible VMDK file.

vmkfstools is one of the ESXi Shell commands for managing VMFS volumes and virtual disks. We can perform many storage operations using the vmkfstools command

Below is the command syntax.

$ vmkfstools -i < input.vmdk > < output.vmdk >

This command will produce two VMDK files like output-flat.vmdk and output.vmdk

Inside the datastore of this Esxi, we can find a folder with the name of the Virtual machine that we have created without a hard disk. We have to move the above two VMDK files inside the Virtual Machine folder.

Now, we can go to the Vsphere browser client and edit the settings of the Virtual machine. In that settings page, we have to choose the existing hard disk option and select the VMDK file which we moved inside the Virtual Machine folder.

Now we can start the VM instance and the VM instance will boot up as usual in the VMware environment.