MichaelCole.com

Virtualization is great!  I can run a full native linux stack on my windows PC.

I want to share this image with others using bittorrent.  To speed up downloads, I want to make the exported files as small as possible. I don't necessarily need to shrink the images files on my host computer, just the exported files.

So the plan is to export it as an ovf (open virtualization format) and vmdk format, and then create a bittorrent.

I accomplished this in three steps:

1) Zerofill inside the virtual machine.

When VirtualBox exports a disk, It compresses the data on the disk.  VirtualBox doesn't manage file systems (that's the OS's job), so we need to blank out any deleted files.

Basically, when the OS deletes files, it "unlinks" the file, leaving the data on the disk.  This is why files can be "undeleted", and people write "secure file shredders" to scrub the data left on the disk.

A whole bunch of Zero's compress very small, so let's zerofill this data.

There are two methods:

Linux:

1. sudo dd if=/dev/zero of=/zerofile; sudo rm /zerofile

Windows (I havn't tried this):

1. Download http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx
2. Start -> Run -> cmd.exe.  Then type sdelete -c c: to zero the disk.

For Linux, the command will make a file full of zero's as big as all the empty space on the disk.  It then delete's the file. 

If the machine's disk is "Dynamic" this will make the vdi bigger.  The host OS will write zero's out on all the disk.  If that's a problem, we can fix it in the next step.

2)  Compact in VirtualBox on the host:

Step 1) wrote zero's on the whole disk.  If you use "Dynamically Expanding Storage" options, then the disk just maxed out it's size on the host.  This isn't a problem for some, but a problem for others.  To "fix" this, use VirtualBox's compact tool.  My host is windows, but it's similar for linux.

Start -> Run -> cmd.exe. 

"cd .VirtualBox"

"C:\Program Files\Sun\xVM VirtualBox\VBoxManage.exe" modifyvdi yourvirtualdisk.vdi compact

This will compact a dynamic disk back down to the size we started with.

3) Export in VirtualBox

Start VirtualBox -> File -> Export Appliance

Select virtual machine and follow the wizard.

Technical details:

We started out with: (F = file,  d = deleted file, . = empty space)

0) FFFFdddd....FFFFFdddddFdFdFFFddddFFFFFddd (2.5gb vdi)

Then we zero filled (this expanded out the disk to it's full size)

1) FFFF........FFFFF.....F.F.FFF....FFFFF............ (8.gb vdi)

Then we compacted the disk with VBoxManage.  (I'm pretty sure VB 2.2.4 just chops off the empty space at the end of the disk)

2) FFFF........FFFFF.....F.F.FFF....FFFFF (2.4gb vdi)

Last, we exported the virtual machine as an ovf and the disk as vmdk: (f= compressed files).

3) fffffffffffffffffff (400mb vmdk file)

Final Note:

I think it's possible to shrink the size of the host's vdi file by defragging the disk FIRST.  This isn't necessary except to make the vdi file smaller on the host.  It won't make the final output smaller.

Linux can't defrag it's file systems, so I didn't do this.  See articles here here and others for why not.

Windows comes with a defragger, and has some free tools here.  I like Defraggler

In the above example this would add a step:

Start)    FFFFdddd....FFFFFdddddFdFdFFFddddFFFFFddd (2.5gb vdi)
Defrag)   FFFFFFFFFFFFFFFFFFFdddddddddddddddddddddd (2.5gb vdi)
Zerofill) FFFFFFFFFFFFFFFFFFF...........................(8gb vdi)
Compact)  FFFFFFFFFFFFFFFFFFF (1.1gb vdi)
Export)   fffffffffffffffffff (400mb vmdk)