Building a New Image from ISO

This tutorial will assume that you’re starting from nothing with just an ISO that was downloaded from the Internet. In the case where an image is already available, then skip ahead to Adding the QEMU Guest Agent to the Image.

Downloading an ISO file

We can build a new Ubuntu 18.04 Server image to include in FIREWHEEL. In order to do this we need the ISO file to do the installation. It can be downloaded from the Ubuntu website. We will also need a desktop version if you choose to follow the steps in Optimizing Disk Image Size. That can be downloaded from Ubuntu Desktop.

Create a Disk

A disk is required to permanently install the operating system. We can use qemu-img create in order to create a new disk image:

$ qemu-img create -f qcow2 ubuntu-18.10-server-amd64.qcow2 10G

The command above creates a qcow2 formatted blank disk of size 10 GB that is named ubuntu-18.10-server-amd64.qcow2. You are free to adjust the size of the disk as needed. Generally, installation is done with a small disk size to keep the disk compact. We can then increase the disk size once the installation is complete. This is covered in the section Optimizing Disk Image Size. To be clear, it is perfectly acceptable to give the disk a larger size at this point and not bother with the optimization steps. In the case where you will not be optimizing the disk, then disk sizes between 20 and 50 GB are usually sufficient.

Note

The permissions on the disk image should enable read and write access. Without correct disk permissions, the image will fail to run scheduled actions.

Starting Networking

Before booting the new disk with the ISO, we need to have a network for communication with the VM. This can be done using the libvirt package.

$ sudo apt-get install libvirt-bin

The libvirt package will handle all the NAT and DHCP settings for us and allow us to communicate with the VM from the underlying physical host. Now that the package is installed, we can start the networking:

$ sudo virsh net-create default

If that command fails then run the following:

$ sudo virsh net-start default

This will create a virtual bridge called virbr0. You can confirm that is has been created by running:

$ ip address

This will print all the networking interfaces on the physical machine. You should see virbr0 in the list. We will use this bridge once the VM has been booted.

Booting the Image

The installation requires that the image get booted with both the disk that was just created and the ISO that was just downloaded. The following command will start the VM with all required settings:

$ sudo  /usr/bin/qemu-system-x86_64 -nographic -nodefaults --enable-kvm -name ubuntu \
-drive file=/opt/firewheel/ubuntu-18.10-server-amd64.qcow2,if=virtio,cache=writeback \
-vnc 0.0.0.0:0 \
-cpu qemu64 -smp sockets=1,cores=4,threads=2 \
-m 8092 -vga std \
-netdev tap,ifname=installer,id=hostnet0,script=no,downscript=no \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:00:00:ff:ff:01 \
-device piix3-usb-uhci -device usb-tablet -device piix3-usb-uhci \
-chardev socket,id=qga0,server,nowait,path=/tmp/ga.sock \
-device virtio-serial \
-device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 \
-drive file=/opt/firewheel/ubuntu-18.10-server-amd64.iso,index=2,media=cdrom

Let’s walk through what some of the options are providing the VM and which ones can be modified.

  • name: This is a name that is provided by the user. It has no bearing on the final image and can be changed to anything you like.

  • drive: This tells the VM where to find the disk image that was just created. This should be the absolute path to the disk that was made above.

  • vnc: This creates a VNC server that allows access to the VM’s console. In this case we chose VNC port 0 (which gets translated to 5900). Feel free to change this to any open VNC port.

  • cpu: We’re using a default of qemu64 for the CPU architecture. Some images may require a specific CPU architecture. CPU options can be viewed by running: qemu-system-x86_64 -cpu ?

  • smp: The SMP options allow a user to specify the number of CPU sockets the image will have, how many cores each socket will have, and how many threads each core will have. Modify these values as needed.

  • netdev: This creates a network device. In this case we are naming it installer. Feel free to change this name.

  • device: This is the the second part to creating a network interface. This plugs a network card into a PCI slot on the VM. It is tied to a specific netdev which is specified as a parameter.

  • chardev: This is a character device that is the first part of creating a virtual serial port. FIREWHEEL uses a serial port to orchestrate VM configuration.

  • device: This virtserialport is the second part of creating a virtual serial port. It is tied to a specific chardev which is specified as a parameter.

  • drive: This last drive specifies that a CDROM should be connected to the VM. The file parameter is the absolute path to the ISO file that was downloaded above.

Connecting the Networking

The qemu-system-x86_64 command above created a network interface called installer. Run the following command to confirm that it exists:

$ ip address

We can put that interface on the bridge that was created above to give the VM access to the network.

$ sudo brctl addif virbr0 installer
$ sudo ip link set dev installer up
$ sudo ip link set dev virbr0 up

This will provide the VM with an IP address via DHCP. Commonly, this ends up being 192.168.122.19/24.

Installing the Operating System

Connect to the VNC server at port 0. An easy way to do this is to use minimega’s version of noVNC. Assuming minimega is installed at /opt/minimega, to start the client, you can use:

$ cd /opt/minimega/misc/web/novnc
$ ./utils/launch.sh

This will launch a VNC client that you can connect to in a browser. Note that you may have to forward the port to a local desktop and change hostnames to 127.0.0.1 if you are running FIREWHEEL on a remote server.

You should see a screen that is giving you the option to install the operating system. Follow the on screen instructions through the completion of the installation. You will eventually be asked to restart the VM. Go ahead and do so.

Adding the QEMU Guest Agent to the Image

Now that you have an image with an operating system installed, we need to make it so that FIREWHEEL can communicate and orchestrate the configuration of the VM. This is done through the QEMU Guest Agent (QGA). The QGA is a process that runs inside the VM and talks to the FIREWHEEL infrastructure through the virtual serial port that is added when the VM is launched. We need to load the QGA into the VM and set it to be a service.

Installing the VirtIO Serial Port Driver on Windows

As mentioned above, FIREWHEEL uses a serial port in order to orchestrate the configuration of VMs. Windows VMs do not have the ability to use a VirtIO serial port by default, therefore the driver needs to be installed. The latest virtio-win ISO file can be found via the virtio-win-pkg-scipts GitHub page. This can be attached to the VM via a CDROM in the same way the operating system ISO was attached to the VM during building (see Booting the Image for more details). These instructions follow the steps provided by Redhat starting at Procedure 10.2. The steps are included below for completeness.

  • Once the VM has booted with the VirtIO ISO attached, we need to get to the Device Manager.

  • This is accessed through the Computer Management window.

  • You can get to the Computer Management window by selecting Start and then right clicking on Computer and selecting Manage from the menu that appears.

  • On the left pane you should see Device Manager in the drop down below System Tools, select Device Manager.

  • You should see a device called Other Devices, expand that drop down arrow.

  • Another device should appear below Other Devices that is labeled PCI Simple Communication Controller.

  • Right click PCI Simple Communication Controller and select Update Driver Software.

  • You will then see a pop-up window asking how to search for the driver software, select Browse my computer for driver software.

  • Click the Browse button and then navigate to virtio-win.iso that was attached as a CDROM.

  • We need the vioserial driver that corresponds to the version of Windows that is being used.

  • This will be a folder and then select OK.

  • Then select the Next button to install the vioserial driver.

  • Once the installation has finished select Close.

Moving the QGA into the VM

There are a couple ways to get the required QGA files into the VM. Which method to use will depend on whether or not the VM is running a SSH server.

QGA Files for Linux

The directions for building a statically compiled QGA can be found in QGA Modifications. The qemu-ga-patched-static binary is the actual guest agent executable. The qemu-guest-agent.service file defines a systemd service. Most versions of Linux support systemd, which makes it a good choice for turning the QGA into a service on the VM.

QGA File for Windows

The QEMU Guest Agent has a MSI installer for Windows. Therefore, if you are building a Windows machine, you only need to move that installer over to the VM using one of the methods listed below. Instructions for building a modified QGA can be found in QGA Modifications and a link to download the latest unmodified qemu-ga.msi can be found via the virtio-win-pkg-scipts GitHub page.

Moving Files with SCP

If the VM has a SSH server running then you can simply SCP the files to the VM. Assuming both the binary and systemd file are in the current directory on the physical host, the VM has an IP address of 192.168.122.19, and the VM has a username of ubuntu then the following command will send the files to the ubuntu user’s home directory:

$ scp qemu-ga-patched-static ubuntu@192.168.122.19:
$ scp qemu-guest-agent.service ubuntu@192.168.122.19:

You can check the IP address of the VM by running ip address from within the VNC session. Change the username above to whatever username was created during installation.

Moving Files over HTTP

If the VM does not have a SSH server running (i.e. Windows or Linux Desktops), then the next best option for getting the files to the VM is to use a web server. A Python web server can easily host the files on the physical host so that they can be accessed from within the VM. Run the following command from directory where the binary and systemd files are located:

$ python3 -m http.server 8000

This will create a web server on the physical host at port 8000 (which is the default port). Feel free to change the port as desired.

From within the VM’s VNC session you can now open a web browser and browse to the QGA files and download them. The files should be available at http://192.168.122.1:8000.

If a browser is not an option then you can use a tool like wget or curl to download the files as well:

$ wget http://192.168.122.1:8000/qemu-ga-patched-static
$ wget http://192.168.122.1:8000/qemu-guest-agent.service

Configuring the QEMU Guest Agent

Configuring the QGA on Linux

Now that the files are located on the VM, there are just a couple more steps before we’re done.

  1. First, move qemu-ga-patched-static to /usr/sbin:

    $ sudo mv qemu-ga-patched-static /usr/sbin

  2. Make sure that the binary is executable:

    $ sudo chmod +x /usr/sbin/qemu-ga-patched-static

  3. Next, move qemu-guest-agent.service to /etc/systemd/system.

    $ sudo mv qemu-guest-agent.service  /etc/systemd/system

  4. The QGA expects to be able to use a directory at /usr/local/var/run.

    $ sudo mkdir -p /usr/local/var/run

  5. Finally, enable the QGA service:

    $ sudo systemctl enable qemu-guest-agent.service

  6. You can now shutdown the VM:

    $ sudo poweroff

Configuring the QGA on Windows

Configuring the guest agent on Windows is generally easier than on Linux. First, ensure that you have booted the Windows image with the QEMU arguments:

-chardev socket,id=qga0,server,nowait,path=/tmp/ga.sock \
-device virtio-serial \
-device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 \

If you followed the instructions in the Booting the Image section, you will have already done this. We have found that the QGA will not run correct if it was installed without these arguments. Once you have started the image, simply run the MSI installer (see QGA File for Windows) as an administrator and you are done. Generally, the user that was created during the installation of a Windows image is a local administrator so if you are logged in as that user then you simply double click the installer to run it. You can shutdown the VM.

Packaging up the Image

It’s a good idea to compress the image file before using it within FIREWHEEL. If you are planning on following the steps in Optimizing Disk Image Size then skip this step for now. It will be revisited once those steps have been completed. This can be done with the following command:

$ xz -k -z ubuntu-18.10-server-amd64.qcow2

This will create a file called ubuntu-18.10-server-amd64.qcow2.xz. The -k flag tells xz to keep the original file after the compression is done. If you do not want to keep the original file then omit the -k flag. This file will be tied to a model component object. That will be discussed in the next section.

Note

FIREWHEEL will automatically detect and decompress images that are using tar or LZMA compression. That is, if your file uses LZMA compression (e.g. the xz utility) or tar compression (including tar with gzip), then it will automatically be decompressed by FIREWHEEL. See The Images field for more details.