Installing discovery
FIREWHEEL installs discovery to help translate the Experiment Graph into minimega commands. You can download the cross-platform discovery binaries from https://github.com/sandia-minimega/discovery and then decompress them by using the command:
sudo tar -C / -xf discovery.tgz
Note
This installs discovery into /opt/discovery.
Installing minimega
To instantiate emulation-based experiments, FIREWHEEL relies on minimega version 2.7 or higher.
minimega should be installed on all Cluster Nodes and configured as a minimega mesh.
Additionally, FIREWHEEL expects that minimega has been configured as a systemd service.
While we recommend that users review minimega documentation, found at https://www.sandia.gov/minimega, this section will provide some details about how to install minimega and configure it for use with FIREWHEEL.
minimega dependencies
Ubuntu
minimega has lots of optional dependencies, but you can install the dependencies necessary for using it with FIREWHEEL by using the following command:
sudo apt install -y openvswitch-switch qemu-kvm qemu-utils dnsmasq ntfs-3g iproute2 libpcap-dev
Once the dependencies are installed, you can follow the instructions found here https://www.sandia.gov/minimega/module-4-installing-minimega/ to download and install minimega package. We recommend using the DEB package, as it is easier.
CentOS
Installing on CentOS is more complicated due to the necessity of installing external repositories in order to install additional dependencies. You will likely need to install EPEL and IUS in order to capture all required packages.
Warning
While many features of minimega/FIREWHEEL will work on default CentOS packages, newer versions of these tools (especially QEMU) are recommended. Specifically, we have encountered VNC-related issues with QEMU 2.0.0 (the default for CentOS) and strongly recommend using version 2.11.0 or higher.
You can install many of the dependencies necessary for using it with FIREWHEEL by using the following command:
sudo yum install -y qemu qemu-kvm qemu-kvm-tools dnsmasq ntfs-3g dosfstools net-tools libpcap libpcap-devel qemu-system-x86
Additionally, minimega requires installing Open vSwitch (OVS). We recommend reviewing the installation procedure for your specific OS. However, for ease of use, we provide some instructions to install OVS from source code, which have been tested on CentOS 7 [1]:
Warning
These instructions likely use an older version of OVS!
# Installing OVS dependencies
sudo yum install -y wget openssl-devel python-sphinx gcc make python-devel openssl-devel kernel-devel graphviz kernel-debug-devel autoconf automake rpm-build redhat-rpm-config libtool python-twisted-core python-zope-interface PyQt4 desktop-file-utils libcap-ng-devel groff checkpolicy selinux-policy-devel python-six unbound unbound-devel
# Create directories
mkdir -p ~/rpmbuild/SOURCES
# Download OVS source code
wget https://www.openvswitch.org/releases/openvswitch-2.12.0.tar.gz
cp openvswitch-2.12.0.tar.gz ~/rpmbuild/SOURCES/
tar xfz openvswitch-2.12.0.tar.gz
# Build the package
rpmbuild -bb --nocheck openvswitch-2.12.0/rhel/openvswitch-fedora.spec
# Install the newly created RPMs
sudo yum localinstall -y ~/rpmbuild/RPMS/x86_64/openvswitch-2.12.0-1.el7.x86_64.rpm
sudo yum localinstall -y ~/rpmbuild/RPMS/x86_64/openvswitch-debuginfo-2.12.0-1.el7.x86_64.rpm
sudo yum localinstall -y ~/rpmbuild/RPMS/x86_64/openvswitch-devel-2.12.0-1.el7.x86_64.rpm
# Restart OVS to ensure it is running
systemctl start openvswitch.service
systemctl enable openvswitch.service
Finally, users should create a new script which will run QEMU with the -enable-kvm option.
This script is installed via the Ubuntu version of the qemu-kvm [2], but not the CentOS version.
To enable the same functionality (which is required for minimega), run the following commands:
sudo echo '#!/bin/sh
exec /usr/libexec/qemu-kvm -enable-kvm "$@"
' >> /bin/kvm
sudo chmod 751 /bin/kvm
Once the dependencies are installed, you can follow the instructions found at https://www.sandia.gov/minimega/module-4-installing-minimega/ to download and install minimega package. We recommend using the RPM package, as it is easier.
Configuring minimega
Once minimega has been installed, we should configure it for use with FIREWHEEL.
First, the minimega systemd service should be installed by running:
echo -n "" | sudo /opt/minimega/misc/daemon/minimega.init install
Note
This assumes that minimega was installed into /opt/minimega.
Next, we need to update minimega’s configuration file located at /etc/minimega/minimega.conf.
The MM_MESH_DEGREE variable should be updated to be the size of your FIREWHEEL Cluster.
For example, if your FIREWHEEL Cluster contained 2 nodes, you should change it to be MM_MESH_DEGREE=2.
Here is a single line find/replace which can be useful in scripting this action:
sudo sed -i 's/MM_MESH_DEGREE=0/MM_MESH_DEGREE=1/g' /etc/minimega/minimega.conf
Next, it is important to set the correct permissions and ownership for both minimega and discovery. We can use the following commands:
sudo chown -R :minimega /opt/minimega
sudo chown -R :minimega /opt/discovery
sudo chmod -R g=u /opt/minimega
Next, the user should join the minimega system group.
For example, if you wanted the fw user to run experiments you can use the command:
sudo usermod -a -G minimega fw
Note
Be sure to log out of the system for the new group permissions to take effect.
The minimega installer leaves the minimega binaries in /opt/minimega/bin.
We recommend linking these as system packages by using:
# Link minimega binaries
sudo ln -s /opt/minimega/bin/minimega /bin/minimega
sudo ln -s /opt/minimega/bin/minimega /bin/mm
Lastly, make sure that the minimega service is running:
sudo systemctl restart minimega