Installing FIREWHEEL
Once all of FIREWHEEL’s Prerequisites have been completed, FIREWHEEL can be installed either via pip or via source.
Creating a Python virtual environment
FIREWHEEL requires a modern version of Python (currently 3.8-3.13) to run.
For ease of use, we recommend installing FIREWHEEL into a virtual environment on all Cluster Nodes.
In this example, we are creating a Python version 3.10 venv
called fwpy
.
python3.10 -m venv fwpy
source fwpy/bin/activate
Warning
The path to FIREWHEEL’s virtual environment MUST be the same on each node in the cluster. Therefore, we recommend a common path (e.g. /opt/firewheel/fwpy
) rather than using a home directory.
Installing from Source Code
First, clone the FIREWHEEL git repository into a desired location (in this case /opt/firewheel
).
For this example, we assume that repo
is a valid entry in a users ~/.ssh/config
file. E.g.
Host repo
Hostname github.com
User git
IdentityFile=~/.ssh/github_key
# Make a directory and give it the correct permissions
sudo mkdir /opt/firewheel
sudo chown -R $(whoami): /opt/firewheel
# Clone FIREWHEEL into the new directory
git clone ssh://repo/firewheel/firewheel.git /opt/firewheel
FIREWHEEL’s install.sh
will make the remainder of the installation process easy.
However, before running that script, there are several environment variables which should be set.
Set Environment Variables
Our installation script allows users to customize installation of FIREWHEEL.
The default values are found in provision_env.sh
, but they can be overwritten by setting the key as an environment variable.
For example, to change the default value of DISCOVERY_PORT
, you can use:
export DISCOVERY_PORT=9090
Generally, the defaults are acceptable for most installations.
However PYTHON_BIN
is likely to change between environments (especially if not using a virtual environment).
Additionally, on clusters of more than one node, FIREWHEEL_NODES
, EXPERIMENT_INTERFACE
, and GRPC_THREADS
should be set.
Finally, if VLANs cannot be used to route traffic between FIREWHEEL nodes, then USE_GRE
should be set to true
.
A full list of environment variables used in install.sh
is shown below.
- sid
- Description:
The user account who will use FIREWHEEL.
- Default:
"$(whoami)"
- FIREWHEEL_NODES
- Description:
The hostnames of the nodes in the FIREWHEEL Cluster in a space separated list.
- Default:
"$(hostname)"
- HEAD_NODE
- Description:
The hostname of the Control Node.
- Default:
"$(echo $FIREWHEEL_NODES | cut --delimiter ' ' --fields 1)"
- FIREWHEEL_ROOT_DIR
- Description:
For users installing FIREWHEEL from source, this is the root directory of the source code repository.
- Default:
/opt/firewheel
- FIREWHEEL_VENV
- Description:
The location of the default FIREWHEEL virtual environment.
- Default:
${FIREWHEEL_ROOT_DIR}/fwpy
- PYTHON_BIN
- Description:
The Python interpreter executable (a path or symlink).
- Default:
python3
- PIP_ARGS
- Description:
Any additional arguments/options required for Pip.
- Default:
""
- EXPERIMENT_INTERFACE
- Description:
The experimental network interface. That is, which NIC connects all nodes in the FIREWHEEL Cluster.
- Default:
lo
- USE_GRE
- Description:
Whether to use GRE tunnels rather than VLANs to segment traffic for minimega.
- Default:
false
- MM_BASE
- Description:
The location of minimega’s run time files (e.g. VM logs, files, etc.)
- Default:
"/tmp/minimega"
- MM_GROUP
- Description:
The minimega user group.
- Default:
minimega
- MM_CONTEXT
- Description:
The context for finding minimega meshage peers. It should distinguish your minimega instances from any others on the network. See https://www.sandia.gov/minimega/using-minimega/ for more information.
- Default:
${HEAD_NODE}
- MM_INSTALL_DIR
- Description:
The installation directory for minimega.
- Default:
"/opt/minimega"
- DISCOVERY_PORT
- Description:
The HTTP port for the Discovery service.
- Default:
8080
- DISCOVERY_HOSTNAME
- Description:
The hostname for the Discovery service.
- Default:
localhost
- GRPC_HOSTNAME
- Description:
The hostname for FIREWHEEL’s GRPC server.
- Default:
${HEAD_NODE}
- GRPC_PORT
- Description:
The port number to use for FIREWHEEL’s GRPC server.
- Default:
50051
- GRPC_THREADS
- Description:
The number of threads to use for FIREWHEEL’s GRPC server.
- Default:
2
- FIREWHEEL_GROUP
- Description:
The FIREWHEEL’s user group (if any).
- Default:
${MM_GROUP}
- MC_BRANCH
- Description:
The Git branch for the model component repositories that will be cloned.
- Default:
master
- MC_DIR
- Description:
The location of the model component repositories.
- Default:
${FIREWHEEL_ROOT_DIR}/model_components
- MC_REPO_GROUP
- Description:
The URL which contains FIREWHEEL’s model component repositories.
- Default:
https://repo/firewheel/model_components
- DEFAULT_OUTPUT_DIR
- Description:
The default directory for FIREWHEEL’s various logs, cached Helpers, and other outputs.
- Default:
/tmp/firewheel
Running install.sh
Once the configuration options have been set, run install.sh
:
cd /opt/firewheel
chmod +x install.sh
./install.sh
This script will create necessary directories, install and configure FIREWHEEL, and install our default model component repositories.
The script will optionally install additional FIREWHEEL development dependencies by using either the -d
or --development
flag:
./install.sh -d
Post Installation Steps
Once the entire cluster has been provisioned, we recommend running the sync command to cache Helpers across the cluster:
firewheel sync
Installing Model Component Repositories
Before running any experiments, you will likely need to install several Model Component repositories. Model Component repositories provide extra features and customizations that give FIREWHEEL experiments their flexibility and modularity.
There are two ways to install Model Components.
The first and easiest way is to use a premade Model Component repository which has been converted into a (pip-installable) Python package.
Several of the most commonly used FIREWHEEL Model Components exist in this format, for example the base
and linux
Model Components.
pip install firewheel_repo_base
A selection of these MCs are installed by the install.sh
script.
The second way to install a Model Component is using FIREWHEEL’s helpers.
This is a good solution for Model Component repositories that are either built or cloned to a local directory.
We recommend putting all these Model Component repositories in /opt/firewheel/model_components
.
Then you can use the repository install to add this location as a known FIREWHEEL Model Component repository:
firewheel repository install /opt/firewheel/model_components
For more information see FIREWHEEL Repositories. For information about available Model Components see Model Component Documentation.
Environment Enhancements
Once FIREWHEEL has been installed, we recommend symbolically linking FIREWHEEL’s CLI to /usr/bin/firewheel
to avoid activating the virtual environment for each new terminal session.
sudo ln -s "$(which firewheel)" /usr/bin/firewheel
We also recommend configuring tab-complete for FIREWHEEL’s CLI. If using bash, the simplest approach is:
sudo cp <path-to-FIREWHEEL>/firewheel_completion.sh /usr/share/bash-completion/completions/firewheel
Alternatively, the following command can be added to a .bashrc
, .zshrc
, etc.
source <path-to-FIREWHEEL>/firewheel_completion.sh
If the FIREWHEEL configuration changes, the tab completion script <path-to-FIREWHEEL>/firewheel_completion.sh
may be automatically regenerated at any point.
python -m firewheel.cli.completion.prepare_completion_script