Installing FIREWHEEL
Once all of FIREWHEEL’s Prerequisites have been completed, FIREWHEEL can be installed either via pip or via source.
Creating a Python virtual environment
FIREWHEEL requires a modern version of Python (currently 3.8-3.13) to run.
For ease of use, we recommend installing FIREWHEEL into a virtual environment on all Cluster Nodes.
In this example, we are creating a Python version 3.10 venv called fwpy.
python3.10 -m venv fwpy
source fwpy/bin/activate
Warning
The path to FIREWHEEL’s virtual environment MUST be the same on each node in the cluster. Therefore, we recommend a common path (e.g. /opt/firewheel/fwpy) rather than using a home directory.
Installing from Source Code
First, clone the FIREWHEEL git repository into a desired location (in this case /opt/firewheel).
For this example, we assume that repo is a valid entry in a users ~/.ssh/config file. E.g.
Host repo
Hostname github.com
User git
IdentityFile=~/.ssh/github_key
# Make a directory and give it the correct permissions
sudo mkdir /opt/firewheel
sudo chown -R $(whoami): /opt/firewheel
# Clone FIREWHEEL into the new directory
git clone ssh://repo/firewheel/firewheel.git /opt/firewheel
FIREWHEEL’s install.sh will make the remainder of the installation process easy.
However, before running that script, there are several environment variables which should be set.
Set Environment Variables
Our installation script allows users to customize installation of FIREWHEEL.
The default values are found in provision_env.sh, but they can be overwritten by setting the key as an environment variable.
For example, to change the default value of DISCOVERY_PORT, you can use:
export DISCOVERY_PORT=9090
Generally, the defaults are acceptable for most installations.
However PYTHON_BIN is likely to change between environments (especially if not using a virtual environment).
Additionally, on clusters of more than one node, FIREWHEEL_NODES, EXPERIMENT_INTERFACE, and GRPC_THREADS should be set.
Finally, if VLANs cannot be used to route traffic between FIREWHEEL nodes, then USE_GRE should be set to true.
A full list of environment variables used in install.sh is shown below.
- sid
- Description:
The user account who will use FIREWHEEL.
- Default:
"$(whoami)"
- FIREWHEEL_NODES
- Description:
The hostnames of the nodes in the FIREWHEEL Cluster in a space separated list.
- Default:
"$(hostname)"
- HEAD_NODE
- Description:
The hostname of the Control Node.
- Default:
"$(echo $FIREWHEEL_NODES | cut --delimiter ' ' --fields 1)"
- FIREWHEEL_ROOT_DIR
- Description:
For users installing FIREWHEEL from source, this is the root directory of the source code repository.
- Default:
/opt/firewheel
- FIREWHEEL_VENV
- Description:
The location of the default FIREWHEEL virtual environment.
- Default:
${FIREWHEEL_ROOT_DIR}/fwpy
- PYTHON_BIN
- Description:
The path to the Python interpreter.
- Default:
python3
- PIP_ARGS
- Description:
Any additional arguments/options required for Pip.
- Default:
""
- EXPERIMENT_INTERFACE
- Description:
The experimental network interface. That is, which NIC connects all nodes in the FIREWHEEL Cluster.
- Default:
lo
- USE_GRE
- Description:
Whether to use GRE tunnels rather than VLANs to segment traffic for minimega.
- Default:
false
- MM_BASE
- Description:
The location of minimega’s run time files (e.g. VM logs, files, etc.)
- Default:
"/tmp/minimega"
- MM_GROUP
- Description:
The minimega user group.
- Default:
minimega
- MM_CONTEXT
- Description:
The context for finding minimega meshage peers. It should distinguish your minimega instances from any others on the network. See https://www.sandia.gov/minimega/using-minimega/ for more information.
- Default:
${HEAD_NODE}
- MM_INSTALL_DIR
- Description:
The installation directory for minimega.
- Default:
"/opt/minimega"
- DISCOVERY_PORT
- Description:
The HTTP port for the Discovery service.
- Default:
8080
- DISCOVERY_HOSTNAME
- Description:
The hostname for the Discovery service.
- Default:
localhost
- GRPC_HOSTNAME
- Description:
The hostname for FIREWHEEL’s GRPC server.
- Default:
${HEAD_NODE}
- GRPC_PORT
- Description:
The port number to use for FIREWHEEL’s GRPC server.
- Default:
50051
- GRPC_THREADS
- Description:
The number of threads to use for FIREWHEEL’s GRPC server.
- Default:
2
- FIREWHEEL_GROUP
- Description:
The FIREWHEEL’s user group (if any).
- Default:
${MM_GROUP}
- MC_BRANCH
- Description:
The Git branch for the model component repositories that will be cloned.
- Default:
master
- MC_DIR
- Description:
The location of the model component repositories.
- Default:
${FIREWHEEL_ROOT_DIR}/model_components
- MC_REPO_GROUP
- Description:
The URL which contains FIREWHEEL’s model component repositories.
- Default:
https://repo/firewheel/model_components
- DEFAULT_OUTPUT_DIR
- Description:
The default directory for FIREWHEEL’s various logs, cached Helpers, and other outputs.
- Default:
/tmp/firewheel
Running install.sh
Once the configuration options have been set, run install.sh:
cd /opt/firewheel
chmod +x install.sh
./install.sh
This script will create necessary directories, installs and configures FIREWHEEL, and clone our default model component repositories.
The script will optionally install additional FIREWHEEL development dependencies by using either the -d or --development flag:
./install.sh -d
Post Installation Steps
Once the entire cluster has been provisioned, we recommend running the sync command to cache Helpers across the cluster:
firewheel sync
Installing Model Component Repositories
Before running any experiments, you will likely need to clone several Model Component repositories and install them.
For example, you will likely want to install our base and linux repository.
We recommend putting all Model Component repositories in /opt/firewheel/model_components.
Then you can install this location so that FIREWHEEL can leverage those model components:
firewheel repository install /opt/firewheel/model_components
For more information see FIREWHEEL Repositories. For information about available Model Components see available_model_components.
Environment Enhancements
Once FIREWHEEL has been installed, we recommend symbolically linking FIREWHEEL’s CLI to /usr/bin/firewheel to avoid activating the virtual environment for each new terminal session.
sudo ln -s "$(which firewheel)" /usr/bin/firewheel
We also recommend configuring tab-complete for FIREWHEEL’s CLI. If using bash, the simplest approach is:
sudo cp <path-to-FIREWHEEL>/firewheel_completion.sh /usr/share/bash-completion/completions/firewheel
Alternatively, the following command can be added to a .bashrc, .zshrc, etc.
source <path-to-FIREWHEEL>/firewheel_completion.sh
If the FIREWHEEL configuration changes, the tab completion script <path-to-FIREWHEEL>/firewheel_completion.sh may be automatically regenerated at any point.
python -m firewheel.cli.completion.prepare_completion_script