Model Component INSTALL file

Some Model Components may require additional Python packages to be installed within FIREWHEEL’s virtual environment or for data to be downloaded. In this case, the Model Component can have an INSTALL file, which can be any executable script (as denoted by a shebang line).

When creating an INSTALL file, it is critical that users will download the exact same data that was originally intended by the Model Component creators. If the data/packages differ, than there is a strong possibly that the experimental outcomes will differ and could produce unintended consequences. Therefore, we strongly recommend that MC creators link to exact versions of software to download, rather than an automatically updating link. For example, if the MC was supposed to install a GitLab runner:

# This will automatically get the latest URL, this is BAD
wget https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

# This will get a specific version, this is GOOD!
wget https://s3.amazonaws.com/gitlab-runner-downloads/v11.4.2/binaries/gitlab-runner-linux-386

In addition to linking to a specific version, we strongly recommend also using a SHA-256 checksum to verify the downloaded file.

When a repository is installed via the repository install Helper, users have the option to can automatically run each MCs INSTALL script using the -s flag (see repository install for more details).

INSTALL Template

The file src/firewheel/control/utils/templates/INSTALL.template contains a template for a Bash-based INSTALL file. When users use the mc generate Helper, this file is automatically added to the MC directory. The current template is shown below.

Listing 1 INSTALL template

#!/bin/bash

#######################################################
# This is a sample install file for {{mc_name}}.
# This file can be used to perform one-time actions
# which help prepare the model component for use.
#
# Common uses of INSTALL files include downloading
# VM Resources from the Internet and installing new
# Python packages into FIREWHEEL's virtual environment.
#
# NOTE: When you are creating these files, it is
# imperative that specific versions of software are
# used. Without being as specific as possible,
# experimental results will **NOT** be repeatable.
# We strongly recommend that any changes to software
# versions are accompanied by a warning and new model
# component version.
#######################################################

# Create a flag for verifying installation
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
INSTALL_FLAG=$SCRIPT_DIR/.{{mc_name}}.installed

#######################################################
# Checking if there this script has already been complete.
#######################################################
function check_flag() {
    if [[ -f "$INSTALL_FLAG" ]]; then
        echo >&2 "{{mc_name}} is already installed!"
        exit 117;  # Structure needs cleaning
    fi
}


#######################################################
# Install python packages into the virtual environment
# used by FIREWHEEL. This takes in an array of packages.
#######################################################
function install_python_package() {
    pkgs=("$@")
    for i in "${pkgs[@]}";
    do
        python -m pip install "$i"
    done
}


#######################################################
# Download using wget and then checksum the downloaded files.
#
# It is important to verify that the downloaded files
# are the files are the same ones as expected.
# This function provides an outline of how to checksum files,
# but will need to be updated with the specific hashes/file names
# that have been downloaded.
#
# This function assumes that the passed in hashes are SHA-256
#######################################################
function wget_and_checksum() {
    downloads=("$@")
    # Uses 2D arrays in bash: https://stackoverflow.com/a/44831174
    declare -n d
    for d in "${downloads[@]}";
    do
        wget "${d[0]}"
        echo "${d[1]}  ${d[2]}" | shasum -a 256 --check || return 1
    done
}


#######################################################
# A function to help users clean up a partial installation
# in the event of an error.
#######################################################
function cleanup() {
    echo "Cleaning up {{mc_name}} install"
    # TODO: Cleanup any downloaded files
    # rm -rf file.tar
    rm -rf $INSTALL_FLAG
    exit 1
}
trap cleanup ERR

# Start to run the script

# Ensure we only complete the script once
check_flag

#######################################################
# Uncomment if there are Pip packages to install
# `pip_packages` should be space separated strings of
# the packages to install
#######################################################
# pip_packages=("requests" "pandas")
# install_python_package "${pip_packages[@]}"


#######################################################
# Uncomment if there is data/VM resources/images to download.
# `file1`, `file2`, etc. should be space separated strings of
# (URL SHASUM-256 FILENAME).
#
# We recommend that explicit versions are used for all Images/VMRs to prevent
# possible differences between instances of a given Model Component.
# Please be mindful of the software versions as it can have unintended
# consequences on your Emulytics experiment.
#
# We require checksums of the files to assist users in verifying
# that they have downloaded the same version.
#######################################################
# Be sure to use SHA-256 hashes for the checksums (e.g. shasum -a 256 <file>)
# file1=("url1" "e0287e6339a4e77232a32725bacc7846216a1638faba62618a524a6613823df5" "file1")
# file2=("url2" "53669e1ee7d8666f24f82cb4eb561352a228b1136a956386cd315c9291e59d59" "file2")
# files=(file1 file2)
# wget_and_checksum "${files[@]}"
# echo "Downloaded and checksummed all files!"


#######################################################
# Add any other desired configuration/packaging here
#######################################################
echo "The {{mc_name}} INSTALL file currently doesn't do anything!"

# Set the flag to notify of successful completion
touch $INSTALL_FLAG