Practice Building Apptainer Containers on CHTC

Container images are one of the most useful ways to make HTC workflows portable and reproducible on CHTC. This guide walks through the basics of building Apptainer containers on CHTC with two examples - a conda environment and installing Linux system packages.

Table of Contents

Before you start

Before getting started, you’ll need to make sure you have the following:

  • A CHTC account on ap2001 or ap2002 or another HTC node.
  • A /staging directory with at least 10 GB of free space

Overview

To build a container:

  1. Write a definition file (.def).
  2. Write a build job submit file.
  3. Submit an interactive build job.
  4. Run apptainer build inside the interactive job to generate a .sif container image file.
  5. Test the container locally in the interactive build session.
  6. Move the .sif file to your staging folder.
  7. Use the image in your job.

We will go through these seven steps for each example below.

Example A — Conda install (numpy and pandas)

This example shows how to include Python and a conda-installed package in your container.

Step 1: Write the container definition file: conda-np-pd.def

For this example, we will create a container that includes numpy and pandas installed via conda. We will start from the CHTC Recipes repo example for conda base installations. You can find the template definition file in the recipes repo. This starts from a miniforge base and installs numpy and pandas into the base conda environment.

Create an apptainer definition file called conda-env.def:

Bootstrap: docker
From: condaforge/miniforge3:latest

%post
    conda install -y -c conda-forge numpy pandas

Steps 2-3: Create the container build job

Create an apptainer build job submit file (build-conda.sub):

log = build-conda.log

transfer_input_files = conda-env.def

+IsBuildJob = True

request_cpus = 8
request_memory = 16GB
request_disk = 30GB

queue

Submit the build job as an interactive session:

condor_submit -i build-conda.sub

Step 4: Build the Apptainer container: conda-np-pd.sif

Once interactive shell starts, build the apptainer container using the apptainer build command:

apptainer build conda-np-pd.sif conda-env.def

Step 5: Test the container within the build session

Test your newly build apptainer container using the apptainer shell command:

apptainer shell conda-np-pd.sif
python -c "import numpy; import pandas;"

Type exit to leave the running container:

exit

Step 6: Store your apptainer container in /staging:

Move the .sif file to your staging folder. Replace b/bbadger with your username!!

mv conda-np-pd.sif /staging/b/bbadger/

Type exit again to leave the interactive job.

exit

Step 7: Run a job

See Running Jobs Using These Containers

Example B — Install via apt-get

This example shows installing system packages directly inside the container using apt-get.

Step 1: Write the container definition file: apt-cowsay.def

For this example, we will create a container that includes cowsay installed via the ubuntu apt-get package manager. We will start with an existing Ubuntu 22.04 Docker Hub container. We will specify this base image using the Bootstrap: docker and From: ubuntu:22.04 definition file syntax.

Create an apptainer definition file called apt-cowsay.def:

Bootstrap: docker
From: ubuntu:22.04

%post
    apt-get update -y
    apt-get install -y cowsay

Base image is ubuntu:22.04 and system packages install at build time.

Steps 2-3: Create the container build job

Create an apptainer build job submit file (build-apt.sub):

log = build-apt.log

transfer_input_files = apt-cowsay.def

+IsBuildJob = True

request_cpus = 8
request_memory = 16GB
request_disk = 30GB

queue

Submit the build job as an interactive session:

condor_submit -i build-apt.sub

Step 4: Build the Apptainer container: apt-cowsay.sif

Once interactive shell starts, build the apptainer container using the apptainer build command:

apptainer build apt-cowsay.sif apt-cowsay.def

Step 5: Test the container within the build session

Test your newly build apptainer container using the apptainer shell command:

apptainer shell apt-cowsay.sif 
cowsay "Hello from apt!"

Type exit to leave the running container:

exit

Step 6: Store your apptainer container in /staging:

Move to the .sif file to your staging folder. Replace b/bbadger with your username!!

mv apt-cowsay.sif /staging/b/bbadger

Type exit again to leave the interactive job.

exit

Step 7: Run a job

See Running Jobs Using These Containers

Running jobs using these containers

Once built, your HTCondor job run.sh can simply invoke the installed tools:

Example run.sh

#!/bin/bash
echo "Running inside container"
cowsay "CHTC says hi"

Example run.sub

container_image = osdf:///chtc/staging/path/to/my-container.sif

executable = run.sh
log = job.log
error = job.err
output = job.out

request_cpus = 2
request_memory = 4GB
request_disk = 8GB

queue

Remember to use the osdf:/// protocol whenever using containers /staging

The OSDF protocol allows HTCondor to more efficiently transfer large container files stored in /staging, improving job startup times and reducing unnecessary data movement.

Note If your container is stored elsewhere (e.g., your group /staging/groups/ directory), please use the file:/// protocol instead.

For more information about these protocols, see the CHTC - Manage large data in /staging.

Common tips & best practices

  • Always place the .sif in /staging to leverage CHTC’s storage and avoid unnecessary transfers.
  • Request enough disk space in your submit file to transfer the container.
  • Use the existing CHTC Recipes repo as templates for common installations.
HTC guides