Computing Guides

Below is a list of guides for some of the most common tasks our users need to carry out as they begin and continue to use the resources at the CHTC. Some of these are general computing solutions; others are specific to HTCondor or to the configuration of CHTC computing resources.

Guides will be added to the list as we can provide them. Please contact us (email at bottom of page) if you find any of the information to be incorrect.

User Expectations

HTC Documentation

Basics and Policies

CHTC /staging and /projects Next Steps HTC System Overview Connect to CHTC Transfer Files between CHTC and your Computer HTC Getting Started Roadmap Automate CHTC Log In Transfer Files Between CHTC and ResearchDrive Use Globus to Transfer Files to and from CHTC Access a Private GitHub Repository Remotely Check Disk Quota and Usage

Job Submission

Practice: Submit HTC Jobs using HTCondor Learn About Your Jobs Using condor_q Submitting Multiple Jobs Using HTCondor Submitting Multiple Jobs in Different Directories Basic Scripting and Job Submission with Arguments Practice: Passing Arguments from the Submit File to the Executable Script Practice: Passing Multiple Arguments to Multiple Jobs with One Submit File Overview: Submit Workflows with HTCondor's DAGMan Simple Example of a DAGMan workflow Intermediate DAGMan: Uses and Features

Handling Data in Jobs

Transfer Small Input and Output Transfer Large Input Files Via Squid Use Large Input and Output Files Via Staging

Software Solutions

Overview: How to Use Software Compile Software in an Interactive Job Use Software Installed in a Docker Container Convert Docker images to Apptainer images Build a Docker Container Image Use Licensed Software Use Apptainer Containers Building an Apptainer Container Advanced Apptainer Example - SUMO

Special Use Cases

Use GPUs Run Machine Learning Jobs Scale Beyond Local HTC Capacity Checkpointing Jobs Submit High Memory Jobs HTC Modules Transition to EL9 Use Software Available in Modules Use Custom Linux Versions in CHTC

Troubleshooting

Windows / Linux Incompatibility Explore and Test Docker Containers Known Issues on the HTC

HPC Documentation

Basics and Policies

HPC System Transition to a New Linux Version (CentOS Stream 9) CHTC /staging and /projects Next Steps HPC Cluster Overview Connect to CHTC Transfer Files between CHTC and your Computer Automate CHTC Log In Transfer Files Between CHTC and ResearchDrive Access a Private GitHub Repository Remotely Known Issues on the HPC

Job Submission

Submitting and Managing Jobs Using SLURM Reviewing Job Information Using SLURM

Software Solutions

Use Software on the HPC Cluster Set Up Spack on HPC Install Software Using Spack Create Custom Modules Using Spack Recreating Spack Installs on a New Operating System Using Apptainer Containers on HPC Building an Apptainer Container

External Documentation