HTC Computing Guides

Get started

CHTC /staging and /projects next steps System overview Roadmap to getting started Log into CHTC Quick reference: Basic shell commands Quick reference: HTCondor commands

Submit jobs

Submit jobs using HTCondor

Job submission basics Submit multiple jobs Submit multiple jobs in different directories Submit an interactive job Monitor your job

Arguments and shell scripting

Basic scripting and job submission with arguments Practice: Pass arguments from the submit file to the executable script Practice: Pass multiple arguments to multiple jobs with one submit file

Manage data

Transfer small input and output Transfer large input files via Squid Transfer large input and output files via /staging Check disk quota and usage Transfer files to/from CHTC

Local computer ResearchDrive Globus Private GitHub repo

Software

Overview and policies Containers

Build a software environment with containers

What are containers? Apptainer/Singularity

Practice: Build a container

Docker

Explore and test Docker containers

Recipes repository

Convert Docker images to Apptainer/Singularity images Use Apptainer containers Use Docker containers

Compile software in an interactive job Licensed software and modules Use software available in modules HTC modules transition to EL9

Workflows

Checkpoint jobs DAGMan workflow manager

Submit workflows with DAGman Practice: Run a simple DAGMan workflow Practice: Run an diamond DAGMan workflow

Special use cases

Use GPUs Scale beyond local HTC capacity Submit high memory jobs Machine learning jobs Custom Linux versions

Troubleshooting

Status page Known issues Windows/Linux incompatibility Get help

External Resources

HTCondor manual Software Carpentry UW-Madison software training for students

Guide Sections

Get started Submit jobs Manage data Software Workflows Special use cases Troubleshooting External Resources