Running Python Jobs

Quickstart: Python

Option A (recommended)

Build a container with Python & packages installed inside:

1. How to build your own container
2. Example container recipes for Python
3. Use your container in your HTC jobs

Option B

Use an existing container with a base installation of Python:

1. Choose an existing container. See OSG Base Containers or DockerHub Python Containers.
2. Use the container in your HTC jobs

More information

All CHTC machines have a base installation of Python 3. The exact versions and packages installed, however, can vary from machine to machine. You should be able to include simple python commands in your calculations, i.e., python3 simple-script.py.

If you need a specific version of Python 3 or would like to install your own packages, we recommend that you use a container as described above.

The example recipes provided above for building your own container are intended for python packages that can be installed using python3 -m pip install. Additional software can be installed when building your own container.

For packages that need to be installed with conda install, see the guide on Conda.

Executable

When using a container, you can use a python .py script as the submit file executable, provided that the first line (the “shebang”) in the .py file is

#!/usr/bin/env python3

with the rest of the file containing the commands that you want to run using Python.

Alternatively, you can use a bash .sh script as the submit file executable, and in that file you can use the python3 command:

#!/bin/bash

python3 my-script.py

In this case, remember to include your .py file in the transfer_input_files line of your submit file.