Powered by:
Open Science Grid
Center for High Throughput Computing

Compiling or Testing Code with an Interactive Job

The CHTC has two "build" machines configured for compiling code interactively. Furthermore, any CHTC execute server can be used for testing commands and programs before submitting large batches of non-interactive jobs ('typical' jobs like our "hello-world" example).

Typically, any compilable code to be run in CHTC should be compiled on our compiling servers.

(Note: If you are using a non-CHTC submit server to submit jobs to a non-CHTC HTCondor Pool, you may have different capabilities for submitting jobs interactively that are not covered here.)


  1. Setup
  2. Creating Interactive Submit Files
  3. Submitting and Working Interactively
  4. Exiting

1. Setup

If you've already downloaded the source code for your program (like a Python-4.3.2.tar.gz file or R-3.3.0.tar.gz) then you can skip this step and move to the next step.

If you haven't already downloaded your source code as a tarball, it's a good idea to create one with all necessary code and compiling sources/libraries you need, so that you don't have to list every file and/or directory separately for "transfer_input_files" in the submit file.

For example, you could use something like the following command within a directory where all of your needed files and directories are located:

[alice@submit]$ tar -czvf combined.tar.gz file1 file2 dir1 (etc.)
The above would produce a single file called "combined.tar.gz" that contains all other files and directories indicated at the end of the command.

2. Creating Interactive Submit Files

Create a file called interactive.sub (or any other name you want), and paste in the below lines:

universe = vanilla
# Name the log file:
log = interactive.log

# Name the files where standard output and error should be saved:
output = process.out
error = process.err

# If you wish to compile code, you'll need the below lines. 
#  Otherwise, LEAVE THEM OUT if you just want to interactively test!
+IsBuildJob = true
requirements = (OpSysMajorVer =?= 6) && ( IsBuildSlot == true )
# If you need CentOS 7 (our new operating system), use the following 
# requirements instead of the above line: 
# requirements = (OpSysMajorVer =?= 7) && ( IsBuildSlot == true )

# Indicate all files that need to go into the interactive job session,
#  including any tar files that you prepared:
transfer_input_files = combined.tar.gz, other_file.txt

# It's still important to request enough computing resources. The below 
#  values are a good starting point, but consider your file sizes for an
#  estimate of "disk" and use any other information you might have
#  for "memory" and/or "cpus".
request_cpus = 1
request_memory = 1GB
request_disk =1GB


See the following points for areas that may need to be customized for your program.

A. Transfer Files

Modify the filename for transfer_input_files from the example filenames (combined.tar.gz, other_file.txt), to the name of your source tarball(s) and/or files. Note that this submit file assumes that the source tarball is present in your home directory on the submit server, in the same folder as this interactive submit file.

B. Operating System

CHTC is currently running two different versions of Linux: Scientific Linux 6 and CentOS 7. The above submit file will select for our Scientific Linux 6 build machines by default. If your code needs newer libraries or compilers that are present on CentOS 7 (our newer Linux version), use the second requirements line in the above submit file and comment out the first one.

If compiling code using Matlab 2018a, you will need to use the CentOS 7 build server, not SL 6

Note that code compiled on Scientific Linux 6 will typically run on both Scientific Linux 6 and CentOS 7 servers, but code compiled on version 7 will likely not work on version 6.

If you choose to build on Linux 7, read the below information

By default, jobs in CHTC will run on both Scientific Linux 6 and CentOS 7. If you compiled on CentOS 7, you should ensure that your jobs run only on this operating system by adding a requirement to the submit file you use for submitting your final jobs. See our Operating System Transition guide, section 3, for information about how to do this.

C. Other Files

Modify the filenames for output, error, and log per your preferences.

3. Submitting and Working Interactively

A. Submitting

When you are ready to go, enter the following with your correct submit file indicated instead:

[alice@submit]$ condor_submit -i interactive.sub 
After several minutes (the time it takes for HTCondor to match new jobs to available CHTC execute servers), an SSH connection will be created to an execute server, and the relevant files will have been transfered into the job working directory for you.

B. Working Interactively

Once the job starts, you are able to un-tar any tar files, compile programs, and test commands as you please. If you are following one of our other guides for building Python, R or Matlab, this is where you should go follow those instructions for installing or compiling.

Typical un-tar command:

[alice@build]$ tar -xzvf combined.tar.gz

C. Output Files

HTCondor will ignore any sub-directories created during the course of the job, but wil return any newly created files in the main directory. In order to return your compiled code, typically you will want to make a new tarball of files you want copied back to the submit server, and delete everything else that you don't want.

4. Exiting

To exit as you would for any SSH shell, type exit. Your interactive job will be terminated, and you will return to the location on the submit node from which you submitted the interactive job. Any new files (your new tarball) will have been copied back to this location.