[an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive]
These guides were replaced by a new, updated process on Monday, October 21, 2019. If you're starting a new project, we recommend using the updated process instead of the guides below. See this page for the new guides.

The CHTC has two "build" machines configured for compiling code interactively. Furthermore, any CHTC execute server can be used for testing commands and programs before submitting large batches of non-interactive jobs ('typical' jobs like our "hello-world" example).

Typically, any compilable code to be run in CHTC should be compiled on our compiling servers.

(Note: If you are using a non-CHTC submit server to submit jobs to a non-CHTC HTCondor Pool, you may have different capabilities for submitting jobs interactively that are not covered here.)

Contents

  1. Setup
  2. Creating Interactive Submit Files
  3. Submitting and Working Interactively
  4. Exiting

1. Setup

If you've already downloaded the source code for your program (like a Python-4.3.2.tar.gz file or R-3.3.0.tar.gz) then you can skip this step and move to the next step.

If you haven't already downloaded your source code as a tarball, it's a good idea to create one with all necessary code and compiling sources/libraries you need, so that you don't have to list every file and/or directory separately for "transfer_input_files" in the submit file.

For example, you could use something like the following command within a directory where all of your needed files and directories are located:

[alice@submit]$ tar -czvf combined.tar.gz file1 file2 dir1 (etc.)
The above would produce a single file called "combined.tar.gz" that contains all other files and directories indicated at the end of the command.

2. Creating Interactive Submit Files

Create a file called interactive.sub (or any other name you want), and paste in the below lines:

universe = vanilla
# Name the log file:
log = interactive.log

# Name the files where standard output and error should be saved:
output = process.out
error = process.err

# If you wish to compile code, you'll need the below lines. 
#  Otherwise, LEAVE THEM OUT if you just want to interactively test!
+IsBuildJob = true
requirements = (OpSysMajorVer =?= 7) && ( IsBuildSlot == true )

# Indicate all files that need to go into the interactive job session,
#  including any tar files that you prepared:
transfer_input_files = combined.tar.gz, other_file.txt

# It's still important to request enough computing resources. The below 
#  values are a good starting point, but consider your file sizes for an
#  estimate of "disk" and use any other information you might have
#  for "memory" and/or "cpus".
request_cpus = 1
request_memory = 1GB
request_disk =1GB

queue

See the following points for areas that may need to be customized for your program.

A. Transfer Files

Modify the filename for transfer_input_files from the example filenames (combined.tar.gz, other_file.txt), to the name of your source tarball(s) and/or files. Note that this submit file assumes that the source tarball is present in your home directory on the submit server, in the same folder as this interactive submit file.

B. Operating System

CHTC primarily runs the Linux flavor CentOS 7. If you need or want to compile on an earlier version of Linux for a specific reason, we support one build server running Scientific Linux 6. You can see how to request a server with Scientific Linux 6 in our Operating System Transition guide.

C. Other Files

Modify the filenames for output, error, and log per your preferences.

3. Submitting and Working Interactively

A. Submitting

When you are ready to go, enter the following with your correct submit file indicated instead:

[alice@submit]$ condor_submit -i interactive.sub 
After several minutes (the time it takes for HTCondor to match new jobs to available CHTC execute servers), an SSH connection will be created to an execute server, and the relevant files will have been transfered into the job working directory for you.

B. Working Interactively

Once the job starts, you are able to un-tar any tar files, compile programs, and test commands as you please. If you are following one of our other guides for building Python, R or Matlab, this is where you should go follow those instructions for installing or compiling.

Typical un-tar command:

[alice@build]$ tar -xzvf combined.tar.gz

C. Output Files

HTCondor will ignore any sub-directories created during the course of the job, but wil return any newly created files in the main directory. In order to return your compiled code, typically you will want to make a new tarball of files you want copied back to the submit server, and delete everything else that you don't want.

4. Exiting

To exit as you would for any SSH shell, type exit. Your interactive job will be terminated, and you will return to the location on the submit node from which you submitted the interactive job. Any new files (your new tarball) will have been copied back to this location.

[an error occurred while processing this directive]