Using CHTC's HTC Submit Servers
If you have not yet requested a CHTC account and met with a
Research Computing Facilitator, please fill out
Recently created CHTC accounts will use our submit server submit-1.chtc.wisc.edu.
Access to other submit servers is granted
for specific purposes and will have been indicated to you by CHTC staff.
Connecting to CHTC Submit Servers
When your CHTC account is active, you can connect to one of our submit servers
with an SSH connection ("Putty" for Windows, putty.org; "Terminal" for Linux/Max).
Note, however, that our submit servers only accept connections from campus
networks. Off-campus connections have been disabled for security purposes.
If you need to connect from off campus, you can first SSH to a computer in your department, and then SSH to our
submit server. You may also be able to use a Virtual Private Network (VPN) to join the campus network when working off-campus.
DoIT provides information on using a VPN.
We do not back up data on CHTC submission points, and users
should treat CHTC compute systems as temporary storage locations for
active, currently-queued computational work. Users should remove data from
CHTC systems upon completion
of a batch computational work and keep copies of all essential files in a non-CHTC
location. CHTC staff reserve the right to delete data from any CHTC file systems at
at any time, to preserve performance, and are not responsible for data loss for file
system corruption, which are possible, especially in the absence of back-ups.
Submit Server User Policies
Because our submit servers are shared by many CHTC users, anyone with an
account on CHTC's submit servers should follow the below policies, in order
to keep the submit servers functional and usable by everyone:
- Do not share accounts. We are happy to make new a new account for any user. Sharing accounts
is against campus policy and, therefore, CHTC policy. Accounts that we notice being shared will be immediately disabled
and a meeting with the PI (faculty advisor) will be necessary to reinstate the account.
- Do not run programs on the submit server. The submit server
is kept plenty busy just managing HTCondor jobs. Computational work,
like compiling code, should be submitted as HTCondor jobs. See our
for the best compiling practices. Small scripts and commands (to compress data,
create directories, etc.) that run within a
few minutes on the submit server are okay, but their use should be minimized when possible.
- Jobs with long runtimes. There is a default run limit of 72 hours for each job
queued in the HTC System, once it starts running. Jobs longer than this will be placed in
HTCondor's "hold" state. If your jobs will be longer, please
email us, and we'll help you to determine the best solution.
- Submitting many jobs from one submit file. HTCondor is designed
to submit thousands of jobs from one submit file. However, submitting too many
jobs at once provides no extra benefit to the submitter and can make the queue
work more slowly. If you are submitting over 10,000 jobs per submit file or have
submitted over 50,000 jobs at once as a single user,
please email us as we have strategies to submit
that many jobs in a way that will ensure you have as many jobs running
as possible without also compromising the queue.
- There is a 20 GB disk quota in your
directory, by default. You can track your use of disk space, and your quota
value, using our Quota
Guide. If you need more space for concurrent work, please send an email
- Submitting jobs with "large" files: Please do not
submit jobs that require single input files larger than 100 MB using
HTCondor's normal file transfer mechanism ("transfer_input_files"),
or that produce output files greater than a few GB. For such jobs
with large files, please see our guide on
Options, and contact us to make arrangements.
- No scheduled jobs. Local execution and job
submission should be manual, or use a DAG. Do not use cron to
schedule jobs, and do not submit jobs with a script that uses "sleep"
or otherwise doesn't complete while you are logged in. Contact us
if you need job scheduling.