Transfer Files Between CHTC and ResearchDrive

Introduction

This guide shows how to transfer (upload/download) files between CHTC’s file systems and ResearchDrive, a shared data storage service for research at UW Madison. The instructions in this guide may also work for accessing other data services on campus from CHTC that interface with the smbclient tool. For UWDF/ResearchDrive integration, see this page.

Table of Contents

Before you transfer

You will need:

  • Access to your PI’s ResearchDrive
  • A CHTC account

Transfer files

To transfer data between ResearchDrive and CHTC, do the following:

  1. Log in:
    1. If you are transferring files to or from a /staging directory, log in to transfer.chtc.wisc.edu.
    2. If you are transferring files and or from your /home directory, log into your usual submit server (typically ap2001.chtc.wisc.edu or ap2002.chtc.wisc.edu).
  2. Choose a folder: Navigate to the folder in CHTC (/staging or /home) where you would like to transfer files.

  3. Connect to ResearchDrive: Run the following command to connect to ResearchDrive, filling in the username of your PI:

     [alice@server]$ smbclient -k //research.drive.wisc.edu/PI-Name

    Your prompt should change to look like this:

     smb: \>

    Note about NetIDs

    If your CHTC account is not tied to your campus NetID or you are accessing a data storage service that doesn’t use your NetID, you’ll need to omit the -k flag above

  4. Choose a folder, part 2: If you type ls now, you’ll see the files in ResearchDrive, not CHTC. Navigate through ResearchDrive (using cd) until you are at the folder where you would like to get or put files.
  5. Move files: To move files, you will use the get and put commands:

    • To move files from CHTC to ResearchDrive, run:

        smb: \> put filename

    • To move files from ResearchDrive to CHTC, run:

        smb: \> get filename
  6. Finish: Once you are done moving files, you can type exit to leave the connection to ResearchDrive.

Transfer a batch of files

The steps described above work well for transferring a single file, or tar archive of many files, at a time and is best for transferring a few files in a session. However, smbclient also provides options for transferring many individual files in a single command using the * wildcard character.

To transfer multiple files at once, first you must turn off the smbclient file transfer prompt, then use either mget or mput for your file transfer.

  1. Follow steps 1-4 above.

  2. Turn off Prompting. This turns off a prompt that appears for every file you want to download.

     smb: \> prompt

  3. Move files

    • Download with mget. The example below uses the wildcard character * to download a group of files that end with “.fastq.gz” to CHTC.

        smb: \> mget *.fastq.gz

    • Upload with mput. The example below uses the wildcard character * to upload a group of files that end with “.tar.gz” to CHTC.

        smb: \> mput *.tar.gz
HTC guides
HPC guides