Data Transfer with Globus

All RCC storage systems are configured as Globus endpoints to facilitate the transfer of data. Globus can be used in a number of different ways:

  • Transfer large amounts of data between RCC storage systems (e.g. Archival to HPC/GPFS)
  • Transfer large amounts of data from an RCC storage system to a personal computer with Globus Connect Personal
  • Transfer large amounts of data between RCC storage systems and storage systems at other institutions
  • Publish and curate data sets with Globus Data Publication
  • Set up automated transfers with the Globus CLI utility or your own application with the Globus APIs As an alternative to SFTP, Globus is a more fully-featured and reliable tool for transferring large datasets.

Setting up a Globus Account and Endpoint

Before you can transfer data to or from an RCC storage system, you must sign up for a Globus account. A Globus Connect Personal endpoint is necessary to transfer data to and from your personal computer.

  1. Visit http://globus.org and sign up for an account. Then sign in.
  2. Browse to the Manage Endpoints screen.
  3. Click the Add Globus Connect Personal link.
  4. When prompted, create an Endpoint name, and click Generate Setup Key
  5. Take note of the Setup Key when it appears.
  6. Click the appropriate client download button for your operating system.
  7. Install the client and follow the on-screen instructions. You'll need your setup key for this step.

Transferring Data

Once you have setup an account and endpoint, you can transfer data anytime.

  1. With the Globus Client running, browse to the Tranfser Files screen at http://globus.org.
  2. On the right-hand side of the screen, enter the name of your endpoint (or click the "..." button to find your endpoint in a list)
  3. Select the path in your file tree that you wish to transfer your files to or from.
  4. On the left-hand site, enter the desired endpoint (e.g. fsurcc#gpfs_home).
  5. In the authorization dialog box that appears, enter your RCC username and password. You do not need to alter any other fields in this dialog box.
  6. You will see the root of the filesystem appear. If you wish to see your home directory, place a tilde (~) in the Path box, and press Go.
  7. Select the files and folders you wish to transfer. Then, click the large blue arrow in the direction that you wish to transfer files.
  8. Your files will now be queued for transfer. You can monitor the progress of your transfer by viewing the View Transfer Activity Page on the Globus website. For more information about using Globus, refer to the official documentation.

RCC Endpoints

All RCC storage systems are available as Globus endpoints. You may find these endpoints by searching for endpoints beginning with fsurcc# or by entering their full names:

Globus CLI

Globus provides a cross-platform CLI utility for initiating transfers and managing endpoints from the command line or via automated scripts. See the Globus CLI documentation for installation and usage instructions.

Globus APIs

Globus provides both HTTP and Python APIs for integrating Globus tasks into your own applications. See the Globus APIs documentation for more information.

Getting Help

If you still need help with Globus, please refer to the Globus documentation or submit a support ticket to the RCC staff.