This page describes how to transfer data to and from the HPC and Spear filesystems (GPFS) and Research Archival using SFTP, SCP, or RSYNC.
Choosing a transfer option
There are three basic ways to move data to and from our storage systems:
- Secure File Transfer Protocal (SFTP) - Best when your data transfer is relatively small and will take only a few minutes, or you are connected on-campus via a reliable high-speed network. This method is quick and convenient, but less reliable for long-running transfers.
- Globus - Best when your data transfer will take a long time (at least several hours) or when the connection may be periodically interrupted during transfer. This method requires a little setup ahead of time, but, is robust and appropriate for any data transfer task. Globus is also the fastest transfer method of these three. Read more about using Globus.
- RSYNC or SCP - Best when you are connected on-campus via a reliable high-speed network and your data transfer is part of a scripted process. Globus is still a more reliable option in this case, but RSYNC/SCP doesn't require much configuration on the client-side.
Use export.rcc.fsu.edu instead of hpc-login.rcc.fsu.edu
If you are using SFTP, RSYNC, or SCP, be sure to connect to export.rcc.fsu.edu and not a HPC Login Node. The export nodes have 40Gb/s uplinks to our GPFS storage system and 20Gb/s uplinks to our Archival system. This is more than double the capacity of the HPC Login Nodes, and will make your transfers run faster.
While it is possible to transfer data via the HPC Login Nodes, we discourage this for all but the smallest tasks.
Using SFTP with Cyberduck (Mac/Windows)
CyberDuck is a popular open source SFTP client for Windows and Mac.
Download and install Cyberduck from the Cyberduck website.
Open the Cyberduck application.
Click the "Open Connection" button on the toolbar.
Select "SFTP (SSH File Transfer Protocol)" from the drop-down.
Enter export.rcc.fsu.edu in the Server box.
Enter your RCC system account username and password, and click Connect.
If the application prompts you to allow the fingerprint, click "Allow"
Your files on the server will appear in the CyberDuck window. You can now drag-and-drop files to and from the window to upload/download files.
Using SFTP from Nautilus File Manager (Linux)
Most popular Linux distributions with graphical environments (Ubuntu, RHEL, CentOS, etc) include the Nautilus File Manager. This means that you can connect directly from your system to our servers without installing additional software.
Open up your file manager, and locate the Connect to Server link on the left side of the file browser.
In the dialog box, enter sftp://export.rcc.fsu.edu to connect to the server.
Enter your RCC system account username and password when prompted.
Once logged in, the file system will appear as a location on the left-hand side of the Nautilus File manager, the same as any other file or folder on your computer. You can drag and drop, copy and paste, and interact with the files on the server the same way that you do with files on your computer. Press the eject button to disconnect.
Using SFTP on the Command-Line (Mac/Linux)
Using SFTP from the command-line, you can make use of the sftp command, which comes bundled with most Linux distributions and Mac OS X. You can also use the scp command on Linux. Examples (replace USERNAME with your FSUID or RCC username [these are usually the same]):
# # SFTP # # Connect to the HPC Filesystem (gpfs) $ sftp USERNAME@export.rcc.fsu.edu USERNAME@export.rcc.fsu.edus password: Connected to export.rcc.fsu.edu. sftp> # List files on the server sftp> ls # Copy local file (/tmp/test.txt) to the server sftp> put /tmp/test.txt /tmp/test.txt 100% 88KB 88.5KB/s 00:00 # Copy remote file on server to computer (downloads to current working directory) sftp> get test.txt # Disconnect sftp> exit # # SCP # # Use scp to copy a file immediately to GPFS home directory (useful for scripting transfers) $ scp /tmp/test.txt USERNAME@export.rcc.fsu.edu:~ # Use scp to copy an entire directory to GPFS with -r option $ scp -r ~/Documents/my_research_data USERNAME@hpc-login.rcc.fsu.edu:~ # # RSYNC # # Use Rsync to copy a directory to your GPFS home directory $ rsync -v -a -e ssh /FILES/TO/COPY USERNAME@export.rcc.fsu.edu:~
Learn more about the SFTP command with this guide from DigitalOcean.