Storage Overview

This page describes the storage systems available at RCC and how to use them.

At A Glance

  • Panasas - The primary parallel storage system available on the High Performance Computing Cluster.
    • Each user is entitled 100gb of personal storage in the General Access volume
    • Certain research groups have access to dedicated volumes with additional storage capacities.
    • Each volume resides at a different path in the system.
    • Refer to the Manage Account section of the website to see which Panasas volumes you have access to.
    • Transfer data to and from Panasas using Globus or SFTP.
  • Lustre - The primary parallel storage system available on the Spear Cluster, and used for various other systems.
    • Each user is entitled to 100gb of personal storage in the General Access volume
    • Certain research groups have access to dedicated volumes with additional storage capacities.
    • Each volume resides at a different path in the system.
    • Refer to the Manage Account section of the website to see which Lustre volumes you have access to.
    • Transfer data to and from Lustre using Globus or SFTP
  • NoleStor Archival Storage - An object-based filesystem (similar to Amazon S3) for integrating with research applications
    • Archival storage is available for purchase at a lower rate than Parallel (Panasas/Lustre) storage.
    • Use object-store API to fetch and retrieve objects in the system.
    • Implementation of the DataDirect Web Object Scalar System
    • Ideal for custom applications with need for large unstructured data storage.  Examples:
      • Storage of unstructured data like image, video, or raw datafiles.
      • Storage of backup files, databasee dumps, and log files,
      • Large data sets that are ideally stored in a flat file-structure.  Object storage does not utilize POSIX filesystem organization tools such as directories, but does allow custom metadata per object/file.

Accessing Filesystems

You can access the Lustre and Panasas by logging into Spear and the HPC, respectively.  You can also access each filesystem to view and transfer files using an SFTP client or Globus.

To use the NoleStor system, you must use either the HTTP REST API, or use a WOS client library (available for Java, C, C++, PHP).  Contact us to learn more.

Setting up Lustre Space

Unlike Panasas, Lustre space is not configured automatically.  If you wish to activate your Lustre space, simply login to any Spear server.  Logging in to Spear will trigger a script to provision your Lustre space.