We provide scratch space on our HPC file system for writing large amounts of temporary data for running jobs. Scratch space is provided first-come, first-serve and there are no individual user quotas.
Data in the scratch volume is periodically deleted by an automated process. The scratch volume is not for long-term data storage.
What is Scratch Space for?
Sometimes a compute job may generate large amounts of temporary data while the job is running, but that data is then discarded when the job completes. Our scratch volumes exist to accomodate this type of data.
If you expect that your compute job will generate more data than your allocated storage quota, you may wish to write temporary job I/O data to our scratch space.
Scratch Space Etiquette
Scratch space is intended to be temporary storage for reading and writing to and from running jobs. Once your jobs complete, you should move any scratch data you have generated to a permanent storage volume (e.g your home directory), or discard it.
Scratch data will periodically be deleted. Since scratch space is a shared resource, we periodically delete any data at any time that we feel is inhibiting the ability of other RCC users to use the resource.
Currently, scratch space access is provided on a a request basis. Any RCC user can request access by sending us a support request.
Owner-Based Scratch Space
If you are a member of an owner-based group, such as IMB or COAPS, your account may have access to owner-based scratch space. Consult with your faculty sponsor or inquire with us.
Using General Access Scratch Space
If you have been granted access to scratch space on the HPC, then you can locate it as follows:
For example, for HPC user jsmith:
For convenience, you may wish to create a shortcut (symlink) from your home directory to your scratch space:
# Create a symbolic link to your scratch directory [jsmith@hpc-login-39 ~]$ ln -s /gpfs/research/scratch/[YOUR-USERNAME] ~/scratch # From now on, you can use ~/scratch to refer to your scratch space: [jsmith@hpc-login-39 ~]$ ls ~/scratch
Now, you can configure your HPC jobs to read/write to this directory during execution.
Checking Storage Usage and Quotas
You can check storage quotes anytime with the following command:
$ gpfs_quota scratch
The output will be similar to:
Block Limits | File Limits Filesystem type blocks quota limit in_doubt grace | files quota limit in_doubt grace Remarks research FILESET 6.829T 15T 15T 278.1G none | 504139 0 0 34639 none DSS01.local
This output shows that 6.8TB are in use out of 15TB available.