Introduction

Software containerization is a popular solution for environment-level reproducibility. Singularity is a popular software Containerization solution for HPC and Cluster systems such as the one RCC runs. It is largely compatible with Docker and is capable of automatically converting Docker containers to Singularity Containers.

Basic Usage

No module is necessary to use Singularity on HPC. It should be available by default when you log in. In order to run a singularity container, all you need to do is run the following where [OPTIONS] are the command line options for the command you want to run and CONTAINERNAME is the name of the container you want to run (a complete reference for the Singularity commands can be found here).

singularity [OPTIONS] run CONTAINERNAME

Full documentation and examples on Singularity use can be found at The Official Singularity User's Guide.

A Note about Docker

RCC does not support Docker itself on the HPC or Spear clusters. We require that all users wishing to use Docker containers convert them to Singularity containers. Documentation on how to do this is provided below. This is due to security concerns that Singularity addresses.

Converting Docker Containers to Singularity

Excellent official documentation on how to convert Docker containers to Singularity can be found at the Docker-to-Singularity Official Documentation.

1. Log into your home directory (ssh USER@hpc-login.rcc.fsu.edu)
2. Create a folder on your home directory to store the container (name it the same as the name of the software in the container)
   1. mkdir SOFTWARENAME # Replace SOFTWNARENAME with the name of the software in the container
3. Move into that folder (cd SOFTWARENAME)
4. Then run singularity pull docker://REPO/SOFTWARENAME:TAGS
   1. Replace REPO with the name of the repository on DockerHub that the software comes from
   2. Replace SOFTWARENAME with the name of the software you want to run
   3. Replace TAGS with any tags necessary to get the version of the software you need
   4. Example (From Singularity Documentation):  singularity pull docker://syslabsio/lolcow:latest
      1. This pulls a docker container called "lolcow" from the "syslabsio" repository and looks for the version tagged "latest"
      2. This will then automatically convert the docker container to Singularity.
      3. This process is not guaranteed to work all the time but it should work most of the time
5. From here, you should be able to run the container by running singularity run CONTAINERNAME where CONTAINERNAME is the name of the container created from the "singularity pull ..." command above (i.e. singularity run lolcow for the example above).
   1. If there are command line options that the Docker container required, then you will need to refer to the Singularity documentation to convert the Docker commands to their Singularity equivalents (see Singularity Run Reference)

Caveats

• By default, Singularity uses the /tmp directory on the local machine (whichever node or login node you happen to be working on) to dump temporary and cache files generated during the build process.  These usually get deleted after a container is built.  However, for particularly large containers (more than a few GB in total size), you may run into an error like the following:
  • FATAL: While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: writing blob: write /tmp/... : no space left on device
  • If you see this, then it most likely means that /tmp got filled up on the machine you're using and you need to do the following:
    • In the directory you have created for the container (See above), create two subfolders called faketmp and fakecache respectively.  These will be the directories you will need to redirect Singularity's default temporary output to.
    • Then re-assign the SINGULARITY_TMPDIR and SINGULARITY_CACHEDIR environment variables as follows:
      • export SINGULARITY_TMPDIR=${HOME}/path/to/CONTAINERNAME/faketmp
      • export SINGULARITY_CACHEDIR=${HOME}/path/to/CONTAINERNAME/fakecache
      • Here "/path/to/CONTAINERNAME/" should be the full path from the top of your home directory (/gpfs/home/FSUID/) to the directory you created for the containerized software.  CONTAINERNAME should be the name of the software.  For example, using the lolcow example above:
        • ssh USER@hpc-login.rcc.fsu.edu  # USER is your FSUID
        • pwd  # should show "/gpfs/home/USER" where USER is my FSUID
        • mkdir lolcow  # make a folder for the lolcow container
        • cd lolcow
        • mkdir faketmp
        • mkdir fakecache
        • export SINGULARITY_TMPDIR=${HOME}/lolcow/faketmp
        • export SINGULARITY_CACHEDIR=${HOME}/lolcow/fakecache
        • singularity pull docker://syslabsio/lolcow
        • singularity run lolcow
    • This procedure should fix any errors about temporary directories and files.  You may want to delete everything in the "faketmp" and "fakecache" directories periodically as long as your containers don't need them to run so you don't run out of space on your home directory.