Jupyter on OpenOnDemand
Jupyter on OpenOnDemand
A very common task with Open OnDemand is accessing and interacting with Jupyter Notebook-Based Software. In order to get started with Jupyter Notebooks in Open OnDemand, there is a very important setup process that needs to be completed. The process is detailed below:
Installing Jupyter Lab in your Home Directory
The first step to using Jupyter in Open OnDemand is to install the Jupyter Lab library in your home directory. In order to do this, you need to first set up a Conda environment using Anaconda or a Python Virtual Environment using
virtualenv. The Conda Environment method is recommended and descibed here.
Installing JupyterLab using Conda Environments
- From the Open OnDemand dashboard, select "Clusters" from the header bar, and select "HPC Shell Access" from the drop down menu that appears. This will bring up a new tab in your browser with a terminal.
- Load the Anaconda 3.7.3 module by typing the following into your terminal
module load anaconda3.7.3
- IF your terminal upon login does not have (base) prepended to it, like this...
(base) [USER@hpc-login-25 ~]$
...THEN first type the following int your terminal. This will initialize the anaconda module's base environment for your home directory.
conda init bash
...ELSE skip this step.
- Now, type the following into your terminal to create a Conda environment. The actual environment name can be anything you like, but in this example, we'll use "jupyterenv":
conda create -n jupyterenv
- Once the environment creation process has completed, activate the environment by typing the following into your terminal.
conda activate jupyterenv
- Once the environment is active, you will see your terminal change FROM something like this:
(base) [USER@hpc-login-25 ~]$
TO something like this:
(jupyterenv) [USER@hpc-login-25 ~]$
The change in the prompt in parentheses indicates the name of the Conda environment you are currently working in.
- Now that the environment is active, you will be able to use tools like "conda install" to install packages into your environment. We will need this to install JupyterLab for use with Open OnDemand. To do this, type the following into your terminal:
conda install -c conda-forge juptyerlab
- Once the installation is complete, Jupyter will now be ready for you to use in Open OnDemand. Make a note of the location of the Conda environment (should be /gpfs/home/USER/.conda/envs/jupyterenv). Proceed now to the section in this document titled, Starting a Jupyter Session in Open OnDemand
Starting a Jupyter Session in Open OnDemand
Now that Jupyter Lab has been installed into your home directory, you will need to use the location of that environment you made a note of in the last step. To get started, follow these steps:
- Open a web browser and navigate to https://ood.rcc.fsu.edu.
- Here, you will be prompted to sign in with your RCC credentials (username and password). Enter those credentials into the provided textboxes in the prompt which will look something like this:
- Once you have signed in to Open OnDemand, the Dashboard page will appear which will look something like this:
- Click on the Interactive Apps tab on the top bar. This will open a drop-down menu.
- In the drop-down menu opened under the Interactive Apps tab, under the "Servers" heading, will be a button called Jupyter Notebook (the menu will look something like the following image). Click the Jupyter Notebook button.
- This will take you to a "Job setup screen" that looks similar to the following image:
- Fill out the form as follows:
- Set Python Version to python3
- Set Path to Jupyter installation to the location of the jupyter binary (executable) in your home directory. For example, if you installed Jupyter in an environment named juptyerenv (as described in the Installing Jupyter Lab in your Home Directory), then your path will be: /gpfs/home/[YOURFSUID]/.conda/envs/jupyterenv/bin/jupyter
- Set Partition to any queue/partition that you have access to, but the basic ones would be either genacc_q, backfill, or backfill2. For details about queues available on the HPC, refer to our documentation page on this website.
- Set Number of Hours, Number of Nodes, and Number of cores to the maximum values you think you might need for your current work session. Open OnDemand jobs work just like regular Slurm jobs; your job may need to wait in-queue for the scheduler to find resources to run your job. Refer to our documentation on estimating resources for jobs.
- Set Amount of memory requested to the amount of memory you expect to need for the workload (up to 128G). Larger values generally require longer wait times.
- If you need to use GPUs, fill in the Generic resources field as follows. Enter gpu:N, where N is a number from 1 to 4 for how many GPUs you think you need. If you don't need GPUs, then leave that field blank.
- Click the Launch button. You will a screen appear similar to the following:
- Once your job starts and resources are available, this screen will change to the following:
- Click on the blue Connect to Jupyter button. This will open a new browser window which will look something like this (a standard Jupyter Lab window - the items and folders listed will be different for you):
- This window shows your home directory on the HPC.
From here, you should be good to go! Your session will run until the time you specified in Step 7.4 or the maximum time limit for the queue that you chose.