Open OnDemand

Open OnDemand provides easy, remote web access to supercomputers and computing clusters, without the need for a command-line interface. Instead of accessing the hydra and ghidorah clusters via your terminal, Open OnDemand provides a user interface for accessing and modifying files and submitting jobs. No installation is required -- everything can be accessed within the browser.

Getting Started

To access Open OnDemand, login here with your StatDS username and password. This is the same username and password you would use to access the clusters via ssh on your terminal.

After logging in, you should be redirected to your file system. You can navigate through your directories just as you would on your desktop. A nice feature of Open OnDemand is you can automatically upload and download files from your desktop to the department's file system; the secure file transfer is done automatically in a user-friendly manner.

Editing Files

To create a new file or directory, simply click "New File" or "New Directory" in the upper right hand corner of the screen.

To edit existing files, navigate to the file in the file system. Click on the dropdown three vertical dots to the right of the file's name. An options list will appear that includes a "View" mode and an "Edit" mode. Clicking view will allow you to view the contents of the file without manipulating them. Edit will redirect you to another tab where you can directly edit the contents of the file. Click the "Save" button on the lefthand side of the screen to save edits; when this button is grayed out, it means there are no additional changes to save.

Accessing the Clusters

To access the clusters via the terminal, navigate to the "Clusters" drop down in the top left of the screen, and select "Statistics and Data Science Shell Access." The shell access is built for both the ghidorahs and the hydras. Beneath the cluster selection, you can view the system status which tells you how many nodes, processors, and GPUs are currently available, as well as how many jobs are running and queued. This will help inform how long a job you've created might take to run on the system/when it will begin running.

Submitting and Tracking Jobs

Default Template

To submit or compose a job, navigate to the "Jobs" drop down in the top left of the screen, and select "Job Composer." This will take you to the job composer page, where you can select the "New Job" drop down. To start from the default template, just click "From Default Template" and in the right hand panel, scroll down to where the .sh script is shone and click "Open Editor" to edit the script itself. Once you are ready to submit the job, simply click the job in your list of potential jobs, and click submit. The job will show a status of "Queued," "Running," or "Completed" and you can view output files in the right hand panel once it has completed running.

Hydras and Ghidorah Templates

If you click "From Template," you can choose to submit a job on the hydras or ghidorahs. Simply pick the template you want to use -- either Hydra Job for the hydras or Ghidorah Job for the ghidorahs -- and click "Create New Job." You can then modify the submit.sh script as before to select which hydra or ghidorah you want to use as follows:

#!/bin/sh
#SBATCH --partition Ghidorahs

# #Select a specific Hydra node to run job on.
# #Un-comment the line below and replace ghidorahXX with the appropriate ghidorah server ~ ghidorah1 in this case.
#SBATCH --nodelist=ghidorah1

# #Number of tasks.
#SBATCH --ntasks 1

# #Number of CPUs per tasks.
#SBATCH --cpus-per-task 1

# #Amount of RAM
#SBATCH --mem=4G

# #Output file for script.
#SBATCH --output=%j_output.txt

# #Error file for script.
#SBATCH --error=%j_error.txt

python3 examplepython.py

The #SBATCH --nodelist=ghidorah1 selects the 1st ghidorah to run the python script on. Change this to #SBATCH --nodelist=hydra1 for the hydras and change the first SBATCH line to Hydras (#SBATCH --partition Hydras).

Submitting a Script from a Specified Path

You can also specify a path to a job you have already created. Click "From Specified Path" and enter the source path, i.e. the path to the directory containing your .sh file. Once you have done that, you can specify the name of the script under Script name, i.e. the name of the .sh file in the directory you've navigated to. You can then click "Run" just as you have with your other jobs.