Open OnDemand
Open OnDemand provides easy, remote web access to supercomputers and computing clusters, without the need for a command-line interface. Instead of accessing the hydra and ghidorah clusters via your terminal, Open OnDemand provides a user interface for accessing and modifying files and submitting jobs. No installation is required -- everything can be accessed within the browser.
Getting Started
To access Open OnDemand, login here with your StatDS username and password. This is the same username and password you would use to access the clusters via ssh on your terminal.
After logging in, you should be redirected to your file system. You can navigate through your directories just as you would on your desktop. A nice feature of Open OnDemand is you can automatically upload and download files from your desktop to the department's file system; the secure file transfer is done automatically in a user-friendly manner.
Editing Files
To create a new file or directory, simply click "New File" or "New Directory" in the upper right hand corner of the screen.
To edit existing files, navigate to the file in the file system. Click on the dropdown three vertical dots to the right of the file's name. An options list will appear that includes a "View" mode and an "Edit" mode. Clicking view will allow you to view the contents of the file without manipulating them. Edit will redirect you to another tab where you can directly edit the contents of the file. Click the "Save" button on the lefthand side of the screen to save edits; when this button is grayed out, it means there are no additional changes to save.
Accessing the Clusters
To access the head node via the terminal, navigate to the "Clusters" drop down in the top left of the screen, and select "Statistics and Data Science Shell Access." The shell allows you to access your files within a terminal on the head compute node. Do not run jobs within this terminal. Beneath the cluster selection, you can view the system status which tells you how many nodes, processors, and GPUs are currently available, as well as how many jobs are running and queued. This will help inform how long a job you've created might take to run on the system/when it will begin running.
Submitting and Tracking Jobs
Default Template
To submit or compose a job, navigate to the "Jobs" drop down in the top left of the screen, and select "Job Composer." This will take you to the job composer page, where you can select the "New Job" drop down. To start from the default template, just click "From Default Template" and in the right hand panel, scroll down to where the .sh script is shone and click "Open Editor" to edit the script itself. Once you are ready to submit the job, simply click the job in your list of potential jobs, and click submit. The job will show a status of "Queued," "Running," or "Completed" and you can view output files in the right hand panel once it has completed running.
Hydra, Leviathan, and Ghidorah Templates
If you click "From Template," you can choose to submit a job on the hydras, leviathans, or ghidorahs. Simply pick the template you want to use -- either Hydra Job for the hydras or Ghidorah Job for the ghidorahs -- and click "Create New Job." You can then modify the submit.sh script as before to select which server you want to use as follows:
#!/bin/sh
#SBATCH --partition Ghidorahs
# #Select a specific Hydra node to run job on.
# #Un-comment the line below and replace ghidorahXX with the appropriate ghidorah server ~ ghidorah1-3.
# #SBATCH --nodelist=ghidorahx
# #Output file for script.
#SBATCH --output=%j_output.txt
# #Error file for script.
#SBATCH --error=%j_error.txt
python3 examplepython.pyThe #SBATCH --nodelist=ghidorah1 selects the 1st ghidorah to run the python script on. Change this to #SBATCH --nodelist=hydra1 for the hydras and change the first SBATCH line to Hydras (#SBATCH --partition Hydras).
Submitting a Script from a Specified Path
You can also specify a path to a job you have already created. Click "From Specified Path" and enter the source path, i.e. the path to the directory containing your .sh file. Once you have done that, you can specify the name of the script under Script name, i.e. the name of the .sh file in the directory you've navigated to. You can then click "Run" just as you have with your other jobs.
Limiting Resource Usage
Limiting your resource usage on servers can help prevent server locks. Use a submit script, similar to the one below, to limit the number of CPU cores and RAM.
#!/bin/sh
#SBATCH --partition Leviathans
#SBATCH --nodelist=leviathan3
# #Output file for script.
#SBATCH --output=%j_output.txt
# #Error file for script.
#SBATCH --error=%j_error.txt
# # Requests number of CPUs per task
#SBATCH --ntasks=1
#SBATCH --cpus-per-task 30
# # Request amount of memory (RAM)
#SBATCH --mem=110G
python3 examplepython.pyWright GPU Cluster
Open OnDemand is also available for the Wright GPU cluster. Documentation can be found here.
