Slurm - Revision history

Ajshen at 02:14, 29 June 2026

2026-06-29T02:14:11Z

← Older revision		Revision as of 02:14, 29 June 2026
Line 1:		Line 1:

	[[Category:Wright Cluster]]		[[Category:Wright Cluster]]
			[[Category:PSC]]

	Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapidly as possible. Because this system manages how you are able to use the resources and when your job is run, it requires you to know additional information before you submit your job. Additionally, given that the PSC uses slurm, one should it as a tool for when you have a really good understanding of your code and have attempted to make it efficient, especially since you’re often constrained by the number of core-hours you use. To submit jobs you’ll need to have 1. an understanding of the time it will take to run your job (actually an over-estimate of such things) [see R and python references on approaches to estimate this - to do.] 2. and (to a lesser extent) the amount of storage and RAM you will be using for the job (this varies across what type of access you have at PSC).		Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapidly as possible. Because this system manages how you are able to use the resources and when your job is run, it requires you to know additional information before you submit your job. Additionally, given that the PSC uses slurm, one should it as a tool for when you have a really good understanding of your code and have attempted to make it efficient, especially since you’re often constrained by the number of core-hours you use. To submit jobs you’ll need to have 1. an understanding of the time it will take to run your job (actually an over-estimate of such things) [see R and python references on approaches to estimate this - to do.] 2. and (to a lesser extent) the amount of storage and RAM you will be using for the job (this varies across what type of access you have at PSC).

Ajshen at 21:46, 28 June 2026

2026-06-28T21:46:17Z

← Older revision		Revision as of 21:46, 28 June 2026
Line 1:		Line 1:

			[[Category:Wright Cluster]]

	Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapidly as possible. Because this system manages how you are able to use the resources and when your job is run, it requires you to know additional information before you submit your job. Additionally, given that the PSC uses slurm, one should it as a tool for when you have a really good understanding of your code and have attempted to make it efficient, especially since you’re often constrained by the number of core-hours you use. To submit jobs you’ll need to have 1. an understanding of the time it will take to run your job (actually an over-estimate of such things) [see R and python references on approaches to estimate this - to do.] 2. and (to a lesser extent) the amount of storage and RAM you will be using for the job (this varies across what type of access you have at PSC).		Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapidly as possible. Because this system manages how you are able to use the resources and when your job is run, it requires you to know additional information before you submit your job. Additionally, given that the PSC uses slurm, one should it as a tool for when you have a really good understanding of your code and have attempted to make it efficient, especially since you’re often constrained by the number of core-hours you use. To submit jobs you’ll need to have 1. an understanding of the time it will take to run your job (actually an over-estimate of such things) [see R and python references on approaches to estimate this - to do.] 2. and (to a lesser extent) the amount of storage and RAM you will be using for the job (this varies across what type of access you have at PSC).

Ajshen: Added language specific parallelization

2026-01-30T07:37:08Z

Added language specific parallelization

← Older revision		Revision as of 07:37, 30 January 2026
Line 263:		Line 263:
	</syntaxhighlight>		</syntaxhighlight>

	---		=== Language-specific Parallelization ===
			Parallelization can also be achieved within the specific script being called in the job. See [[Parallelization in python]] and [[Parallelization in r]] for more details.

	==SLURM Demo==		==SLURM Demo==
	For a hands on demonstration using SLURM, click [https://info.stat.cmu.edu/index.php?title=SLURM_Demo here].		For a hands on demonstration using SLURM, click [https://info.stat.cmu.edu/index.php?title=SLURM_Demo here].

Ajshen at 07:35, 30 January 2026

2026-01-30T07:35:26Z

Show changes

Jacobg2: /* SLURM Demo */

2025-08-28T12:43:48Z

SLURM Demo

← Older revision		Revision as of 12:43, 28 August 2025
Line 97:		Line 97:

	== SLURM Demo ==		== SLURM Demo ==
	For a hands on demonstration of using SLURM, click [https://info.stat.cmu.edu/index.php?title=SLURM_Demo here].		For a hands on demonstration using SLURM, click [https://info.stat.cmu.edu/index.php?title=SLURM_Demo here].

Jacobg2 at 12:32, 28 August 2025

2025-08-28T12:32:02Z

← Older revision		Revision as of 12:32, 28 August 2025
Line 97:		Line 97:

	== SLURM Demo ==		== SLURM Demo ==
	For a hands on demonstration of using SLURM, click here.		For a hands on demonstration of using SLURM, click [https://info.stat.cmu.edu/index.php?title=SLURM_Demo here].

Jacobg2: /* SLURM Demo */

2025-08-28T12:31:26Z

SLURM Demo

← Older revision		Revision as of 12:31, 28 August 2025
Line 97:		Line 97:

	== SLURM Demo ==		== SLURM Demo ==
			For a hands on demonstration of using SLURM, click here.

Jacobg2 at 12:30, 28 August 2025

2025-08-28T12:30:55Z

← Older revision		Revision as of 12:30, 28 August 2025
Line 95:		Line 95:
	#<code>set -x</code>, stores console lines from job runs into a log file (good for debugging). Without a line above saying <code>SBATCH -o log_file_$SLURM_ARRAY_TASK_ID.tx</code> it will select the file’s name itself.		#<code>set -x</code>, stores console lines from job runs into a log file (good for debugging). Without a line above saying <code>SBATCH -o log_file_$SLURM_ARRAY_TASK_ID.tx</code> it will select the file’s name itself.
	#<code>Rscript code.R $SLURM_ARRAY_TASK_ID 100</code>, the actual job we’re running. In this case we’re running an <code>R</code> file with input parameters associated with our job array number in line 4 and a second parameter 100. For python users, this could easily be replaced with <code>ipython code.py $SLURM_ARRAY_TASK_ID 100</code>.		#<code>Rscript code.R $SLURM_ARRAY_TASK_ID 100</code>, the actual job we’re running. In this case we’re running an <code>R</code> file with input parameters associated with our job array number in line 4 and a second parameter 100. For python users, this could easily be replaced with <code>ipython code.py $SLURM_ARRAY_TASK_ID 100</code>.

			== SLURM Demo ==

Skipper: Created page with " Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapid..."

2024-03-30T00:01:19Z

Created page with " Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapid..."

New page

Slurm is a job management system that deals with job prioritization and the allocation of resources. On high-performance computers slurm helps make sure jobs are run as rapidly as possible. Because this system manages how you are able to use the resources and when your job is run, it requires you to know additional information before you submit your job. Additionally, given that the PSC uses slurm, one should it as a tool for when you have a really good understanding of your code and have attempted to make it efficient, especially since you’re often constrained by the number of core-hours you use. To submit jobs you’ll need to have 1. an understanding of the time it will take to run your job (actually an over-estimate of such things) [see R and python references on approaches to estimate this - to do.] 2. and (to a lesser extent) the amount of storage and RAM you will be using for the job (this varies across what type of access you have at PSC).

== Slurm’s most powerful feature: parallelization ==

The most powerful feature of slurm is probably its ability to run jobs in parallel outside of the programming language [see articles for inside language parallelization in [[Parallelization_in_r|R]] and [[Parallelization_in_python|python]]]. Parallelization is often useful in statistical analysis when the analysis can be broken up into many subparts (e.g. parameter tuning, data processing, simulations, etc.). When using a system with slurm, it’s smart to parallelize with slurm (outside of the programming language) if not too costly so that slurm can schedule smaller (in core-hour) jobs and get them actually run more quickly.

== Basic / commonly used commands ==

Because slurm decides when your code will run, it is important to know what resources are being used when you’re about to run a job. Similar to the department server, conscientious usage of the servers with slurm is the best approach (this varies across what type of access you have at PSC). Additionally, given that slurm can constrain the number of core hours you use, it’s useful to know how to cancel your jobs if something is not processing correctly.

{| class="wikitable"
|-
! width="22%" | Commmand
! width="30%" | Description
! width="47%" | Unix similarity
|-
| <code>squeue</code>
| get a complete list of all jobs running
| <code>top</code>
|-
| <code>squeue -u username</code>
| all jobs running for just a single user
| <code>top -u username</code>
|-
| <code>sinfo</code>
| provide information about the state of each node (look for idle nodes)
|  
|-
| <code>scancel</code>
| cancel jobs
| slightly similar to canceling a job with a [[Terminal_multiplexers|terminal multiplexer]]
|-
| <code>scancel -u username</code>
| cancel all jobs for a single user (only can do with yourself)
|  
|-
| <code>scancel _JobID_</code>
| cancel job with id: _JobID__
|  
|-
| <code>sbatch _bash_script_</code>
| submit batch request with a <code>.sh</code> bash script
| slightly similar to running a job in a [[Terminal_multiplexers|terminal multiplexer]]
|}

Additional documentation: [https://slurm.schedmd.com/ slurm.schedmd], [https://www.osc.edu/book/export/html/2861 osc.edu], local astro-PSC docs: [https://vera-doc.psc.edu/ vera-doc.psc.edu]

=== Some additional useful tools ===

Based on feedback we wanted to give you some additional references to getting useful information about your jobs:

<ol>
<li>How to see the number of jobs you have Running, Pending, Completed, and Failed: [https://stackoverflow.com/questions/53037185/in-slurm-is-there-a-quick-command-to-determine-the-total-number-of-jobs-pendin slackoverflow] </li>
<li>How to understand the amount of core hours you'ved used: To see the time used for each individual job: <syntaxhighlight lang="bash">sacct -S2020-08-09 -u username -ojobid,start,end,alloccpu,cputime | column -t </syntaxhighlight>
where: - <code>ojobid</code>: job ID - <code>alloccpu</code>: number of cpus allocated - <code>cputime</code>: amount of cpu time used To get the total number of core-hours used (summing up column 5) run:
<syntaxhighlight lang="bash">sacct -S2020-08-09 -u username -ojobid,start,end,alloccpu,cputime | column -t | awk '{print $5}' | awk '{s+=$1}END{print s}'</syntaxhighlight>
</li>
<li>How to transfer files: see [[Linux_and_department_servers|Linux on the server]] commands (<code>ssh</code>, <code>sftp</code>, GUI based transferring) </li>
</ol>

 

== Example of how execute jobs ==

To actually execute a job it’s common practice to describe what you’d like to do in a bash file. This helps keep all the specifications about the job request in one place.

An example bash script <code>batch-script.sh</code> for running a set of <code>.R</code> files (with a few inputs) can be seen below:
<syntaxhighlight lang="bash">#!/bin/bash
#SBATCH -t 01:00:00
#SBATCH -N 1
#SBATCH --ntasks-per-node 1
#SBATCH --array=11-101:2
#SBATCH --mail-user=andrew_id@andrew.cmu.edu
#SBATCH --mail-type=END

set -x

Rscript code.R $SLURM_ARRAY_TASK_ID 100</syntaxhighlight>

To run this file we’d run (on the server):
<syntaxhighlight lang="bash">sbatch batch-script.sh</syntaxhighlight>

The following structures should be observed:

#<code>#!/bin/bash </code>[Required], this reminds the compiler that this is a bash script
#<code>SBATCH -t 01:00:00 </code>[Required], this tells slurm that we expect a maximum of 1 hour to be need to run each job ('''if the job is running past 1 hour it will cancel - so be conservative in this estimate!''')
#<code>SBATCH -N 1 </code>[Required], this tells slurm that we only need 1 nodes for each job (a node contains multiple cores)
#<code>SBATCH --ntasks-per-node 1</code>, this tells slurm we only need 1 core for each job (in this example we are parallelizing with slum)
#<code>SBATCH --array=11-101:2</code> this tells slurm that we will run jobs with index 11, 13, …, to 101 (the power of slurm). See [https://slurm.schedmd.com/job_array.html slurm.schedmd] ) for details on job arrays and how you can use them to parallelize your code.
#<code>SBATCH --mail-user=andrew_id@andrew.cmu.edu</code>, this makes slurm send you an email when all your jobs from this file is done (so you don’t have to keep logging in to see if the job failed / completed)
#<code>SBATCH --mail-type=END</code>, defines when you’ll get the email (at the end of all all jobs)
#<code>set -x</code>, stores console lines from job runs into a log file (good for debugging). Without a line above saying <code>SBATCH -o log_file_$SLURM_ARRAY_TASK_ID.tx</code> it will select the file’s name itself. 
#<code>Rscript code.R $SLURM_ARRAY_TASK_ID 100</code>, the actual job we’re running. In this case we’re running an <code>R</code> file with input parameters associated with our job array number in line 4 and a second parameter 100. For python users, this could easily be replaced with <code>ipython code.py $SLURM_ARRAY_TASK_ID 100</code>.