Research Computing Dos and Don'ts - Revision history

Jacobg2 at 13:32, 13 April 2026

2026-04-13T13:32:42Z

← Older revision		Revision as of 13:32, 13 April 2026
Line 11:		Line 11:

	=== Don't Hog the Servers ===		=== Don't Hog the Servers ===
	The servers in our environment are meant to be shared ~~with~~ everyone. It's ok to fully use a couple of servers at a time, but monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty, don't assume others won't want to use them at some point too.		The servers in our environment are meant to be shared by everyone. It's ok to fully use a couple of servers at a time, but monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty, don't assume others won't want to use them at some point too.

	=== Do Learn How to use SLURM ===		=== Do Learn How to use SLURM ===

Jacobg2: /* Don't Hog the Servers */

2026-04-13T13:32:06Z

Don't Hog the Servers

← Older revision		Revision as of 13:32, 13 April 2026
Line 11:		Line 11:

	=== Don't Hog the Servers ===		=== Don't Hog the Servers ===
	The servers in our environment are meant to be shared with everyone. ~~Monitor~~ your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty ~~when you log on~~, don't assume others won't want to use them at some point too.		The servers in our environment are meant to be shared with everyone. It's ok to fully use a couple of servers at a time, but monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty, don't assume others won't want to use them at some point too.

	=== Do Learn How to use SLURM ===		=== Do Learn How to use SLURM ===

Jacobg2: /* 1 - Do Respect Your Storage Usage */

2026-04-13T13:28:13Z

1 - Do Respect Your Storage Usage

← Older revision		Revision as of 13:28, 13 April 2026
Line 1:		Line 1:
	=== ~~1 -~~ Do Respect Your Storage Usage ===		=== Do Respect Your Storage Usage ===
	The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it can fill up quick. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">		The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it can fill up quick. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">
	du -sh ~/.[!.]* \| sort -hr		du -sh ~/.[!.]* \| sort -hr
	</syntaxhighlight>		</syntaxhighlight>

	=== ~~2 -~~ Don't Max Out the Server ===		=== Don't Max Out the Server ===
	Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish.		Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish.

	=== ~~3 -~~ Do Utilize Scratch Storage ===		=== Do Utilize Scratch Storage ===
	Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, write it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].		Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, write it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].

	=== ~~4 -~~ Don't Hog the Servers ===		=== Don't Hog the Servers ===
	The servers in our environment are meant to be shared with everyone. Monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty when you log on, don't assume others won't want to use them at some point too.		The servers in our environment are meant to be shared with everyone. Monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty when you log on, don't assume others won't want to use them at some point too.

	=== ~~5 -~~ Do Learn How to use SLURM ===		=== Do Learn How to use SLURM ===
	SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].		SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].

	=== ~~6 -~~ Don't Waste Your Time ===		=== Don't Waste Your Time ===
	Our computing environment can be a lot to take in at first. If you are new to the department, haven't used computing resources before, or are just confused, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff (remarks@stat.cmu.edu), and the Computing TA.		Our computing environment can be a lot to take in at first. If you are new to the department, haven't used computing resources before, or are just confused, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff (remarks@stat.cmu.edu), and the Computing TA.

Jacobg2: /* # - Don't Waste Your Time */

2026-04-13T13:27:13Z

# - Don't Waste Your Time

← Older revision		Revision as of 13:27, 13 April 2026
Line 16:		Line 16:
	SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].		SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].

	=== # - Don't Waste Your Time ===		=== 6 - Don't Waste Your Time ===
	Our computing environment can be a lot to take in at first. If you are new to the department or haven't used computing resources before, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.		Our computing environment can be a lot to take in at first. If you are new to the department, haven't used computing resources before, or are just confused, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff (remarks@stat.cmu.edu), and the Computing TA.

Jacobg2: /* 1 - Do Respect Your Storage Usage */

2026-03-24T22:35:58Z

1 - Do Respect Your Storage Usage

← Older revision		Revision as of 22:35, 24 March 2026
Line 1:		Line 1:
	=== 1 - Do Respect Your Storage Usage ===		=== 1 - Do Respect Your Storage Usage ===
	The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it ~~may not seem as big as you think~~. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">		The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it can fill up quick. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">
	du -sh ~/.[!.]* \| sort -hr		du -sh ~/.[!.]* \| sort -hr
	</syntaxhighlight>		</syntaxhighlight>

Jacobg2: /* 1 - Do Respect Your Storage Usage */

2026-03-24T22:34:37Z

1 - Do Respect Your Storage Usage

← Older revision		Revision as of 22:34, 24 March 2026
Line 1:		Line 1:
	=== 1 - Do Respect Your Storage Usage ===		=== 1 - Do Respect Your Storage Usage ===
	The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you ~~divide that by the number of total~~ users, it ~~can fill up quick~~. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">		The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it may not seem as big as you think. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash">
	du -sh ~/.[!.]* \| sort -hr		du -sh ~/.[!.]* \| sort -hr
	</syntaxhighlight>		</syntaxhighlight>

Jacobg2: /* 3 - Do Utilize Scratch Storage */

2026-03-24T17:07:11Z

3 - Do Utilize Scratch Storage

← Older revision		Revision as of 17:07, 24 March 2026
Line 8:		Line 8:

	=== 3 - Do Utilize Scratch Storage ===		=== 3 - Do Utilize Scratch Storage ===
	Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, ~~save~~ it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].		Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, write it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].

	=== 4 - Don't Hog the Servers ===		=== 4 - Don't Hog the Servers ===
	The servers in our environment are meant to be shared by everyone. Monitor ~~you~~ usage and make sure that you aren't preventing others from running jobs. If the servers seem ~~unused~~, ~~keep space available for~~ others.		The servers in our environment are meant to be shared with everyone. Monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty when you log on, don't assume others won't want to use them at some point too.

	=== 5 - Do Learn How to use SLURM ===		=== 5 - Do Learn How to use SLURM ===
	SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you submit a job with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].		SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].

	=== # - Don't Waste Your Time ===		=== # - Don't Waste Your Time ===
	Our computing environment is a lot. If you are new to the department or haven't used computing resources before~~. Don~~'t be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.		Our computing environment can be a lot to take in at first. If you are new to the department or haven't used computing resources before, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.

Jacobg2 at 16:48, 24 March 2026

2026-03-24T16:48:58Z

← Older revision		Revision as of 16:48, 24 March 2026
Line 12:		Line 12:
	=== 4 - Don't Hog the Servers ===		=== 4 - Don't Hog the Servers ===
	The servers in our environment are meant to be shared by everyone. Monitor you usage and make sure that you aren't preventing others from running jobs. If the servers seem unused, keep space available for others.		The servers in our environment are meant to be shared by everyone. Monitor you usage and make sure that you aren't preventing others from running jobs. If the servers seem unused, keep space available for others.

			=== 5 - Do Learn How to use SLURM ===
			SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you submit a job with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand\|page]].

	=== # - Don't Waste Your Time ===		=== # - Don't Waste Your Time ===
	Our computing environment is a lot. If you are new to the department or haven't used computing resources before. Don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.		Our computing environment is a lot. If you are new to the department or haven't used computing resources before. Don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.

Jacobg2 at 14:10, 24 March 2026

2026-03-24T14:10:03Z

← Older revision		Revision as of 14:10, 24 March 2026
Line 9:		Line 9:
	=== 3 - Do Utilize Scratch Storage ===		=== 3 - Do Utilize Scratch Storage ===
	Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, save it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].		Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, save it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].

			=== 4 - Don't Hog the Servers ===
			The servers in our environment are meant to be shared by everyone. Monitor you usage and make sure that you aren't preventing others from running jobs. If the servers seem unused, keep space available for others.

			=== # - Don't Waste Your Time ===
			Our computing environment is a lot. If you are new to the department or haven't used computing resources before. Don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff, and the Computing TA.

Jacobg2 at 13:24, 24 March 2026

2026-03-24T13:24:18Z

← Older revision		Revision as of 13:24, 24 March 2026
Line 6:		Line 6:
	=== 2 - Don't Max Out the Server ===		=== 2 - Don't Max Out the Server ===
	Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish.		Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish.

			=== 3 - Do Utilize Scratch Storage ===
			Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, save it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage\|here]].