Research Computing Dos and Don'ts: Difference between revisions
No edit summary |
No edit summary |
||
| (7 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
=== | === Do Respect Your Storage Usage === | ||
The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you | The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it can fill up quick. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.<syntaxhighlight lang="bash"> | ||
du -sh ~/.[!.]* | sort -hr | du -sh ~/.[!.]* | sort -hr | ||
</syntaxhighlight> | </syntaxhighlight> | ||
=== | === Don't Max Out the Server === | ||
Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish. | Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish. | ||
=== | === Do Utilize Scratch Storage === | ||
Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, | Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, write it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found [[Scratch Storage|here]]. | ||
=== | === Don't Hog the Servers === | ||
The servers in our environment are meant to be shared by everyone. | The servers in our environment are meant to be shared by everyone. It's ok to fully use a couple of servers at a time, but monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty, don't assume others won't want to use them at some point too. | ||
=== | === Do Learn How to use SLURM === | ||
Our computing environment | SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this [[Open OnDemand|page]]. | ||
=== Don't Waste Your Time === | |||
Our computing environment can be a lot to take in at first. If you are new to the department, haven't used computing resources before, or are just confused, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff (remarks@stat.cmu.edu), and the Computing TA. | |||
Latest revision as of 13:32, 13 April 2026
Do Respect Your Storage Usage
The main storage location at "/home/yourusername" is located on a network storage device (NAS) and shared by every user in the department. The NAS has a large amount of space, but when you consider ~100 users share it, it can fill up quick. Please monitor your storage usage and delete unused data. Run the command below to view the size of your home directories. The larger your directory, the longer this command will take to run.
du -sh ~/.[!.]* | sort -hrDon't Max Out the Server
Our servers are much more powerful than your laptop, but still can't handle running a dozen high output jobs at once. If you run too much on a single server, your session can become slow or even time out. Other users may not be able to connect to the server, rendering it useless for everyone until your jobs finish.
Do Utilize Scratch Storage
Each Leviathan server is equipped with local SSD based storage at /scratch. This location provides faster access to your data and prevent's slow downs on the NAS. If you are loading data into a job, move the data to /scratch before hand. If you are writing out data, write it to /scratch. Once the job is complete, you can move the data back to your /home directory. Be sure to clean up data that is not in use to save space. More information on scratch storage can be found here.
Don't Hog the Servers
The servers in our environment are meant to be shared by everyone. It's ok to fully use a couple of servers at a time, but monitor your usage and make sure that you aren't preventing others from running jobs. If the servers seem empty, don't assume others won't want to use them at some point too.
Do Learn How to use SLURM
SLURM is a job scheduling software that many research centers use. Instead of running scripts directly on the servers in a terminal, you "submit a job" with another script that runs your script. SLURM delegates resources and makes sure your job stays within the confined hardware that you define within your submit script. To learn more about SLURM within our environment, visit this page.
Don't Waste Your Time
Our computing environment can be a lot to take in at first. If you are new to the department, haven't used computing resources before, or are just confused, don't be afraid to ask for help! Help resources include the Computing Wiki, Stat department IT Staff (remarks@stat.cmu.edu), and the Computing TA.
