What resources when

To try to answer the common question “what resources do I have access to and when should I use them”, below we present to approaches to answering the question. The first of which focuses on what resources exist and what they should be used for and the second tries to answer resource needs / current computing problems with resources to fit those needs/ solve those problems.

After you’re found the resource you think you’d like to use (or even when you’re still deciding), we strongly encourage you to attend an computing office hour to help discuss all the things you should be thinking about and get you on the right track quickly.

What computing resources do you have access to

Your personal computer

Your personal computer is great for small jobs that take less than 30 minutes, uses few cores and not too much RAM.

Pro:

1. Naturally the easiest tool to use since it’s yours.
2. Install whatever you want.
3. Nobody else is hogging it.

Cons:

• capacity:
  • few cores: 4-8 cores common on laptops (hyperthreading means 8-16 “cores”).
  • low RAM: 4-32 GB of common on laptops, used across all cores
  • “no” GPUs: you don’t have GPUs that are optimized for Deep Learning
• You're using your laptop for other things, and running a big job will slow everything down when you're trying to do your homework
• If your job lasts many hours, you may have to leave your laptop at home or leave it running overnight; or you may have problems when you close your laptop to take the bus.

This does not have to be you (though it's good to take breaks) https://xkcd.com/303/

Conclusion: Your personal computer isn’t the best for when doing a lot of computing and its easy to put jobs elsewhere!

Common use-cases

1. Most class projects
2. Non-data problems
3. Non-complex structure problems

Associated Articles:

• parallelization

The department servers

The departmental servers should be seen as a “sandbox”. You can experiment with lots of things on the servers and it doesn’t cost you much to do. The Leviathans and Hydras have more RAM than a typical laptop. The Leviathans also have top-of-the-line CPUs that should be faster than most laptops, though the Hydras have older CPUs that may not be faster than running code on your own laptop.

Pros:

1. Best for long-running jobs
2. Plenty of RAM per machine (128 GB on Leviathans + hydra7 and 8, 64 GB on other hydras)
3. Very low barrier to entry (aka easy to get started)
4. File management system shares across all servers

Cons:

1. usage: The department server is a bit of a “wild-west” in terms of usage (it varies how many people use it at a given time)
2. cores per server: the number of cores per server is only 12 (Hydras) or 16 (Leviathans)

Conclusion: Great place to start for more intensive computing.

Associated Articles:

• Linux and department servers

Server Monitoring

• Hydra Cluster Usage Monitor
• Leviathan Cluster Usage Monitor

PSC (Pittsburgh supercomputer center)

The Pittsburgh supercomputer center provides supercomputers than have massive amounts of cores (but also can find some with large RAMs and GPUs). Generally speaking, the supercomputing resources are often used when you have extremely parallelizable code that would take a very long time if you ran it on just a few cores (e.g. your computer or a few hydras).

Pros:

1. A massive amount of cores and RAM that can be leveraged for extremely parallelizable computing
2. Easier to access in the statistics department

Cons:

1. Requires learning of new job management system (though quick to learn - see [Slurm])
2. Some options to access the resources can take time or require going through your advisor
3. currently (2021) has been having problems with servers crashing

Associated Articles:

• PSC
• Slurm

GPU Computing

The Statistics & Data Science Department offers two main GPU computing environments.

Ghidorahs

• 3 physical servers
• RTX 4090 GPUs
• Connected to the Stats network share
• Your statistics directory account will grant you access

Wright

• For a detail overview, please check here
• You must request access
• The Statistics network share is not accessible

Computing resource flowchart

These flow charts try to help capture what’s available and when you might use them.

Visit the links below to view current computing resource usage. Select a server with the least amount of users and lowest resource usage.

• Hydra Cluster Usage Monitor

• Ghidorah Cluster Usage Monitor

From laptopmag
     2. ↑ From digitaltrends