Main Page new

Welcome to the CMU Department of Statistics and Data Science Computing Wiki! To get the most out of this resource, please note the following:

1. The main purpose of this wiki is to familiarize readers with computing resources provided by the department provided by the department. We encourage everyone, especially newcomers, to at least skim all of the articles in the Quick Start section below
3. This wiki is editable by any current member of the CMU Statistics department. If you find any small errors, feel free to correct them.

Quick Start

• A beginner's guide to computation in statistical research projects
• Introduction to department computational Sesources
• Miscellaneous Services

Getting to know the computing resources & getting support

If you're interested in knowing about what resources you have access to (and/or) what resources you have access to that might solve a problem you're having, please visit our "what resources when" article.

To get support that is beyond this website please attend office hours below and/or email remarks@stat.cmu.edu for support (for more details about how the support ticketing works through remarks@stat.cmu.edu + ServiceNow please

Support Resources

For assistance with issues not covered in this wiki, please reach out to the following contacts:

1. General tech support for department computing resources: remarks@stat.cmu.edu
2. Live assistance for research projects (Graduate Students Only): Alex Shen, PhD Computing TA.
   • Office Hours, Mondays 2pm-3pm @ FMS Interview Room 320
   • Direct inquiries over email (ajshen@andrew.cmu.edu) and slack
3. Live assistance for computing resource onboarding and troubleshooting (Graduate Students Only):
   • Carl Skipper: office hours by appointment, requested via email (skipper@andrew.cmu.edu)
   • Jake Gordon: office hours by appointment, requested via email (jacobg2@andrew.cmu.edu)

Department Contacts

• events@stat.cmu.edu: seminars, conferences & workshops, and calendar updates

• news@stat.cmu.edu: dept/dept member-related public news & announcements

• lists@stat.cmu.edu: department email lists

• webmaster@stat.cmu.edu: website updates

• remarks@stat.cmu.edu: computer/tech requests

• rooms@stat.cmu.edu: request space for meetings and TA office hours

• reimbursements@stat.cmu.edu: travel and reimbursement requests

Computing Wiki Pages

Managing servers and environments

Introductory Tutorials

Introduction to Basic Linux Commands

Virtual environments

• Why use virtual environments?
• Mamba (Conda) environments.
• R's Renv
• Python's virtual environments (2 options)
• Old instructions for conda with Python

Containers

• Introduction to containers
• Docker, and why-you-should-use-it-for-your-research
• Basic tutorial
• Running Docker locally and on Hydras

Department computing resources

Servers

• Department Servers.
  1. Using the Servers Overview -This wiki introduces you to the Leviathan and Hydra servers, basic Linux commands, and more.
  2. Terminal multiplexers (screen and tmux). This wiki presents tools for use on the servers that can allow you to run jobs even when your not on the server, and much much more!
  3. Python on the server. This wiki introduces uses command line approaches to running python on a virtual machine, and how to use jupyter notebooks through the server as well.
  4. R on the server. This wiki introduces uses command line approaches to running R on a virtual machine
  5. Software - overview of the software installed and ready for use.
  6. Monitor Server Usage

• Department GPU Servers, for small jobs needing 1 GPU.
• GPU Cluster, for larger jobs that may need multiple GPUs and many CPUs.
• PSC, the Pittsburgh Supercomputing Center. Free accounts and usage credits are available
  1. ACCESS
  2. SLURM. This wiki introduces the job management system uses at PSC.
  3. Parallelization_on_PSC/Bridges-2
  4. Access PSC compute nodes directly from VSCode
• Open OnDemand - StatDS
• Open OnDemand - Wright

Development guides

Parallelization

• R-specific parallelization
• Python-specific parallelization

Package development

• R package development
• Python package development

Terminal automation. This wiki introduces approaches to automate Linux/OS command line calls.

• Bash files. This wiki presents potential uses of Bash files including automated file/folder creation, parallelization and much more!
• Make. This wiki presents potential uses of make and Makefiles including automated workflow and dependency tracking
• Jumping to frequently used folders and automating environment setups

Version Control

See the guide provided in the site of 36750 - Statistical Computing

Reference management

1. Zotero
2. Mendeley

Integrated Development Environments (IDEs)

1. VSCode
2. PyCharm
3. RStudio
4. Eclipse

Text Editors

1. Vim
2. Nano
3. Emacs
4. Atom
5. Sublime

See also the guide provided in the site of 36750 - Statistical Computing

AI Tools

• ChatGPT and other LLMs

• GitHub Copilot

Miscellaneous

1. Windows Subsystem for Linux (WSL) and Python
2. Amphetamine: How to keep your computer awake

General Department Computing Resources

• Fall 2021 Computing Seminar Resources (video and slides)

Ideas for the Wiki

Please see the TODO page to post suggestions and errata.

Using the Wiki

• Here is some Generic Wiki Information that you may be interested in, such as a users' guide and other resources for Wiki users.

Archived Pages

These area pages that are potentially useful for some users, but no longer reflect the computing resources of the department writ large.

Azure

1. Azure: getting started. This wiki walks you through the steps to create a virtual machine through Azure
2. How to obtain a virtual machine with a GPU
3. Attach a disk to a virtual machine

Delete me?

• Computing office hour usage (condensed into front page desc)
• Local PC Backup (replaced by official google help article)
• Bulding your website with Hugo (replaced by Jekyll)
• How to host your webpage under stat.cmu.edu (moved into misc services)