Main Page new: Difference between revisions

From Stat DS Computing Wiki
← Older edit
VisualWikitext
No edit summary
No edit summary
 
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Welcome to the CMU Department of Statistics and Data Science Computing Wiki! To get the most out of this resource, please note the following:
Welcome to the CMU Department of Statistics and Data Science Computing Wiki! To get the most out of this resource, please note the following:


# The main purpose of this wiki is to familiarize readers with computing resources provided by the department '''provided by the department.''' We encourage everyone, especially newcomers, to at least skim all of the articles in the '''Quick Start''' section below
<div style="background-color: #e2e3e5; border: 5px solid #6c757d; padding: 15px; margin-bottom: 20px; margin-top: 15px;">
#
'''Purpose & Contribution'''<br>
# This wiki is editable by any current member of the CMU Statistics department. If you find any small errors, feel free to correct them directly. If you have a question about a specific article, post it on the Talk section of that article.
The main purpose of this wiki is to familiarize readers with the computing resources provided by the department.
{| class="wikitable"
* '''Newcomers:''' Please read the [[Beginner's Guide to Computation in Statistical Research]] to familiarize yourself with statistical computing tools and best practices as you begin you research.
|+Important Announcements
* '''Computational Researchers:''' See the [[Article Catalog]] for a full selection of topics and pages and the [[Directory of Computing Resources for Statistical Research]] for available computational resources.
!Date
* '''Contributors and Maintainers:''' Feel free to leave a comment on any page, or create your own pages. Please see [[Generic Wiki Information]] for more information.
!Category
</div>
!Description
 
|-
== Important Announcements ==
|2026-04-01
Before using any department computing resources, please see [[Research Computing Dos and Don'ts]]<div style="background-color: #fff8e6; border-left: 4px solid #ffc107; padding: 15px; margin-bottom: 20px;">
|PSA  
* '''[2026-04-10] New Article:''' Introducing [[Globus]], a tool for sharing large files within and between institutions.
|If you use VS Code and have been encountering problems with Python environments, look for an extension called "[https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-python-envs Python Environments]" in your VS Code installation and uninstall this extension immediately
* '''[2026-04-01] PSA:''' If you use VS Code, look for "[https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-python-envs Python Environments]" in your extensions and '''uninstall it immediately''' due to environment conflicts.
|-
* '''[2026-04-01] Updated Article:''' The [[Linux and department servers]] page has been updated with instructions on how to use <code>/scratch</code> to avoid disk I/O slowdowns. See [[Linux and department servers#Using /scratch]].
|2026-04-01
</div>
|Updated Article
|The [[Linux and department servers]] page has been updated with instruction on how to use /scratch to avoid slowdowns due to excessive disk I/O
|}


== Quick Start ==
== Quick Start ==
<div style="background-color: #f8f9fa; border-left: 4px solid #005A9C; padding: 15px; margin-bottom: 10px;">
'''[[Beginner's Guide to Computation in Statistical Research]]'''<br>
The essential starting point for understanding how to scale your statistical workflows and choose the right tools for your research.
</div>
<div style="background-color: #f8f9fa; border-left: 4px solid #28a745; padding: 15px; margin-bottom: 10px;">
'''[[Directory of Computing Resources for Statistical Research]]'''<br>
A comprehensive index of all available clusters, servers, and computing environments accessible to department members.
</div>
<div style="background-color: #f8f9fa; border-left: 4px solid #6f42c1; padding: 15px; margin-bottom: 10px;">
'''[[Miscellaneous Services]]'''<br>
Additional department tools, printing, web hosting, software licenses, and specialized computing services.
</div>


* [[Beginner's Guide to Computation in Statistical Research]]
<div style="background-color: #f8f9fa; border-left: 4px solid #eff07d; padding: 15px; margin-bottom: 10px;">
* [[Directory of Computing Resources for Statistical Research]]
'''[[Article Catalog]]'''<br>
* [[Miscellaneous Services]]
A comprehensive list of articles organized by topic
</div>


== Support Resources ==
== Support Resources ==
For assistance with issues not covered in this wiki, please reach out to the following contacts:
For assistance with issues not covered in this wiki, please reach out to the following contacts:


# '''General tech support for department computing resources:''' [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu]
<div style="background-color: #f8f9fa; border-left: 4px solid #6c757d; padding: 15px; margin-bottom: 10px;">
# '''Live assistance for research projects (Graduate Students Only):''' Alex Shen, PhD Computing TA.
'''General Tech Support'''<br>
#* Office Hours, Mondays 2pm-3pm @ FMS Interview Room 320
For general issues with department computing resources:<br>
#* Direct inquiries over email ([mailto:ajshen@andrew.cmu.edu ajshen@andrew.cmu.edu]) and slack
'''Email:''' [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu]
# '''Live assistance for computing resource onboarding and troubleshooting (Graduate Students Only):'''  
</div>
#* Carl Skipper: office hours by appointment, requested via email ([mailto:skipper@andrew.cmu.edu skipper@andrew.cmu.edu])
#* Jake Gordon: office hours by appointment, requested via email ([mailto:jacobg2@andrew.cmu.edu jacobg2@andrew.cmu.edu])
 
== Department Contacts ==
{| class="wikitable"
|+
!Email Address
!Description
|-
|[mailto:events@stat.cmu.edu events@stat.cmu.edu]
|seminars, conferences & workshops, and calendar updates
|-
|[mailto:news@stat.cmu.edu news@stat.cmu.edu]
|dept/dept member-related public news & announcements
|-
|[mailto:lists@stat.cmu.edu lists@stat.cmu.edu]
|department email lists
|-
|[mailto:webmaster@stat.cmu.edu webmaster@stat.cmu.edu]
|website updates
|-
|[mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu]
|computer/tech requests
|-
|[mailto:rooms@stat.cmu.edu rooms@stat.cmu.edu]
|request space for meetings and TA office hours
|-
|[mailto:reimbursements@stat.cmu.edu reimbursements@stat.cmu.edu]
|travel and reimbursement requests
|}
 
== Old Computing Wiki Pages to be migrated ==
our [[What_resources_when|"what resources when" article]].
 
=== Managing servers and environments ===
 
==== Introductory Tutorials ====
[[Basic Linux Commands|Introduction to Basic Linux Commands]]
 
==== Virtual environments ====
*[[Virtual Environment Introduction|Why use virtual environments?]]
*[https://info.stat.cmu.edu/index.php?title=Mamba Mamba (Conda) environments].
*R's [[Renv]]
*Python's [[Python_virtual_environments|virtual environments (2 options)]]
*Old instructions for [[Conda with Python (Reference Only)|conda with Python]]
 
==== Containers ====
*[[Introduction to containers]]
*[[Docker, and why-you-should-use-it-for-your-research]]
*[[Tutorials|Basic tutorial]]
*[[Running Docker locally and on Hydras]]
 
=== Department computing resources ===
 
==== Servers ====
*[[Department Servers]]. 
*#[https://info.stat.cmu.edu/index.php?title=Department_Servers#Software Software] - overview of the software installed and ready for use.
*#[https://info.stat.cmu.edu/index.php?title=Monitoring_with_Grafana Monitor Server Usage]
 
*[[PSC]], the Pittsburgh Supercomputing Center. Free accounts and usage credits are available
*#[[ACCESS]]
*#[[Parallelization_on_PSC/Bridges-2]]
*[[Open OnDemand|Open OnDemand - StatDS]]
*[[Open OnDemand Wright|Open OnDemand - Wright]]
 
=== Development guides ===
 
==== Package development ====
*[[Developing_R_packages|R package development]]
*[[Developing_python_packages|Python package development]]
[[Terminal automation]]. This wiki introduces approaches to automate Linux/OS command line calls.
*[[Bash files]]. This wiki presents potential uses of Bash files including automated file/folder creation, parallelization and much more!
*[[Make]]. This wiki presents potential uses of <code>make</code> and Makefiles including automated workflow and dependency tracking
*[[Jumping to frequently used folders and automating environment setups]]
 
==== Version Control ====
See the guide provided in the site of [https://36-750.github.io/tools/version-control/ 36750 - Statistical Computing]
 
==== Integrated Development Environments (IDEs) ====
#[[VSCode|VSCode]]
#PyCharm
#RStudio
#Eclipse
 
See also the guide provided in the site of [https://36-750.github.io/tools/editors/ 36750 - Statistical Computing]
 
==== Miscellaneous====
#[[Windows_Subsystem_for_Linux_(WSL)_and_Python|Windows&nbsp;Subsystem for Linux (WSL) and Python]]
 
===General Department Computing Resources===
 
*Fall 2021 [[Computing_seminar_resources|Computing Seminar Resources]] (video and slides)
 
==Ideas for the Wiki==
 
Please see the [[TODO|TODO page]] to post suggestions and errata.
 
==Using the Wiki==
 
*Here is some&nbsp;[[Generic_Wiki_Information|Generic Wiki Information]]&nbsp;that you may be interested in, such as a users' guide and other resources for Wiki users.
 
== Archived Pages ==
These area pages that are potentially useful for some users, but no longer reflect the computing resources of the department writ large.
 
Azure
#[[Azure_getting_started|Azure: getting started]]. This wiki walks you through the steps to create a virtual machine through Azure
#[[How to obtain a virtual machine with a GPU]]
#[[Attach a disk to a virtual machine]]
 


Delete me?
<div style="background-color: #f8f9fa; border-left: 4px solid #17a2b8; padding: 15px; margin-bottom: 10px;">
'''Research Project Assistance (Graduate Students Only)'''<br>
Live assistance from Alex Shen, PhD Computing TA, for project-specific computing questions:<br>
'''Office Hours:''' Mondays 2:00 PM - 3:00 PM @ FMS Interview Room 320<br>
'''Contact:''' [mailto:ajshen@andrew.cmu.edu ajshen@andrew.cmu.edu] or reach out directly via Slack.
</div>


* [[Computing office hour usage]] (condensed into front page desc)
<div style="background-color: #f8f9fa; border-left: 4px solid #17a2b8; padding: 15px; margin-bottom: 10px;">
* [[Local PC Backup]] (replaced by official google help article)
'''Onboarding & Troubleshooting (Graduate Students Only)'''<br>
* [[Bulding your website with Hugo]] (replaced by Jekyll)
Live assistance for setting up and troubleshooting computing resources:<br>
* [[How to host your webpage under stat.cmu.edu]] (moved into misc services)
'''Carl Skipper:''' Office hours by appointment. Request via [mailto:skipper@andrew.cmu.edu skipper@andrew.cmu.edu]<br>
* [[Department GPU Servers]] (merged into main department servers article)
'''Jake Gordon:''' Office hours by appointment. Request via [mailto:jacobg2@andrew.cmu.edu jacobg2@andrew.cmu.edu]
* [[Access PSC compute nodes directly from VSCode]] (just a link to external article, will have entire article about vs code remote)
</div>


== Suggestions Box and Todos ==


Reference management
<div style="background-color: #aff0ff; border: 4px solid #17a2b8; padding: 15px; margin-bottom: 10px;">
#Zotero
* '''[New Article]''' Beginner's Guide -> Scaling -> Version Control -> Experiment Management
#Mendeley
* '''[New Article]''' Beginner's Guide -> Scaling -> Big Data Management -> Articles about file formats and compression
* '''[New Article]''' Beginner's Guide -> Software Development -> Package Creation -> Writing Documentation
* '''[New Article]''' Beginner's Guide -> Software Development -> Package Repositories


</div>


Text Editors
This section contains work in progress and future suggestions for the wiki. If you would like to leave a suggestions, you may:
#Vim
* Directly contact the Computing TA
#Nano
* Leave a comment on a specific wiki article with your suggestions
#Emacs
* Leave a comment on this page with your suggestions
#Atom
<br>
#Sublime

Latest revision as of 03:20, 30 June 2026

Welcome to the CMU Department of Statistics and Data Science Computing Wiki! To get the most out of this resource, please note the following:

Purpose & Contribution
The main purpose of this wiki is to familiarize readers with the computing resources provided by the department.

Important Announcements

Before using any department computing resources, please see Research Computing Dos and Don'ts

Quick Start

Beginner's Guide to Computation in Statistical Research
The essential starting point for understanding how to scale your statistical workflows and choose the right tools for your research.

Directory of Computing Resources for Statistical Research
A comprehensive index of all available clusters, servers, and computing environments accessible to department members.

Miscellaneous Services
Additional department tools, printing, web hosting, software licenses, and specialized computing services.

Article Catalog
A comprehensive list of articles organized by topic

Support Resources

For assistance with issues not covered in this wiki, please reach out to the following contacts:

General Tech Support
For general issues with department computing resources:
Email: remarks@stat.cmu.edu

Research Project Assistance (Graduate Students Only)
Live assistance from Alex Shen, PhD Computing TA, for project-specific computing questions:
Office Hours: Mondays 2:00 PM - 3:00 PM @ FMS Interview Room 320
Contact: ajshen@andrew.cmu.edu or reach out directly via Slack.

Onboarding & Troubleshooting (Graduate Students Only)
Live assistance for setting up and troubleshooting computing resources:
Carl Skipper: Office hours by appointment. Request via skipper@andrew.cmu.edu
Jake Gordon: Office hours by appointment. Request via jacobg2@andrew.cmu.edu

Suggestions Box and Todos

  • [New Article] Beginner's Guide -> Scaling -> Version Control -> Experiment Management
  • [New Article] Beginner's Guide -> Scaling -> Big Data Management -> Articles about file formats and compression
  • [New Article] Beginner's Guide -> Software Development -> Package Creation -> Writing Documentation
  • [New Article] Beginner's Guide -> Software Development -> Package Repositories

This section contains work in progress and future suggestions for the wiki. If you would like to leave a suggestions, you may:

  • Directly contact the Computing TA
  • Leave a comment on a specific wiki article with your suggestions
  • Leave a comment on this page with your suggestions


Retrieved from "https://info.stat.cmu.edu/index.php?title=Main_Page_new&oldid=892"