Beginner's Guide to Computation in Statistical Research: Difference between revisions
No edit summary |
|||
| Line 14: | Line 14: | ||
Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows '''scalable and efficient'''. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully the department also provides several options for remote high-performance computing for even the largest of research workflows. | Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows '''scalable and efficient'''. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully the department also provides several options for remote high-performance computing for even the largest of research workflows. | ||
# Parallelization: Making the most of your Machine | # [[Parallelization: Making the most of your Machine]] | ||
# Accessing Department Servers | # Accessing Department Servers | ||
# Job Management and Advanced Parallelization on Servers (Tmux, slurm, etc) | # Job Management and Advanced Parallelization on Servers (Tmux, slurm, etc) | ||
Revision as of 03:34, 27 March 2026
While the foundations of statistical methods are grounded in mathematical theory, we often need to demonstrate that these methods are useful in practice. This can done via toy examples, simulation studies, and applications to real-world datasets. We do not recommend spending hundreds of hours running calculations on datasets by hand. Thus, we will introduce the reader to core concepts of statistical computing and relevant resources in the CMU Department of Statistics and Data Science.
Implementation
The main aim for statistical computing is to implement proposed algorithms in code such that they can be applied to a variety of inputs very quickly. These implementations can be as simple as computing the OLS coefficients to optimizing a billion-parameter neural network.
These implementations are usually achieved through computer programming. As a student in the Statistics department, you will be exposed to a variety of programming languages and tools. Below is a reading list of first steps to get you started on your computation journey
- Choosing and Installing a Programming Language
- Installing and Configuring an Integrated Development Environment
- Resources for Learning to Program
Scale and Efficiency
Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows scalable and efficient. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully the department also provides several options for remote high-performance computing for even the largest of research workflows.
- Parallelization: Making the most of your Machine
- Accessing Department Servers
- Job Management and Advanced Parallelization on Servers (Tmux, slurm, etc)
Reproducibility
Yes
- GitHub
- Virtual Environments
- Docker
Collaboration
Topics like
