Beginner's Guide to Computation in Statistical Research: Difference between revisions
| Line 25: | Line 25: | ||
# | # | ||
==Scale and Efficiency== | == Scale and Efficiency == | ||
Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows '''scalable and efficient'''. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully, the department also provides several options for remote high-performance computing for even the largest of research workflows. | Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows '''scalable and efficient'''. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully, the department also provides several options for remote high-performance computing for even the largest of research workflows. | ||
<div style="background-color: #f8f9fa; border-left: 4px solid #C41230; padding: 10px 15px; margin-bottom: 10px;"> | |||
'''[[Parallelization: Making the most of your Machine]]''' <br> | |||
How to run multiple experiments at once | |||
</div> | |||
<div style="background-color: #f8f9fa; border-left: 4px solid #C41230; padding: 10px 15px; margin-bottom: 10px;"> | |||
'''[[Building Workflows on Remote Servers]]''' <br> | |||
Using the department servers and other computing clusters | |||
</div> | |||
<div style="background-color: #f8f9fa; border-left: 4px solid #C41230; padding: 10px 15px; margin-bottom: 10px;"> | |||
'''[[Big Data Management]]''' <br> | |||
How to handle datasets that are measured in terabytes | |||
</div> | |||
==Reproducibility and Collaboration== | ==Reproducibility and Collaboration== | ||
Revision as of 04:12, 26 June 2026
At some point, you will need to demonstrate that your statistical methods actually work in practice. Whether you are running a toy simulation, analyzing real-world data, or optimizing a complex neural network, you will need to write code. If you are new to computing, the wiki can be overwhelming. Read this page from top to bottom. It is designed to take you from installing your first language to running massive workflows on department servers.
Implementation
Your first goal is to translate mathematical algorithms into functional code. It is important to familiarize yourself with these concepts early on.
Choosing and Installing a Programming Language
Help deciding between R, Python, Julia, or other languages and how to get them running on your personal machine.
Installing and Configuring an Integrated Development Environment
Your IDE (Integrated Development Environment) is your workshop. Learn how to set up VS Code or RStudio.
Beyond Print Statements: Breakpoint Debugging
Your IDE (Integrated Development Environment) is your workshop. Learn how to set up VS Code or RStudio.
Command Line Basics
Essential survival skills for navigating your computer without graphical windows
Scale and Efficiency
Once you have gotten the hang of implementing statistical algorithms in code, the next major hurdle will be making your workflows scalable and efficient. For code running on your own machine, you want to make sure you are using every CPU core and every bit of RAM as efficiently as possible. Even then, your personal computer alone can only get you so far. Thankfully, the department also provides several options for remote high-performance computing for even the largest of research workflows.
Parallelization: Making the most of your Machine
How to run multiple experiments at once
Building Workflows on Remote Servers
Using the department servers and other computing clusters
Big Data Management
How to handle datasets that are measured in terabytes
Reproducibility and Collaboration
While it is important that you are able to obtain results efficiently, it is also equally important for other member of the academic community to arrive at your same results independently. Specifically, others should be able to run your workflow and obtain the exact same output as you did with minimal tinkering and friction.
| Title | Description | Reading Time? |
|---|---|---|
| Version Control | Keeping track of code/experiment history | |
| Dependency Management | Making sure anyone else in the world can run your workflow |
Software Development
Topics like
