Stat DS Computing Wiki - User contributions [en]

Main Page

2024-05-03T15:59:01Z

Lmassera: /* General Department Computing Resources */

= Welcome =

Anyone can read this wiki.  If you want to write/edit stuff in the wiki, you will need to log in with your account provided by the Statistics & Data Science Department computing group. This Wiki is intended to be a repository for instruction, information and ideas related to the many computing environments available to members and affiliates of the Statistics & Data Science Department.

== Getting to know the computing resources & getting support ==

If you're interested in knowing about what resources you have access to (and/or) what resources you have access to that might solve a problem you're having, please visit our [[What_resources_when|"what resources when" article]].

To get support that is beyond this website please attend office hours below and/or email [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu] for support (for more details about how the support ticketing works through [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu] + ServiceNow please see the [[Service_now|ServiceNow]] article.

== Department Contacts ==

• [mailto:events@stat.cmu.edu events@stat.cmu.edu]: seminars, conferences & workshops, and calendar updates

• [mailto:news@stat.cmu.edu news@stat.cmu.edu]: dept/dept member-related public news & announcements

• [mailto:lists@stat.cmu.edu lists@stat.cmu.edu]: department email lists

• [mailto:webmaster@stat.cmu.edu webmaster@stat.cmu.edu]: website updates

• [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu]: computer/tech requests

• [mailto:rooms@stat.cmu.edu rooms@stat.cmu.edu]: request space for TA office hours

• [mailto:reimbursements@stat.cmu.edu reimbursements@stat.cmu.edu]: travel and reimbursement requests

 

= Computing Office Hours =

These office hour opportunities provide a dedicated time for you to come ask qusetions about research computing.  These include code advice, how to access certain resources, what the best resources are for your project, or to learn about new connections to help you progress in your education and research. Staff Office hour times will be sent out each week on Monday as a calendar invite; accept any of the invitations to add it to your personal calendar. PhD TA Office Hours are below, unless a different time is communicated.

#PhD TA Office Hours are held by Luca Masserano in FMS Interview Room 1 on Tuesday from 9 am to 11 am
#Staff Office Hours are held by Carl Skipper in BH 132G

See [[Computing_office_hour_usage|computing office hour usage]] for what office hours can be used for.

= Computing Wiki Pages =

== General Computing (personal computing) ==

'''Printing'''

*[[Department_Printers|Department Printers]]

'''Virtual environments'''

#R's [[Renv|Renv]]
#Python's [[Python_virtual_environments|virtual environments (2 options)]]

'''[[Parallel_computing|Parallelization]]'''

#[[Parallelization_in_r|R-specific parallelization]]
#[[Parallelization_in_python|Python-specific parallelization]]

'''Package development'''

#[[Developing_R_packages|R package development]]
#[[Developing_python_packages|Python package development]]

'''Version Control'''

See the guide provided in the site of [https://36-750.github.io/tools/version-control/ 36750 - Statistical Computing]

'''Reference management'''

#Zotero
#Mendeley

'''Integrated Development Environments (IDEs)'''

#[[VSCode|VSCode]]
#PyCharm
#RStudio
#Eclipse

'''Containers'''

#[[Introduction_to_containers|Introduction to containers]]
#[[Docker,_and_why-you-should-use-it-for-your-research|Docker, and why-you-should-use-it-for-your-research]]
#[[Tutorials|Basic tutorial]]
#[[Running_Docker_locally_and_on_Hydras|Running Docker locally and on Hydras]]

'''Text Editors'''

#Vim
#Nano
#Emacs
#Atom
#Sublime

See also the guide provided in the site of [https://36-750.github.io/tools/editors/ 36750 - Statistical Computing]

'''Web Hosting'''

#[[How_to_host_your_webpage_under_stat.cmu.edu|How to host your webpage under stat.cmu.edu]]
#[[Bulding_your_website_with_Hugo|Bulding your website with Hugo]]

'''Miscellanea'''

#[[Windows_Subsystem_for_Linux_(WSL)_and_Python|Windows Subsystem for Linux (WSL) and Python]]
#[[Amphetamine|Amphetamine: How to keep your computer awake]]

== General Department Computing Resources ==

*Fall 2021 [[Computing_seminar_resources|Computing Seminar Resources]] (video and slides)
*[[Linux_and_department_servers|Department Servers]]. This wiki introduces you to the '''hydra''' servers, basic Linux commands, and more.
*#[[Terminal_multiplexers|Terminal multiplexers (screen and tmux)]]. This wiki presents tools for use on the ''hydra'' that can allow you to run jobs even when your not on the server, and much much more!
*#[[Python_on_the_server|Python on the server]]. This wiki introduces uses command line approaches to running <code>python</code> on a virtual machine, and how to use jupyter notebooks through the server as well.
*#[[R_on_the_server|R on the server]]. This wiki introduces uses command line approaches to running <code>R</code> on a virtual machine.
*#[[Terminal_automation|Terminal automation]]. This wiki introduces approaches to automate Linux/OS command line calls.
*##[[Bash_files|Bash files]]. This wiki presents potential uses of Bash files including automated file/folder creation, parallelization and much more!
*##[[Make|Make]]. This wiki presents potential uses of <code>make</code> and Makefiles including automated workflow and dependency tracking
*##[[Jumping_to_frequently_used_folders_and_automating_environment_setups|Jumping to frequently used folders and automating environment setups]]

*[[Department_GPU_Servers|Department GPU Servers]]
*[[GPU_Cluster|GPU Cluster]]
*[[PSC|PSC]]
*#[[ACCESS|ACCESS]]
*#[[Slurm|Slurm]]. This wiki introduces the job management system uses at PSC.
*#[[Parallelization_on_PSC/Bridges-2|Parallelization_on_PSC/Bridges-2]]
*#[[Access_PSC_compute_nodes_directly_from_VSCode|Access PSC compute nodes directly from VSCode]]

*Azure
*#[[Azure_getting_started|Azure: getting started]]. This wiki walks you through the steps to create a virtual machine through Azure
*#[[How_to_obtain_a_virtual_machine_with_a_GPU|How to obtain a virtual machine with a GPU]]
*#[[Attach_a_disk_to_a_virtual_machine|Attach a disk to a virtual machine]]

 

= Ideas for the Wiki =

Please see the [[TODO|TODO page]] to post suggestions and errata.

= Using the Wiki =

*Here is some [[Generic_Wiki_Information|Generic Wiki Information]] that you may be interested in, such as a users' guide and other resources for Wiki users.

Main Page

2024-05-03T15:58:06Z

Lmassera: /* General Department Computing Resources */

= Welcome =

Anyone can read this wiki.  If you want to write/edit stuff in the wiki, you will need to log in with your account provided by the Statistics & Data Science Department computing group. This Wiki is intended to be a repository for instruction, information and ideas related to the many computing environments available to members and affiliates of the Statistics & Data Science Department.

== Getting to know the computing resources & getting support ==

If you're interested in knowing about what resources you have access to (and/or) what resources you have access to that might solve a problem you're having, please visit our [[What_resources_when|"what resources when" article]].

To get support that is beyond this website please attend office hours below and/or email [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu] for support (for more details about how the support ticketing works through [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu] + ServiceNow please see the [[Service_now|ServiceNow]] article.

== Department Contacts ==

• [mailto:events@stat.cmu.edu events@stat.cmu.edu]: seminars, conferences & workshops, and calendar updates

• [mailto:news@stat.cmu.edu news@stat.cmu.edu]: dept/dept member-related public news & announcements

• [mailto:lists@stat.cmu.edu lists@stat.cmu.edu]: department email lists

• [mailto:webmaster@stat.cmu.edu webmaster@stat.cmu.edu]: website updates

• [mailto:remarks@stat.cmu.edu remarks@stat.cmu.edu]: computer/tech requests

• [mailto:rooms@stat.cmu.edu rooms@stat.cmu.edu]: request space for TA office hours

• [mailto:reimbursements@stat.cmu.edu reimbursements@stat.cmu.edu]: travel and reimbursement requests

 

= Computing Office Hours =

These office hour opportunities provide a dedicated time for you to come ask qusetions about research computing.  These include code advice, how to access certain resources, what the best resources are for your project, or to learn about new connections to help you progress in your education and research. Staff Office hour times will be sent out each week on Monday as a calendar invite; accept any of the invitations to add it to your personal calendar. PhD TA Office Hours are below, unless a different time is communicated.

#PhD TA Office Hours are held by Luca Masserano in FMS Interview Room 1 on Tuesday from 9 am to 11 am
#Staff Office Hours are held by Carl Skipper in BH 132G

See [[Computing_office_hour_usage|computing office hour usage]] for what office hours can be used for.

= Computing Wiki Pages =

== General Computing (personal computing) ==

'''Printing'''

*[[Department_Printers|Department Printers]]

'''Virtual environments'''

#R's [[Renv|Renv]]
#Python's [[Python_virtual_environments|virtual environments (2 options)]]

'''[[Parallel_computing|Parallelization]]'''

#[[Parallelization_in_r|R-specific parallelization]]
#[[Parallelization_in_python|Python-specific parallelization]]

'''Package development'''

#[[Developing_R_packages|R package development]]
#[[Developing_python_packages|Python package development]]

'''Version Control'''

See the guide provided in the site of [https://36-750.github.io/tools/version-control/ 36750 - Statistical Computing]

'''Reference management'''

#Zotero
#Mendeley

'''Integrated Development Environments (IDEs)'''

#[[VSCode|VSCode]]
#PyCharm
#RStudio
#Eclipse

'''Containers'''

#[[Introduction_to_containers|Introduction to containers]]
#[[Docker,_and_why-you-should-use-it-for-your-research|Docker, and why-you-should-use-it-for-your-research]]
#[[Tutorials|Basic tutorial]]
#[[Running_Docker_locally_and_on_Hydras|Running Docker locally and on Hydras]]

'''Text Editors'''

#Vim
#Nano
#Emacs
#Atom
#Sublime

See also the guide provided in the site of [https://36-750.github.io/tools/editors/ 36750 - Statistical Computing]

'''Web Hosting'''

#[[How_to_host_your_webpage_under_stat.cmu.edu|How to host your webpage under stat.cmu.edu]]
#[[Bulding_your_website_with_Hugo|Bulding your website with Hugo]]

'''Miscellanea'''

#[[Windows_Subsystem_for_Linux_(WSL)_and_Python|Windows Subsystem for Linux (WSL) and Python]]
#[[Amphetamine|Amphetamine: How to keep your computer awake]]

== General Department Computing Resources ==

*Fall 2021 [[Computing_seminar_resources|Computing Seminar Resources]] (video and slides)
*[[Linux_and_department_servers|Department Servers]]. This wiki introduces you to the '''hydra''' servers, basic Linux commands, and more.
*#[[Terminal_multiplexers|Terminal multiplexers (screen and tmux)]]. This wiki presents tools for use on the ''hydra'' that can allow you to run jobs even when your not on the server, and much much more!
*#[[Python_on_the_server|Python on the server]]. This wiki introduces uses command line approaches to running <code>python</code> on a virtual machine, and how to use jupyter notebooks through the server as well.
*#[[R_on_the_server|R on the server]]. This wiki introduces uses command line approaches to running <code>R</code> on a virtual machine.
*#[[Terminal_automation|Terminal automation]]. This wiki introduces approaches to automate Linux/OS command line calls.
*##[[Bash_files|Bash files]]. This wiki presents potential uses of Bash files including automated file/folder creation, parallelization and much more!
*##[[Make|Make]]. This wiki presents potential uses of <code>make</code> and Makefiles including automated workflow and dependency tracking
*##[[Jumping_to_frequently_used_folders_and_automating_environment_setups|Jumping to frequently used folders and automating environment setups]]

  

*[[Department_GPU_Servers|Department GPU Servers]]
*[[GPU_Cluster|GPU Cluster]]
*[[PSC|PSC]]
*#[[ACCESS|ACCESS]]
*#[[Slurm|Slurm]]. This wiki introduces the job management system uses at PSC.
*#[[Parallelization_on_PSC/Bridges-2|Parallelization_on_PSC/Bridges-2]]
*#[[Access_PSC_compute_nodes_directly_from_VSCode|Access PSC compute nodes directly from VSCode]]

 

*Azure
*#[[Azure_getting_started|Azure: getting started]]. This wiki walks you through the steps to create a virtual machine through Azure
*#[[How_to_obtain_a_virtual_machine_with_a_GPU|How to obtain a virtual machine with a GPU]]
*#[[Attach_a_disk_to_a_virtual_machine|Attach a disk to a virtual machine]]

 

= Ideas for the Wiki =

Please see the [[TODO|TODO page]] to post suggestions and errata.

= Using the Wiki =

*Here is some [[Generic_Wiki_Information|Generic Wiki Information]] that you may be interested in, such as a users' guide and other resources for Wiki users.