Virtual Environment Introduction

For the purposes of statistics research, a virtual environment can be thought of as a self-contained installation of a programming language and any associated packages. Their notable properties include

• Specificity: Both the programming language and the associated packages inside a virtual environment are specific versions. For example, a Python virtual environment may consist of Python 3.8.5 and Pandas 2.2.3. This is to ensure no ambiguity in setting up a virtual environment on multiple computers.
• Independence: Changes in a virtual environment do not affect other virtual environments or the host computer (the computer on which this virtual environment is running). For example, upgrading Python in virtual environment A will not affect the Python version used in virtual environment B or the default Python version used on the host computer.
• Completeness: A virtual environment will contain all necessary tools needed to carry out a specific goal (e.g. reproducing results from a paper) and should not require additional user configuration after setup.

One host computer can contain any number of virtual environments for any number of programming languages. Once a virtual environment is set up for the first time, it is very easy to duplicate it on other host computers.

What problem do virtual environments solve?

Virtual environments are useful for anyone who uses the same machine and programming language to work on multiple projects. Consider a hypothetical researcher who uses Python and a fictional package called Nompie:

• In 2022, the researcher uses Python 3.8.5 and Nompie 1.2 to generate results for a paper that ultimately gets published
• In 2023, the researcher starts a completely new project. They upgrade to Python 3.10.1 and Nompie 2.0 to take advantage of more efficient implementation of certain algorithms
• An external collaborator contacts the researcher about their 2022 paper. They want the researcher to run the code under a slightly different scenario to test a hypothesis.
• The researcher finds out that Nompie 2.0 completely changed the syntax of several core functions and doesn't work with the 2022 paper code, and furthermore Nompie 1.2 is not compatible with Python 3.10.1

Without virtual environments, the researcher will find it very difficult to simultaneously work on their 2023 project and fulfill the external collaborator's request. In the worst case, the researcher will need to constantly uninstall and reinstall specific versions of Python and Nompie. However, the story changes with virtual environments:

• In 2022, the researcher creates virtual environment A, which contains Python 3.8.5 and Nompie 1.2, to generate results for a paper that ultimately gets published
• In 2023, the researcher starts a completely new projects. They create virtual environment B, which contains Python 3.10.1 and Nompie 2.0, to start working on the project with updated algorithms.
• An external collaborator contacts the researcher about their 2022 paper. They want the researcher to run the code under a slightly different scenario to test a hypothesis.
• The researcher simply uses virtual environment A to run the 2022 paper code for the external collaborator. Even if this leads to more involved work that requires additional Python packages, these can all be installed into virtual environment A with no impact to virtual environment B.
• Another third party, who has never heard of Nompie, is skeptical of the 2022 paper results and wants to reproduce its results. They can simply duplicate virtual environment A from the paper's GitHub page, without knowing anything about the difference between Nompie versions.

This example shows how the properties of virtual environments are useful in practice, especially in statistical research.

Disadvantages of Virtual Environments

Like any other choice regarding computational tools, using virtual environments comes with potential downsides

• It can be difficult to decide when to create a new virtual environment as your research progresses. Making too many virtual environments that do not differ substantially can lead to confusion, while making too few can cause the virtual environments to become bloated and outdated (since existing packages inside a virtual environment are usually not updated)
• Since every virtual environment is independent, packages used in multiple virtual environment need to be installed multiple times on the same machine. This can become a problem for very large packages, such as PyTorch.

Setting up Virtual Environments

See instructions for:

• Setting up Renv for R
• Setting up virtual environments for Python
• Setting up Mamba
• Setting up Conda with Python (Reference Only)

Additional Resources and Documentation:

• Renv Documentation
• Anaconda and Miniconda (Due to licensing issues, we do not recommend using these as your virtual environments)
• Mamba (A fully open source alternative to Anaconda/Miniconda)