R on the server

General

Generally speaking (except suprisingly on the PSC), one can’t run Rstudio on a server. This means one needs to run R from the command line when you want to run jobs on the server. (It’s also more efficient when you’re looking to run lots of job.)

After getting onto the server (see Linux and the department server), you can start using R but opening an interactive session on the console with

R

You’ll be able to run code (except for visualization as the server can’t display graphics), from the console.

Library Management

Compared to your personal computer (and your personal computer using Rstudio), installing packages has a few nuances.

1. The first time installing a package, e.g.

   server$ R
   
   R > install.packages("tidyverse")

   you’ll be asked to set up a local folder to store your files.

2. The server’s version of R may different than you personal computer. It will also occasionally update. This will require you to re-install packages (seeRenv’s wiki (R package environments) for a way to make this less painful).
3. Compared to Rstudio, you may need to debug package installation and dependence structure. Good luck!

Running Jobs

Beyond just opening up an interactive R session with R, you can run files in 2 ways (as described below).

Rscript

Rscript is a more commonly used tool to run .R files, as it allows you to pass additional arguments to the file. For example:

Rscript file.R arg1 arg2

In order to use arg1 and arg2 in your R code, you’ll access them (as strings) using commandArgs(), for example:

args <- commandArgs(trailingOnly = TRUE)
if (length(args) > 0) {
  first_parameter <- args[1]
  second_parameter <- as.numeric(args[2])
  ...
}

Rscript also allows for standard output (printed to the terminal while running).

Example uses of arguments

Examples of argument values abound in statistical analysis. If you're using [Slurm|Slurm] you may use arguments to parallelize a task. These types of tasks could be running through complicated data observations, examining outcomes across a grid of hyperparameters, etc. It could also be useful to pass in the name of a file to process or folder to pull from.

R CMD BATCH

R CMD BATCH is an older approach (compared to Rscript you cannot pass in additional arguments nor see printouts to the standard output, but a file_name.Rout file will be generated with all print outs of the code.

R CMD BATCH file.R

If you don't have any input arguments to access and want to save errors and messages that came from running your .R file, this might be a better approach.

Additional useful tools

When approaching running code on different servers (and even on your own computer), it may be useful to understand the amount of time an analysis (or a part of an analysis) will run and how much memory it uses. This can help you decide what computing resources you need / what would make your analysis feasible. Knowing this information can also help you decide if you need to make your code more efficient (which can be done with the help of profiles and potentially C++ functions - link to come soon).

1. Estimating time usage: To measure the amount of time code takes to run, one approach is to put

   start <- Sys.time()

   at the start and at the end

   used_time <- Sys.time() - start
   print(used_time)

2. Understanding memory usage: When you are running code in R or RStudio, you can get some understanding of how large an object is (and therefore get a lower bound on your potential RAM usage) using the package pryr. Calling pryr::object_size(object_name) will tell you what the approximate size of the object is. See adv-r on the package usage. For example,

   R> library(pryr)
   R> dm_small <- as.matrix(dist(1:10))
   R> object_size(dm_small)
   2.1 kB
   R> dm_medium <- as.matrix(dist(1:10-))
   R> object_size(dm_medium)
   87.7 kB
   R> dm_large <- as.matrix(dist(1:1000))
   R> object_size(dm_large)
   8.07 MB
   R> dm_large <- as.matrix(dist(1:10000))
   R> object_size(dm_large)
   801 MB

   If we did

   R> dm_large <- as.matrix(dist(1:1000000))

   we'd expect object_size(dm_large) to be around 80 GB, which should crash your personal computer's R/Rstudio since you probably don't have that much RAM.

   
   
   

   It is possible to also check your amount of RAM usage with OS-centric tools (e.g. Mac's: Activity Monitor or Window's: Dask Management).