Terminal automation

Automating terminal commands

There are a variety of articles on this wiki that include “bash” commands, from Linux and department servers to Python on the server and R on the server. This article presents on two ways to bundle a set of commands (MakeFiles and .sh bash files). Both of these tools can help automate processes (like file and folder creation and running code) and can also “parallelize” your code. These techniques should work on your personal computer as well as the department servers as more.

What are they good for?

bash

Bash files are often used to automate the creation of files/folders, small changes of files, and processing a set of files in a similar manner. There are also ways to leverage bash commands to run parallel processes (in a slightly more hacky/more dangerous way than Slurm, but do not require Slurm syntax).

Please see the Bash files] article for a deeper dive into bash and .sh files, and 2 real examples (circa 2021).

make

The use of Makefiles and the make command focus a bit more on the description of processes and dependencies of steps of a process. Because of this, make is often used for describing and automating research pipelines (and can be seen if “photo”-ready code repositories are associated with a published paper).

Please see make for a deeper dive into make and Makefiles.