Python on the server

There are multiple ways to run python on the server. To be the most productive coder, it’s important to understand how virtual environments can help product reproducibility and stability of your analysis (Python virtual environments) and how to keep running jobs even when you are not connected to the server (Terminal Multiplexers). If you’re less aware than you’d like, we encourage you to visit either of these pages. General Linux commands and getting connected & passing files between you and the server can all be found on our Linux and the department server page.

Submitting jobs via the command line

Before saving a code, you can start up python with a ipython or python call to the command line (possibly after activating a virtual environment). This will give you an interactive terminal to test python code out.

Beyond this iterative coding / explorative approach, you may find that writing up complex analysis that requires running a large number of lines of code (without having to manually run) large jobs that you’d like to run on a server, a common approach is to write a .py file to be run all at once, which can be done follows:

ipython filename.py
python filename.py

Especially when it comes to code run on the server, it’s possible that you’d like to take in a few non-fixed parameters when you run your file. This can be done through the following approach (and is also documented in a tutorial: geeksforgeeks).

In your python file (filename.py) (often at the top) you deal with arguments passed in from the command line:

import sys

input_arguments = sys.argv
arg1 = int(sys.argv[1]) # the first argument (sys.argv[0]) is the filename.
arg2 = sys.argv[2]
...

If you then ran the following in the command line:

ipython filename.py 1000 happy
python filename.py 1000 happy

You’d then find the lines below arg2 = sys.argv[2] would have arg1 = 1000 (an integer), and arg2 = "happy". Note that all parameters are pasted through as strings.

interactive ipython/python

Especially since you’ll be doing more of these command line running of files, if you are not 100% sure your code will run as expected or would like to be able to manipulate some of the objects you’ve made after run your file you can run your scripts interactively. If the script fails (or just when it ends), you’ll obtain an interactive ipython / python output. Below are two examples of how to do so (just use the -i command).

ipython -i filename.py 1001 sad
python -i filename.py 1001 sad

Jupyter notebooks via a server

You can still run jupyter notebooks from a server, which can be useful with large data sets, need for more computing power, or just the desire to not have your personal computer’s cores being taxed. Starting a jupyter notebook from the server is pretty simple, and we outline how to below (coderwall is has a good resource).

1. The first step is to initialize the notebook server just like you would on your own computer. On your remote server do:

   remote_server$ jupyter notebook --no-browser --port=8889

   Note that you should think about be running this code after a screen or tmux call so it lasts even if you close / lose connection to the the connection that ran this call.

2. Then on your local machine, you can run:

local$ ssh -N -f -L localhost:8888:localhost:8889 user_id@remote_server

3. Finally, you can return to the server terminal you launched the jupyter notebook from and you’ll see something like:

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://localhost:8889/?token=13cc54b4a3e13877cb07211346a363f9147a8cf2109263f1&token=13cc54b4a3e13877cb07211346a363f9147a8cf2109263f1

4. Copy this address but substitute 8889 with 8888 on your local machine’s web server.
5. start coding