Local Python and PyTorch Install - Summit Nodes

Local install of Python versions newer than 3.7

Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory ~/ on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq (for example, this gets you a 30 min session)

Once your interactive session is started, go to the directory where you put the python tarball and unzip it with:

tar -xzf Python-3.your.version.tgz

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /usr/local/ for the entire system, but this one lives in your home directory):

mkdir -p ~/local

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

./configure --prefix=$HOME/local --enable-optimizations

Note: --prefix needs to match where you created the user specific 'local' directory. Once that is complete, build python with:

make -j

Next, run the following to install the python executables and libraries in local.

make install

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run export PATH="$HOME/local/:$PATH". Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in ~/local, those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running which python3 in your terminal. This should show the path to the new python installed in ~/local/bin/.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

python3 -m venv NameOfYourEnv

This will create a folder named NameOfYourEnv which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

source ~/NameOfYourEnv/bin/activate.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run deactivate at any time.

While your venv is active, you should now be able to use python and pip instead of python3 and pip3 to run python or pip. Verify this with which python and which pip. These commands should now point to the ~/local/bin/ dir instead of the system wide installs from anaconda which are in /projects/tools/anaconda3/bin/.

pip installing python packages

While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

pip install --upgrade pip

Installing PyTorch or any other python package available through pip is now as simple as:

pip install torch (This is specifically for PyTorch)

Replicating a python environment

You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file pip freeze > requirements.txt. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

pip install -r ~/requirements.txt

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.