Local Python and PyTorch Install - Summit Nodes

PAGE UNDER CONSTRUCTION

Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory ~/ on any of the PHASTA group machines. You can do this in a multitude of ways, an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate: scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'

Next, log on to jumpgate and then start an interactive session on any of the summit nodes: qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

tar -xzf Python-3.your.version.tgz

make a new local dir if it doesn’t exist already mkdir -p ~/local cd into the python dir that has since appeared when you unziped the tarball. Next, configure the python build: ./configure --prefix=$HOME/local --enable-optimizations Once that is complete, build python with make -j Next, run the following to install the libraries in the local dir we just created. make install

Next, add ~/local to your $PATH if it's not already there. Do this by adding export PATH=~/local/:$PATH your .bashrc file. This will put ~/local to the start of PATH whenever you log on.

You should now have a working python installation. Verify by running which python3 in your terminal. This should show the path to the new python installed in the ~/local/bin/. To verify python itself can run, do python3 in your terminal.

Next, you should create a virtual environment for the project you are currently working on. Any pip installs you do while in this virtual env will be saved and you can easily activate that environment whenever you return to working on that project. From your home directory, run </code>python3 -m venv MyEnv</code> This will create a folder name MyEnv which will store the virtual environment.

You must activate this virtual environment with source ~/MyEnv/bin/activate. You should now see a (MyENV) tag at the start of your command line.