PHASTA Wiki - User contributions [en]

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:55:29Z

Jeffhadley: /* Local install of Python versions newer than 3.7 */

=Local install of Python versions newer than 3.7=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory '''~/''' on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code> (for example, this gets you a 30 min session)

Once your interactive session is started, go to the directory where you put the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to '''/usr/local/''' for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add '''~/local''' to '''$PATH''' if it's not already there. If you only want to do this for your current session, run <code>export PATH="$HOME/local/:$PATH"</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put '''~/local''' to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in '''~/local/''' before looking in the remaining directories of '''PATH'''. If you have other software installed in '''~/local''', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in '''~/local/bin/'''.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named '''NameOfYourEnv''' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a '''(NameOfYourEnv)''' tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

If you're submitting a batch job with qsub that will use this python environement, simply add the above source command to your job submission script (before any python executable calls) to ensure your venv is activated.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the '''~/local/bin/''' dir instead of the system wide installs from anaconda which are in '''/projects/tools/anaconda3/bin/'''.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:49:52Z

Jeffhadley: Jeffhadley moved page Python and PyTorch Install Summit Nodes to Local Python and PyTorch Install - Summit Nodes: Better name for the page

=Local install of Python versions newer than 3.7=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory '''~/''' on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code> (for example, this gets you a 30 min session)

Once your interactive session is started, go to the directory where you put the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to '''/usr/local/''' for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add '''~/local''' to '''$PATH''' if it's not already there. If you only want to do this for your current session, run <code>export PATH="$HOME/local/:$PATH"</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put '''~/local''' to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in '''~/local/''' before looking in the remaining directories of '''PATH'''. If you have other software installed in '''~/local''', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in '''~/local/bin/'''.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named '''NameOfYourEnv''' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a '''(NameOfYourEnv)''' tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the '''~/local/bin/''' dir instead of the system wide installs from anaconda which are in '''/projects/tools/anaconda3/bin/'''.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Python and PyTorch Install Summit Nodes

2026-02-19T22:49:52Z

Jeffhadley: Jeffhadley moved page Python and PyTorch Install Summit Nodes to Local Python and PyTorch Install - Summit Nodes: Better name for the page

#REDIRECT [[Local Python and PyTorch Install - Summit Nodes]]

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:47:52Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory '''~/''' on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code> (for example, this gets you a 30 min session)

Once your interactive session is started, go to the directory where you put the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to '''/usr/local/''' for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add '''~/local''' to '''$PATH''' if it's not already there. If you only want to do this for your current session, run <code>export PATH="$HOME/local/:$PATH"</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put '''~/local''' to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in '''~/local/''' before looking in the remaining directories of '''PATH'''. If you have other software installed in '''~/local''', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in '''~/local/bin/'''.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named '''NameOfYourEnv''' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a '''(NameOfYourEnv)''' tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the '''~/local/bin/''' dir instead of the system wide installs from anaconda which are in '''/projects/tools/anaconda3/bin/'''.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:43:25Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory '''~/''' on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to '''/usr/local/''' for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add '''~/local''' to '''$PATH''' if it's not already there. If you only want to do this for your current session, run <code>export PATH="$HOME/local/:$PATH"</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put '''~/local''' to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in '''~/local/''' before looking in the remaining directories of '''PATH'''. If you have other software installed in '''~/local''', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in '''~/local/bin/'''.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named '''NameOfYourEnv''' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a '''(NameOfYourEnv)''' tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the '''~/local/bin/''' dir instead of the system wide installs from anaconda which are in '''/projects/tools/anaconda3/bin/'''.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:41:36Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add '''~/local''' to '''$PATH''' if it's not already there. If you only want to do this for your current session, run <code>export PATH="$HOME/local/:$PATH"</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put '''~/local''' to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in '''~/local/''' before looking in the remaining directories of '''PATH'''. If you have other software installed in '''~/local''', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in '''~/local/bin/'''.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named '''NameOfYourEnv''' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a '''(NameOfYourEnv)''' tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the '''~/local/bin/''' dir instead of the system wide installs from anaconda which are in '''/projects/tools/anaconda3/bin/'''.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:33:30Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and Summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from [https://www.python.org/downloads/source/ python.org], and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball to your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the <code>~/local/bin/</code> dir instead of the system wide installs from anaconda which are in <code>/projects/tools/anaconda3/bin/</code>.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:31:42Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org[https://www.python.org/downloads/source/ python.org], and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the <code>~/local/bin/</code> dir instead of the system wide installs from anaconda which are in <code>/projects/tools/anaconda3/bin/</code>.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:28:50Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. To use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the <code>~/local/bin/</code> dir instead of the system wide installs from anaconda which are in <code>/projects/tools/anaconda3/bin/</code>.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:27:10Z

Jeffhadley: /* pip installing python packages */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. Tp use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the <code>~/local/bin/</code> dir instead of the system wide installs from anaconda which are in <code>/projects/tools/anaconda3/bin/</code>.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> (This is specifically for PyTorch)

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:26:40Z

Jeffhadley: /* Local install of Python versions newer than 3.7 available from base summit environment */

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. Tp use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code> to run python or pip. Verify this with <code>which python</code> and <code>which pip</code>. These commands should now point to the <code>~/local/bin/</code> dir instead of the system wide installs from anaconda which are in <code>/projects/tools/anaconda3/bin/</code>.

==pip installing python packages==
While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv. Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> This is specifically for PyTorch

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T22:19:10Z

Jeffhadley:

=Local install of Python versions newer than 3.7 available from base summit environment=
Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes. This explanation assumes you already know how to log on to jumgate and summit nodes, and have a good proficiency with terminal commands, bash, etc.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a 'user local' version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. If you only want to do this for your current session, run <code>export PATH=~/local/:$PATH</code>. Otherwise, if you always want it to appear at the front of PATH, add that same command somewhere that makes sense in your .bashrc file. Adding it to .bashrc will automatically put ~/local to the start of PATH whenever you log on. Note that compilers and bash commands will now look for executables and libraries located in ~/local/ before looking in the remaining directories of PATH. If you have other software installed in '~/local', those versions will always be accessed first.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual python environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch). Return to your home directory, then run

<code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'NameOfYourEnv' which will store all the requirements for the virtual environment. You only need to create the venv once. Tp use a venv you've created, you must activate it by sourcing it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (NameOfYourEnv) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time. While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv.

While your venv is active, you should now be able to use <code>python</code> and <code>pip</code> instead of <code>python3</code> and <code>pip3</code>. Verify this with <code>which python</code> and <code>which pip</code>, it should show the path to the <code>~/local/bin</code> python install.

==pip installing python packages==
Before pip installing any packages, update pip to the latest version available with

<code>pip install --upgrade pip</code>

Installing PyTorch or any other python package available through pip is now as simple as:

<code>pip install torch</code> This is specifically for PyTorch

=Replicating a python environment=
You have a working version of python and all your favorite packages on machine 'A', and you want to duplicate that in your new venv for the Summit nodes. You can easily do this with the following steps, ASSUMING YOU HAVE ALREADY INSTALLED NEWER VERSION OF PYTHON ON SUMMIT NODES AS WAS DONE ABOVE:

On machine A, export your current python environment set-up to a requirements.txt file <code>pip freeze > requirements.txt</code>. This file now lives in whatever directory you were in when you ran the command.

copy that file into your home directory on the summit nodes:

<code>scp -r /machine/A/path/to/requirements.txt username@jumpgate-phasta.colorado.edu:'~/.'</code>

Now, from within a session on one of the summit node, create and activate a clean venv as shown above, and then run the following:

<code>pip install -r ~/requirements.txt</code>

pip should take care of the rest. If any errors occur, read the error messages and modify the requirements.txt accordingly. Sometimes it's easiest to open the requirements.txt file and remove all of the version numbers (==x.x.x) from the packages listed, this permits pip to figure out the best dependency maps for the current active venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T21:15:54Z

Jeffhadley:

PAGE UNDER CONSTRUCTION

Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways; an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:

<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:

<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>

Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

We're now going to build and install a local version of python 3.your.version. First, make a new user specific 'local' dir if it doesn’t already exist (analogous to /local/ for the entire system, but this one lives in your home directory):

<code>mkdir -p ~/local</code>

Next, cd into the python dir that appeared when you unziped the tarball. We now need to configure the python build from within that directory:

<code>./configure --prefix=$HOME/local --enable-optimizations</code>

Note: --prefix needs to match where you created the user specific 'local' directory.
Once that is complete, build python with:

<code>make -j</code>

Next, run the following to install the python executables and libraries in local.

<code>make install</code>

We now need to add ~/local to $PATH if it's not already there. Do this by adding <code>export PATH=~/local/:$PATH</code> somewhere that makes sense in your .bashrc file. This will put ~/local to the start of PATH whenever you log on. Note that if you have other software installed in '~/local', this will always access these '~/local' versions first when calling their executables.

You should now have a working python-3.your.version installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in <code>~/local/bin/</code>.

Next, you should create a virtual environment for the project you are currently working on. Any pip installs you do while in this virtual environment will be saved. A venv permits you to easily 'activate' that same environment setup whenever you return to working on that project with a new session on the summit nodes (interactive or batch).
Return to your home directory, then run

</code>python3 -m venv NameOfYourEnv</code>

This will create a folder named 'MyEnv' which will store the virtual environment. You only need to do this once. Once you've created a venv you must activate it with:

<code>source ~/NameOfYourEnv/bin/activate</code>.

You should now see a (MyENV) tag at the start of your command line. If you ever need to deactivate this venv, simply run <code>deactivate</code> at any time.

While your venv of choice is active, you can pip install any of the python libraries you require, and the venv will save those installs for the next time you need to activate the venv.

Local Python and PyTorch Install - Summit Nodes

2026-02-19T20:46:04Z

Jeffhadley: Created page with "PAGE UNDER CONSTRUCTION Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes...."

PAGE UNDER CONSTRUCTION

Below is the process of obtaining and installing newer versions of Python than what is currently available through anaconda on the summit/dell nodes.

First, obtain a .tgz tarball of the python version you want from python.org, and put it in your home directory <code>~/</code> on any of the PHASTA group machines. You can do this in a multitude of ways, an easy way is to download the tarball your personal machine and scp'd it to your home directory on jumpgate:
<code>scp -r local/machine/path/to/python.tgz username@jumpgate-phasta.colorado.edu:'~/.'</code>

Next, log on to jumpgate and then start an interactive session on any of the summit nodes:
<code>qsub -I -l select=1:ncpus=16:mpiprocs=1 -q workq</code>
Once your interactive session is started, go to the directory where you scp'd the python tarball and unzip it with:

<code>tar -xzf Python-3.your.version.tgz</code>

make a new local dir if it doesn’t exist already <code>mkdir -p ~/local</code>
cd into the python dir that has since appeared when you unziped the tarball. Next, configure the python build:
<code>./configure --prefix=$HOME/local --enable-optimizations</code>
Once that is complete, build python with
<code>make -j</code>
Next, run the following to install the libraries in the local dir we just created.
<code>make install</code>

Next, add ~/local to your $PATH if it's not already there. Do this by adding <code>export PATH=~/local/:$PATH</code> your .bashrc file. This will put ~/local to the start of PATH whenever you log on.

You should now have a working python installation. Verify by running <code>which python3</code> in your terminal. This should show the path to the new python installed in the <code>~/local/bin/</code>. To verify python itself can run, do <code>python3</code> in your terminal.

Next, you should create a virtual environment for the project you are currently working on. Any pip installs you do while in this virtual env will be saved and you can easily activate that environment whenever you return to working on that project. From your home directory, run </code>python3 -m venv MyEnv</code>
This will create a folder name MyEnv which will store the virtual environment.

You must activate this virtual environment with <code>source ~/MyEnv/bin/activate</code>. You should now see a (MyENV) tag at the start of your command line.

The On Ramp/Level 1/Solve (Incompressible)

2025-04-30T19:44:56Z

Jeffhadley: /* Create Solver.inp */

=== Exporting to PHASTA ===
After the partitioning performed via Chef in the last steps, we now have the problem domain in a form that the PHASTA executable can read. In your 8-1-Chef directory, create a sub-directory named "Run". This will contain all of the simulation data from this case, which we will use in our PHASTA run. Remember that for this on-Ramp tutorial we have partitioned our case of interest to 8 parts. Therefore, we need to create a sub-directory named "8-procs_case" within the /8-1-Chef/Run/ directory. Once you have mkdir'd and cd'd into this new Run/8-procs_case/ subdirectory, make softlinks ("ln -s <path/file*>") to the N=8 restart and geombc "checkpoint" files that where constructed by Chef located in the 8-1-Chef/8-procs_case/ directory. When in the Run/8-procs_case sub-directory, a good command to do this is the following:

ln -s ../../8-procs_case/restart* .
ln -s ../../8-procs_case/geombc* .

Also create a numstart.dat file. This file will specify the time and timestep that the simulation has completed thus far. For our case, we have not yet run the simulation, thus our time and timesteps are 0 0. Use the following command in the "Run/8-procs_case" directory to create the numstart.dat file:

echo 0 0 > numstart.dat

=== Build the executable/specify runtime parameters===
What remains is to determine the version of PHASTA to build and run. Since there are a bunch of researchers working on PHASTA at any given time, there are many branches/versions of the main code.

'''Note:''' You may not have access to the phasta-next repo yet. If that is the case, ask Dr. Jansen to have you added to the repo. In the meanwhile, you can just use the regular phasta repo (by replacing all "phasta-next" with "phasta" in these instructions).

==== Retrieve/build a version of PHASTA code ====
To have your first run of PHASTA, you will need an executable of the PHASTA code. There are many ways to do this, and below is intended to get you a generic executable. The many nuances to this proccess can be found on the Level 2 page [[Compiling_PHASTA_With_CMake|here]]. Navigate to your home directory. Create and enter a directory here named "git-phasta". In a web browser, navigate to the online git repository for phasta-next and select the "clone" or "code" icon. The result should be similar to the following picture, where a pop-up gives a web address:

[[File:GitClone.png]]

Copy this address and within the "git-phasta" directory execute the following command and enter your github credentials:

git clone https://github.com/PHASTA/phasta-next.git

After this is finished there will be a subdirectory created named "phasta-next" that contains the code tree that you wish to build.
Back within the git-phasta directory create another subdirectory named "build_phasta-next". Now we must set the environment so that the compiler has the necessary libraries. If you perform the following command, the listed environment libraries that ken often uses are listed, and are often sufficient:

more ~kjansen/soft-core.sh

At the time that this page is created, the relevant commands to load the needed environment are the following:

soft add +gcc-6.3.0
soft add +openmpi-gnu-1.10.6-gnu49-thread
soft add +simmodeler-6.0-171202

If the user needs additional libraries they can often be found with the following command:

softenv

lets build this code! Navigate into this build_phasta-next subdirectory and create a file named "unpack_buildFiles.sh" with the following content:

#!/bin/bash

Target=Example_Build
mkdir Example_Build
rm -r $Target/*
cd $Target

export PKG_CONFIG_PATH=/users/skinnerr/tools/git-petsc/build_ompi210_gnu63/lib/pkgconfig/

cmake \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_CXX_COMPILER=g++ \
-DCMAKE_Fortran_COMPILER=gfortran \
-DCMAKE_BUILD_TYPE=Debug \
-DPHASTA_INCOMPRESSIBLE=ON \
-DPHASTA_COMPRESSIBLE=OFF \
-DPHASTA_USE_LESLIB=ON \
-DLESLIB=/users/matthb2/libles1.5/libles-debianjessie-gcc-ompi.a \
-DCASES=/home/mrasquin/develop-phasta/phastaChefTests \
-DPHASTA_TESTING=OFF \
../../phasta-next/

make -j8
echo "Target: $Target"
date

'''Note:''' You can create this file by typing <code>vi unpack_buildFiles.sh</code> into the command line. This will create an empty shell script file named unpack_buildFiles.sh and enter you into the Vim editor mode, where you can practice your recently adopted Vim commands to copy the above script into the file, making sure to save and quit after you're done. Enter the following command to make sure your shell script was saved successfully with all the required text:

more unpack_buildFiles.sh

Now, we must turn the above .sh file into an executable by typing the following command:

chmod +x unpack_buildFiles.sh

Finally, we are ready to run the executable and build the PHASTA code!

./unpack_buildFiles.sh

You can check that your executable has been built by locating:

Example_Build/bin/phastaIC.exe

===== Additional notes =====
If there is a specific branch off of phasta-next that you'd like to build, navigate to phasta-next and use the following command:
git checkout "branchname".
If this is a branch that I will be working on for a while, I tend to alter the build and code directories according to the branch-name. Note that the respective pointer in the "unpack_buildFiles.sh" file (ie the last line) will have to be set accordingly.

=== Create Solver.inp ===
The <code>input.config</code> file in the newly created <code>build_phasta/Example_Build/</code> directory contains all possible options that could be set in the <code>solver.inp</code> file (which we will use for our PHASTA run). Create the <code>solver.inp</code> file in the <code>.../PrepAndRun/8-1-Chef/Run/</code> directory, and then specify all of the parameters that you wish to change for your run case. '''NEVER EVER Modify the input.config file itself'''. The rest of the parameters need not be specified. For example my solver.inp looks as follows:

# ibksiz flmpl flmpr itwmod wmodts dmodts fwr taucfct
# PHASTA Version 1.5 Input File
#
# Basic format is
#
# Key Phrase : Acceptable Value (integer, double, logical, or phrase
# list of integers, list of doubles )
#
#
#SOLUTION CONTROL
#{
Equation of State: Incompressible
Number of Timesteps: 10
Time Step Size: 1e-1 # Delt(1)
Turbulence Model: RANS # No-Model # DES97 # DDES iturb=0, RANS =-1 LES=1 #}
#}

#MATERIAL PROPERTIES
#{
Viscosity: 1.50e-5 # fills datmat (2 values REQUIRED if iLset=1)
Density: 1.0 # ditto
Body Force Option: None # ibody=0 => matflag(5,n)
Body Force: 0 0.0 0.0 # (datmat(i,5,n),i=1,nsd)
Thermal Conductivity: 27.6e-1 # ditto
Scalar Diffusivity: 27.6e-1 # fills scdiff(1:nsclrS)
#}

OUTPUT CONTROL
{
Number of Timesteps between Restarts: 5 #replaces nout/ntout
Number of SyncIO Files: 0
Print Error Indicators: False
Number of Error Smoothing Iterations: 0 # ierrsmooth
# Print ybar: True
# Print vorticity: True
# Print Wall Fluxes: False
# Print Statistics: False
Number of Force Surfaces: 1
Surface ID's for Force Calculation: 1
# Ranks per core: 4 # for varts only
# Cores per node: 16 # for varts only
}

#LINEAR SOLVER
# Solver Type: GMRES sparse
Solver Type: ACUSIM with P Projection
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Number of Krylov Vectors per GMRES Sweep: 200 # replaces Kspace
Scalar 1 Solver Tolerance : 1.0e-4
Tolerance on Momentum Equations: 0.05 # epstol(1)
Tolerance on ACUSIM Pressure Projection: 0.01 # prestol
Number of Solves per Left-hand-side Formation: 1 #nupdat/LHSupd(1)
ACUSIM Verbosity Level : 0 #iverbose
Minimum Number of ACUSIM Iterations per Nonlinear Iteration: 10 # minIters
Maximum Number of ACUSIM Iterations per Nonlinear Iteration: 200 # maxIter
#}

#DISCRETIZATION CONTROL
#{
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
# Time Integration Rule: Second Order # Second Order sets rinf next
# Time Integration Rho Infinity: 0.0 # rinf(1) Only used for 2nd order
Tau Matrix: Diagonal-Shakib #itau=1
Tau Time Constant: 1.0 #dtsfct
Include Viscous Correction in Stabilization: True # if p=1 idiff=1
# if p=2 idiff=2
Lumped Mass Fraction on Left-hand-side: 0.0 # flmpl
Lumped Mass Fraction on Right-hand-side: 0.0 # flmpr
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 # switch to >250 if sgi
#}

TURBULENCE MODELING PARAMETERS
{
Turbulence Wall Model Type: None #itwmod=2 RANSorLES
}

#STEP SEQUENCE
#{
Step Construction : 0 1 10 11 0 1 10 11
#}

=== Running the Solver ===
Create the runPHASTA.sh bash script in your <code>8-1-Chef/Run</code> directory. Where you see <pathtoBuildDir> include your path to your build directory where you retrieved the <code>input.config</code> file. The #export line is an example that I have used myself, yours should look similar:

#!/bin/bash

rm $1-procs_case/doubleRun-check

#export PHASTA_CONFIG=/users/jopa6460/git-phasta/build_phasta/Example_Build
export PHASTA_CONFIG=<pathtoBuildDir>

mpirun -np $1 $PHASTA_CONFIG/bin/phastaIC.exe 2>&1 | tee $1.out

Remember to turn the file into an executable as was done for the build script above. Hint: <code>chmod +x</code> command.

NOTE: Before running PHASTA, it is important to check that nobody else is running important jobs on the computer you're connected to. Run <code>top</code> in the command line to check what other processes are running, and if you see other people running large jobs you should ask them if you can run your job as well, or else switch to a different computer.

Now you have everything you need to run your first PHASTA simulation! In the Run directory execute the following line:

./runPHASTA.sh 8

Output of a proper run of PHASTA contains information about 1) the step number of the simulation; 2) the relative residual (convergence relative to this run's initial residual); and 3) various other outputs based on runtime parameters like BC's and IC's. As the simulation continues it will produce successive checkpoint restart files that contain the solution data at that timestep. For our example, these restart files will be located in the <code>.../Run/8-procs_case</code> directory. These restart files and the geombc files are what Paraview post-processes to give us a visualization of flow field solutions.

Now it's time to plot the results in the [[Level 1 Post-Process]] step.

=== A few helpful video tutorials===

[https://fluid.colorado.edu/tutorials/tutorialVideos/CloneFromGithubAndBranch.mov CloneFromGithubAndBranch.mov ]

[[Tutorial_Video_Overviews#CloneFromGithubAndBranch.mov|Video Notes]]

[https://fluid.colorado.edu/tutorials/tutorialVideos/BlancoBuidingPHASTA.mov BlancoBuidingPHASTA.mov ]

[[Tutorial_Video_Overviews#BlancoBuidingPHASTA.mov|Video Notes]]

[https://fluid.colorado.edu/tutorials/tutorialVideos/PHASTA_workflow_RB.mkv PHASTA_workflow_RB.mkv ]

[[Tutorial_Video_Overviews#PHASTA_workflow_RB.mkv|Video Notes]]

[https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemoPrepSolvePost.mov RajDemoPrepSolvePost.mov ]

[[Tutorial_Video_Overviews#RajDemoPrepSolvePost.mov|Video Notes]]

[https://fluid.colorado.edu/tutorials/tutorialVideos/PrepSolvePostBLandC.mp4 PrepSolvePostBLandC.mp4 ]

[[Tutorial_Video_Overviews#PrepSolvePostBLandC.mp4|Video Notes]]

PHASTA Group Machines

2024-05-28T16:08:15Z

Jeffhadley: /* Setting Up Two-Factor Authentication */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple next steps you will take in order to access and interact with the PHASTA machines. wThe steps in the video are more thoroughly documented in the remainder of this page and the [[VNC]] page of the wiki.

To get started, open a command line terminal, enter <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>, and the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal on your personal computer and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>viz002</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-05-28T15:41:21Z

Jeffhadley:

If you're looking how to build a PETSc version to use with libCEED Fluids Example (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run, which is currently 1 directory above the one you should be in.
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

To run the job in Totalview, simply add <code>-tv</code> after <code>-np 2</code> in the previous command. To see how to debug in Totalview I suggest to watch the video linked at the top of this page starting at timestamp 16:20.

Tada, you've built and run the HONEE fluids solver using libCEED and PETSc!

Spack

2024-03-06T21:41:56Z

Jeffhadley: Initial set up of this page

Viznodes now has Spack! Wooo!

To install Spack in your environment, run the following command:
source /usr/local/spack/v0.16/load.sh

To see how to use Spack to install packages, refer to their [https://spack.readthedocs.io/en/latest/basic_usage.html Basic Usage] website

The On Ramp

2024-02-28T22:57:45Z

Jeffhadley: /* Level 2 */

Welcome to the '''PHASTA On Ramp'''! This page is meant to organize information for getting up to speed with the overall PHASTA workflow, as well as help you up your game as a PHASTA developer.

== Level 0 ==
Ready to get your feet wet? Maybe you need to brush up on some fundamental skills. Level 0 provides tutorials and information on basic developer skills you will need to work with PHASTA.
* [[PHASTA_Group_Machines|Logging in]]
* [[VNC|Setting up your VNC]]
* [[UNIX|Getting started with Linux]]
* [[Git | Getting Git]]
* [[Vim|Getting comfortable with Vim]]
* [[Fortran|New to Fortran?]]
* [[Making A New Wiki | Editing this Wiki]]

== Level 1 ==

[[File:Picture5.png]]

So you think you know how to <code>grep</code> for keywords and <code>mkdir</code> some folders? Now you're probably ready to jump into PHASTA.

* [[Level 1 Model/Mesh| Model/Mesh Workflow]]
* [[Level 1 Partition| Partition Workflow]]
* [[Level 1 Solve| Solve Workflow]]
* [[Level 1 Post-Process| Post-Process Workflow]]

See the [[The_On_Ramp/Level_1|Level 1 base page]] as well.

== Level 2 ==
Now that you've plotted some "Colorful Fluid Dynamics" (CFD) you're probably interested in digging into the guts of PHASTA, and getting it running on some more interesting hardware.
* [https://github.com/PHASTA/phasta Building PHASTA from scratch] (Follow the README at the bottom of the page)
* [https://github.com/SCOREC/core/wiki/General-Build-instructions#Manual_Install Building Chef and other SCOREC core tools]
* [[ Building ParaView 5.8.1 and Shoreline Plug-ins on viz003 | Building ParaView with Shoreline Plug-ins ]]
* [https://fluid.colorado.edu/wiki/index.php/Building_PETSc_and_libCEED_to_run_HONEE_Fluid_solver_on_Viznode Building PETSc and libCEED to run HONEE Fluid solver on Viznode]

== Level 3 ==
ParaView isn't cutting it for your research? Now its time to start writing your own analysis code. How to do custom pre and post processing is described here, along with more complex ways of using PHASTA for interesting CFD simulations
* [[Synthetic Turbulence Inflow Generator | Synthetic Turbulence Generator]]
* [[VTKpytools| Python + VTK using vtkpytools]]

== Level 4 ==
You're probably close to defending at this point, so before you go get that sweet industry job, you should drop some of your lofty knowledge here!

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:48:16Z

Jeffhadley: Jeffhadley moved page Building PETSc and libCEED to Building PETSc and libCEED to run HONEE Fluid solver on Viznode: Page title was not descriptive enough

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run, which is currently 1 directory above the one you should be in.
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

To run the job in Totalview, simply add <code>-tv</code> after <code>-np 2</code> in the previous command. To see how to debug in Totalview I suggest to watch the video linked at the top of this page starting at timestamp 16:20.

Tada, you've built and run the HONEE fluids solver using libCEED and PETSc!

Building PETSc and libCEED

2024-02-28T22:48:16Z

Jeffhadley: Jeffhadley moved page Building PETSc and libCEED to Building PETSc and libCEED to run HONEE Fluid solver on Viznode: Page title was not descriptive enough

#REDIRECT [[Building PETSc and libCEED to run HONEE Fluid solver on Viznode]]

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:45:00Z

Jeffhadley: /* Running a Fluids Example case */

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:44:10Z

Jeffhadley: /* Getting PETSc and LibCEED */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

To run the job in Totalview, simply add <code>-tv</code> after <code>-np 2</code> in the previous command. To see how to debug in Totalview I suggest to watch the video linked at the top of this page starting at timestamp 16:20.

Tada, you've built and run the HONEE fluids solver using libCEED and PETSc!

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:42:42Z

Jeffhadley: /* Running a Fluids Example case */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

To run the job in Totalview, simply add <code>-tv</code> after <code>-np 2</code> in the previous command. To see how to debug in Totalview I suggest to watch the video linked at the top of this page starting at timestamp 16:20.

Tada, you've built and run the HONEE fluids solver using libCEED and PETSc!

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:41:35Z

Jeffhadley:

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

To run the job in Totalview, simoply add <code>-tv</code> after <code>-np 2</code> in the previous command. To see how to debug in Totalview I suggest to watch the video linked at the top of this page starting at timestamp 16:20.

Tada, you've built and run the HONEE fluids solver using libCEED and PETSc!

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:35:16Z

Jeffhadley: /* Running a Fluids Example case */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

We will now run the example case. To do so run the following command:
mpirun -np 2 ../navierstokes -options_file blasiusGenIC.yaml -ts_max_time 10

Explanation of above command:
'-np 2' is running the job on 2 processors
'../navierstokes' is the executable we want to run
'-options_file blasisuGenIC.yaml' is passing the initial conditions file to the solver
'-ts_max_time 10' is changing the ts_max_time initial condition in the IC file from 0 (set in file) to 10. This is an example of
modifying an options_file from the command line.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T22:24:36Z

Jeffhadley: /* Running a Fluids Example case */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/STGRand_12-30.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>. Note that the changes made to the <code>.yaml</code> file in the video are already applied to the one you just copied.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T21:46:27Z

Jeffhadley: /* Running a Fluids Example case */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tutorials/PETSc_libCEED_Demo/blasiusGenIC.yaml .
cp /projects/tutorials/PETSc_libCEED_Demo/STGInflow_12-30_SPD.dat .
cp /projects/tutorials/PETSc_libCEED_Demo/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T21:42:07Z

Jeffhadley: /* Running a Fluids Example case */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Next we need to copy over the initial condition, mesh, and inflow files:
cp /projects/tools/libCEED/libCEED/examples/fluids/blaResizedQ1wQ3mesh/blasiusGenIC.yaml .
cp /projects/tools/libCEED/libCEED/examples/fluids/blaResizedQ1wQ3mesh/STGInflow_12-30_SPD.dat .
cp /projects/tools/libCEED/libCEED/examples/fluids/blaResizedQ1wQ3mesh/everyThird_yspacing.dat .

Timestamp 9:30 in the video linked above dives into the Initial Condition file <code>blasiusGenIC.yaml</code>.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T21:11:23Z

Jeffhadley: /* Building libCEED and the Fluids Solver */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. To do so run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

== Running a Fluids Example case ==

We will now set up to run an example case. There's a few files that we need to grab in order to run the example, so within the <code>.../libCEED/examples/fluids/</code> directory create a new directory and cd into it:
mkdir FluidsDemo
cd FluidsDemo

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T21:08:24Z

Jeffhadley: /* Getting PETSc and LibCEED */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git libCEED

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. Run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T21:08:11Z

Jeffhadley: /* Building PETSc */

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place! The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

== Getting PETSc and LibCEED ==

First things first, let's get the source code for PETSc and libCEED. You can get clone the sources from the following:
PETSc SSH: git@gitlab.com:petsc/petsc.git
PETSc HTTPS: https://gitlab.com/petsc/petsc.git

libCEED SSH: git@github.com:CEED/libCEED.git
libCEED HTTPS: https://github.com/CEED/libCEED.git

Clone the repositories somewhere that makes sense for you. For example I have mine in <code>jeffhadley/Builds/src/</code>:

PETSc
git clone -b release https://gitlab.com/petsc/petsc.git petsc
git pull # obtain new release fixes (since a prior clone or pull)
libCEED
git clone https://github.com/CEED/libCEED.git

You can replace the https url's with the SSH ones from above if you have SSH setup.

== Setting up Environment ==
Next we need to make sure we have the correct versions of gcc, openmpi, totalview, and Python. We will soft add gcc, openmpi and totalview with the following commands:
soft add +gcc-10.2.0
soft add +openmpi-gnu-4.0.2-gnu49-thread
soft add +totalview-2023.2.15

The version of Python available as a soft add is not sufficient to run PETSc and libCEED, so we need to enable a newer version through Anaconda. Paste the following block of code at the end of your <code>~/.bashrc</code> or <code>~/.zshrc</code> file:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/projects/tools/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/projects/tools/anaconda3/etc/profile.d/conda.sh" ]; then
. "/projects/tools/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/projects/tools/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

This block of text will automatically activate Conda when you open a new terminal window, which we do not actually want to happen. To fix this, run the following command:
conda config --set auto_activate_base False

You now need to manually activate Conda whenever you want a more up to date version of Python. To do so run the following command:
conda activate

Check the freshly enabled version of Python with <code>python --version</code>. The most up to date version as of 02/28/2024 is 3.7.9. For more information on using Anaconda and its sub-packages, see the wiki page [https://fluid.colorado.edu/wiki/index.php/Anaconda here].

== Building PETSc ==
Next we will build PETSc. Navigate into the top level of the cloned PETSc directory. We need to run a set-up script that's different than what comes natively with the clone of the repository, so run the following copy command to grab the set-up script which we will use for the VizNodes:
cp /projects/tools/petsc/petsc/testDebugSetup.py .

Next we will run this script to configure the Makefiles we will need to build the PETSc package:
python testDebugSetup.py

Once the configure step is complete you will get a message in your terminal stating it's complete, and that you can now compile PETSc with a command similar to the following (Note: copy and paste the one generated in your terminal, the following is just an example of what it will look like):
make PETSC_DIR=/users/username/location/of/petsc/ PETSC_ARCH=64DebTest all

This will run the Makefiles and build the PETSc libraries for the VizNodes. Once PETSc succesfully builds, you will have a message stating "Now to check if the libraries are working do:". We will skip this step and instead export the PETSc directory and architecture names to consol with the following:
export PETSC_DIR=/users/username/location/of/petsc
export PETSC_ARCH=64DebTest

Note: the PETSC_DIR and PETSC_ARCH will be printed in the "...check if libraries..." message, so best to just copy and paste those into the export command.

We need to export these variables as the fluids example executable we will later create with libCEED is going to require them.

To make sure the libraries built successfully run the following command and confirm there are <code>.so</code> files there:
cd $PETSC_ARCH/lib/

== Building libCEED and the Fluids Solver ==
Next we will build libCEED, so navigate to the top level of the cloned repository. Nothing special to do here, simply run:
make -j

This should take all of 1 second to build. Next we will navigate to the Fluids example folder to build the fluids solver. Run the following commands:
cd examples/fluids/
make -j

The fluids solver executable is called <code>navierstokes</code>, verify that you can locate it in the <code>.../libCEED/examples/fluids/</code> directory.

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T20:55:46Z

Jeffhadley: /* Building PETSc */

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T20:43:09Z

Jeffhadley: /* Building PETSc */

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T20:40:24Z

Jeffhadley:

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T20:27:16Z

Jeffhadley:

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T20:16:42Z

Jeffhadley:

Building PETSc and libCEED to run HONEE Fluid solver on Viznode

2024-02-28T18:11:52Z

Jeffhadley: First page save. Working progress.

If you're looking how to build a PETSc version to use with libCEED (HONEE) you've come to the right place. The following steps will summarize and include the commands needed to perform the install process and example run we followed in this [https://fluid.colorado.edu/tutorials/tutorialVideos/PETSc_libCEED_BuildRunTotalview.mp4 video].

PHASTA/Reduce Restart and Geombc Files

2023-10-31T16:51:03Z

Jeffhadley: /* M2N_input.dat */

== Reduction from M to N parts in parallel for PHASTA files initially partitioned with NSpre_parallel, phParAdapt-SCOREC or chef ==

=== Prerequisites ===
The tool described in this section apply to phasta files initially partitioned with phParAdapt-SCOREC or chef. We consider here that M parts were generated from N parts with M>N (M is usually a multiple of N). A vertex mapping between the M parts and N parts needs to be generated by phParAdapt-SCOREC or chef through the variable "buildMapping 1" in adapt.inp.

=== Building <code>M2N</code> and <code>M2NFixBnd</code> ===
* Download the M2N and M2NFixBdn packages from
cd your-develop-dir-cmake/phasta
git clone ssh://username@jumpgate-phasta.colorado.edu/users/mrasquin/develop-FlowControl-SyncIO-Cmake/phasta/M2N
git clone ssh://username@jumpgate-phasta.colorado.edu/users/mrasquin/develop-FlowControl-SyncIO-Cmake/phasta/M2NFixBnd

* Build both packages with cmake. For instance, on the viz nodes with gcc:

cd your-develop-dir-cmake

mkdir build-M2N
cd build-M2N
CC=gcc CXX=g++ FC=gfortran cmake -DCMAKE_BUILD_TYPE=Release ../phasta/M2N/src
make -j 8 VERBOSE=1
cd ..

mkdir build-M2NFixBnd
cd build-M2NFixBnd
CC=gcc CXX=g++ FC=gfortran cmake -DCMAKE_BUILD_TYPE=Release ../phasta/M2NFixBnd/src
make -j 8 VERBOSE=1
cd ..

=== M2N ===

==== Running <code>M2N</code> ====

<code>M2N</code> must be run on M processes

mpirun -np M your-develop-dir-cmake/build-M2N/M2N

==== M2N Inputs ====
M2N reads:
* The geombc files files under the SyncIO format (M parts);
* The solution, dwal, error and ybar fields from a set of restart files under the SyncIO format (M parts) for adaptation or post-processing later;
* The vertex mapping field from a set of restart files under the SyncIO format (M parts).
* An input file named "M2N_input.dat" described below

==== M2N Outputs ====
M2N produces:
* N restart files under the Posix format, which is what phParAdapt requires for now.

==== M2N_input.dat ====

Note that usually, the vertex mapping field is included in the restart files of the first time step of your computation, whereas the solution, dwal, error and dwal come from the last time step of the computation.

The "M2N_input.dat" must include the following information:
120 # time step of the restart files for fields to reduce (solution, ybar, etc)
100 # time step of the restart files for partID and vtxID (mapping between M and N)
1 # Reduce the ybar field? (0/1)
1 # Reduce the error field? (0/1)
2 # Reduce the phase_average field? (0/n phase average fields)
1 # Reduce the vorticity field? (0/1)
1 # Reduce the dwal? (0/1)
1 # Print debug statement (0/1)

### What follows is a short manual and is not read by M2N ###
This Script is used to reduce different fields from M parts to N parts, using the mapping fields (partID. vtxID) gerenated by phParAdapt.
By default, this script reduces the solution field but it can also handle others fields such as mentioned above.

=== M2NFixBnd ===
The value of the solution, dwal, errors and ybar fields still need to be updated on the part boundaries of the N parts, which is the purpose of M2NFixBnd

==== Running M2NFixBnd ====

<code>M2NFixBnd</code> is run on N processes.

mpirun -np N your-develop-dir-cmake/build-M2NFixBnd/M2NFixBnd

==== M2NFixBnd Inputs ====

M2NFixBnd reads
* The solution, dwal, errors and ybars field from the N restart files under the Posix format produced by M2N in the previous step;
* The ilwork array from the N geombc files under the Posix format that matches the restart files produced by M2N.

==== M2NFixBnd Outputs ====
M2NFixBnd produces
* N restart files under the posix format that overwrite the ones generated by M2N.

=== Example ===

See

/users/mrasquin/Models/CrossFlowTest/1-A0-Mixed-9.0/SyncIO-phPA/Runs-pumi-Tet-ts100/ReduceM2N-16TetPumi-8MixedSim/runReduceM2N.sh

== Reduction from M parts to 1 part in serial for phasta files initially partitioned with NSpre_parallel ==

*cd to the procs_case directory that contains the files you want to reduce
*Run the executable:
~mrasquin/develop/phasta/phPost/phPost/bin/x86_64_linux-icc/reduce-openmpi-O -sn
where sn is the time step number. For example, the restart files corresponding to time step 5 would be restart.5.* and geombc.dat.* .
*The reduce operation will generate restart.##.0 in the procs_case directory, where ## stands for the step number you have reduced.
*restart.##.0, combined with geombc.dat.1 (which contains the boundary conditions for all the parts - it is created when you run NSPre_Parasolid), are what you need to see the solution on 1 part.

[[Category:PHASTA]]

VNC

2023-10-04T20:58:00Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the <code>ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu</code> command, where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of <code>ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu</code>

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA machine side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
- Same concept as above... close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name:
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH.
- Next, look to the panel on the left side of the PuTTY window, find the "Connection -> SSH -> Tunnels" menu option and click on "Tunnels". This will
open a settings page for the tunnel we are trying to establish.
- In the "Source Port" field enter <code>59zw</code>, and in the "Destination" field enter
<code>portal1:59zw</code>, again replacing <code>zw</code> with your assigned port number.
- Click the "Add" button.
- Click on "Open". You now have an established VNC connection with <code>portal1</code> and it will ask you to enter your (jumpgate) password.
- Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

VNC

2023-10-04T20:56:52Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the <code>ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu</code> command, where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of <code>ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu</code>

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA machine side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
- Same concept as above... close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name:
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH.
- Next, look to the panel on the left side of the PuTTY window, find the "Connection -> SSH -> Tunnels" menu option and click on "Tunnels". This will
open a settings page for the tunnel we are trying to establish.
- In the "Source Port" field enter <code>59zw</code>, and in the "Destination" field enter
<code>portal1:59zw</code>, again replacing <code>zw</code> with your assigned port number.
- Click the "Add" button.
- Click on "Open". You now have an established VNC connection with <code>portal1</code>.
- Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

PHASTA Group Machines

2023-09-21T18:29:16Z

Jeffhadley: /* Logging In */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple next steps you will take in order to access and interact with the PHASTA machines. wThe steps in the video are more thoroughly documented in the remainder of this page and the [[VNC]] page of the wiki.

To get started, open a command line terminal, enter <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>, and the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, it is recommended that you start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>viz002</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

PHASTA Group Machines

2023-09-21T18:28:39Z

Jeffhadley: /* Logging In */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple next steps you will take in order to access and interact with the PHASTA machines. wThe steps in the video are more thoroughly documented in the remainder of this page and the [[VNC]] page of the wiki.

To get started, open a command line terminal, and the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, it is recommended that you start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>viz002</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

PHASTA Group Machines

2023-09-21T18:27:13Z

Jeffhadley: /* Logging In */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple next steps you will take in order to access and interact with the PHASTA machines. wThe steps in the video are more thoroughly documented in the remainder of this page and the [[VNC]] page of the wiki.

In the command line terminal, the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, it is recommended that you start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>viz002</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

PHASTA Group Machines

2023-09-21T18:25:30Z

Jeffhadley: /* Logging In */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple steps which are documented in the Logging In and the VNC portion of the wiki.

In the command line terminal, the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, it is recommended that you start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>viz002</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

VNC

2023-09-21T18:05:06Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the <code>ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu</code> command, where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of <code>ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu</code>

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA machine side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
Same concept, so close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name, replacing zw with your assigned port number:
<code>-L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH. In the "Port" field, enter the <code>zw</code> assigned port number. Click on "Open". You now have an established VNC connection with <code>portal1</code>. Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

VNC

2023-09-21T18:03:26Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the <code>ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu</code> command, where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of <code>ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu</code>

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA Machien side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
Close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name, replacing zw with your assigned port number:
<code>-L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH. In the "Port" field, enter the <code>zw</code> assigned port number. Click on "Open". You now have an established VNC connection with <code>portal1</code>. Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

VNC

2023-09-21T18:03:06Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the <code>ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu</code> command

where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of <code>ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu</code>

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA Machien side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
Close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name, replacing zw with your assigned port number:
<code>-L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH. In the "Port" field, enter the <code>zw</code> assigned port number. Click on "Open". You now have an established VNC connection with <code>portal1</code>. Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

VNC

2023-09-21T18:00:08Z

Jeffhadley: /* Start VNC Server (Step 1) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59zw on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the following command

ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu

where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of:

ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA Machien side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
Close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name, replacing zw with your assigned port number:
<code>-L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH. In the "Port" field, enter the <code>zw</code> assigned port number. Click on "Open". You now have an established VNC connection with <code>portal1</code>. Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.

VNC

2023-09-21T17:55:08Z

Jeffhadley: /* Clients (Step 2) */

'''Virtual Network Computing (VNC)''' is a tool which projects a GUI session over the network. If may be useful if you want to use GUI tools remotely when X forwarding performs poorly.

VNC works on a client-server architecture, where a remote system (server) runs the programs and sends the virtual screen output to the local system (client). Here, the local system would be your laptop/desktop, while the server is located on the [[PHASTA Group Machines]].

'''For On Ramp purposes, you only need to follow the sub sections identified with steps 1 through 3: Start VNC Server, Clients, and Starting a VNC Viewer'''

== VNC on PHASTA Machines ==
'''Warning: The VNC password is transmitted in clear text over the network and should not be considered secure'''

== Server-Side Setup for PHASTA Machines ==
=== Start VNC Server (Step 1) ===
<code>portal1</code> is designated to host VNC sessions.

First connect to jumpgate via ssh in a terminal on your personal machine (If following the On Ramp you should already have this terminal opened). To start a VNC session on portal1 type the following in your jumpgate terminal with a return after each line:
ssh portal1
source /etc/profile.d/vncscript.sh
start_vnc.sh

This will setup the VNC viewer for your session on the PHASTA machine side of things. Read through the output from <code>start_vnc.sh</code> as it contains crucial information and some tips which are elaborated on in this wiki.

IMPORTANT: Make sure to remember the generated password and port number ("You should connect to 59** on portal1", "Your password is abcd1234ExamplePW") so that you can reuse the session you just started. Hint: take a screenshot of the output of <code>start_vnc.sh</code> and store it somewhere safe.

It is common practice to leave your VNC session running on <code>portal1</code>. Next time you want to access your desktop, just ssh into jumpgate with a tunnel between portal1's VNC port (59**) and some port on your machine. Then use a VNC client to connect to the port on your machine. See the Client (Step 2) section below for an explanation of this process.

=== Killing a VNC Server ===
If for some reason you want to end your session and kill your virtual desktop, run

source /etc/profile
stop_vnc.sh

ONLY run this if you want to kill your virtual desktop. Most users will never need to do this as the idea is to create one session and continue to use that one for all future usage.

=== Changing the VNC Password ===
While ssh'd into your session on <code>portal1</code>, type the following command in your terminal:
/opt/tigervnc/bin/vncpasswd

Follow the on screen instructions.

=== View Only Mode ===

To share your desktop with another user in view only mode set a view only password
by running
vncpasswd

Have the other user connect in the same way you would but have them set their viewer to be in view only mode and use your view only password. Typically this is done as follows:
vncviewer -viewonly

=== Changing the Size (Resolution) of an Existing Session ===
''Note: In most modern VNC clients, this can (possibly should) be done via the locally running client. See your VNC client's documentation for details. The below is kept for posterity''

You can usually use the <code>xrandr</code> tool to change the resolution of a running VNC session. First you'll need to know your session's display number (this should be the last digit or two of the port number). For example, if your VNC session is running on port 5902, then your screen number should be :2. For this example, we'll use screen 2.

Once you know your screen number, you can see the list of supported modes as follows:
xrandr -display :2

Once you pick the one you want (generally the same size or smaller than the native resolution of your client), you can choose it by running a command like
xrandr -s 1400x1050 -display :2

(this example will set the resolution to 1400 pixels by 1050 pixels)

You'll probably be disconnected at this point, but when you reconnect your screen size should be changed (hopefully without crashing your running programs).

=== Finding an Existing Session ===
SSH to portal1 and then run:
/opt/vnc_script/findsession.sh

Which will return the shortened port number of each of your currently running sessions.

== Client-Side Setup for PHASTA Machines ==
=== Clients (Step 2) ===

First you need to have a VNC viewer installed on your personal machine. <code>portal1</code> uses TurboVNC from the VirtualGL project, available from [http://www.virtualgl.org/Downloads/TurboVNC their website]

Other VNC viewers will also work, such as [https://www.tightvnc.com/download.php TightVNC] and [https://www.realvnc.com/en/connect/download/viewer/ RealVNC]. Download and install the one you like best for your personal machine (RealVNC seems to be the go to for Mac users in the group).

The instructions given when you started the session on <code>portal1</code> (the ones you should have taken a screen shot of in Step 1 above) are OK, but it always tells you to start a session (in a terminal on your mac or linux machine command line) with the following command

ssh -L5905:portal1:59zw jumpgate-phasta.colorado.edu

where <code>zw</code> will be different for each user. That suggestion is OK if you never plan to connect to anyone else's session, but since we often collaborate by sharing VNC sessions, the better practice we adopt is to use <code>zw</code> in place of the suggested 05 (which just an arbitrary local port on your laptop). For example, when I created a session the last four numbers of the connection were <code>5923</code>, thus <code>zw</code> for me is <code>23</code> and the best practice is to ignore the script suggestion in favor of:

ssh -L5923:portal1:5923 jumpgate-phasta.colorado.edu

The easiest way to set up the portal from the get-go (after you have created the VNC session on the PHASTA Machien side of things and left it open, AKA Step 1 above) is to close out of the terminal on your personal machine, reopen it, and run only this command where <code>zw</code> is your assigned port number:

ssh -L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu

'''Windows 8'''
Close the terminal that was established with PuTTY. Start a new one using PuTTY and entering the following in the Host Name, replacing zw with your assigned port number:
<code>-L59zw:portal1:59zw USERNAME@jumpgate-phasta.colorado.edu</code>
Again ensuring connection type is set to SSH. In the "Port" field, enter the <code>zw</code> assigned port number. Click on "Open". You now have an established VNC connection with <code>portal1</code>. Remaining steps are the same as non-Windows 8.

You will then be prompted to enter your password (not the one that was just created for the VNC, but the one that you usually use to connect to jumpgate). After entering your password you will be ready for "Starting a VNC Viewer (Step 3)" in the next section. The reason that you must exit and reopen the command prompt is because you cannot connect to the VNC portal zw if you are already connected to jumpgate or portal1; you must connect to jumpgate and your session zw simultaneously for the VNC portal to work correctly.

=== Starting a VNC viewer (Step 3) ===

Whether you followed the Mac or Linux or Windows instructions above, successful completion will have established a tunnel from your laptop to <code>portal1</code>. The last step is to start a VNC viewer (graphical windowing program) that uses this tunnel to display your <code>portal1</code> session on the screen of your laptop. There are a variety of choices for each platform and they evolve but their operation seems, thankfully, pretty universal.

The most basic input is that they provide a box that says "VNC Server". You will want to type <code>localhost:zw</code> in that box and then apply start (or whatever the particular applications action button is). The VNC Viewer will them prompt you for the password that was generated for this "zw" numbered session (Your version of the generated "abcd1234ExamplePW" from previous steps. Note if you changed that VNC password already, use the one you created).

The <code>zw</code> here will need to be whatever you used for the <code>-L59zw:portal1:59zw</code> part of the tunnel. What the number is actually doing is telling this local VNC viewer on which port you have established a tunnel with the session for which you are starting the VNC viewer. Note it is possible to have several different simultaneous sessions. Each needs its own tunnel (established by repeating the process above with a unique <code>zw</code>) and then a corresponding "new" VNC viewer. In this way we might continue to work in our own session but then take a break to look at a collaborator's session without having to close out the tunnel and viewer pair.

Now that you've connected to the PHASTA machines, it's time to learn a bit mroe about using a UNIX based system (Linux). To continue with the On Ramp click [[UNIX|here]]

=== Web Based Viewer ===

If you can't or don't want to install a VNC viewer you can use a Java based one. You will need a JVM and a Java browser plugin. You will also need the port that the start_vnc script assigned you to be free on your local computer

Forward your session through jumpgate as before before, adding a second port, 580n. For example, if the script tells you to

ssh -L5905:portal1:5902 jumpgate-phasta.colorado.edu you should
ssh -L5902:portal1:5902 -L5802:portal1:5802 jumpgate-phasta.colorado.edu
Then point your browser to http://localhost:5802 and log in with the password specified by the script when prompted. (Replace 2 with the value specified by the script)

== OpenGL ==

<code>portal1</code> is equipped with a VirtualGL install which will allow you to use OpenGL programs (which do not use pthreads)

Simply wrap your OpenGL program with the <code>vglrun</code> command
vlgrun glxgears

Our lab has 2 VirtualGL servers you can connect to from <code>portal1</code>. You must connect to one of them for large memory and/or computationally intensive processes.
The names of the servers are <code>viz002</code> and <code>viz003</code> (<code>viz001</code> is probably never coming back to life).
<code>portal1</code> doesn't have a particularly fast graphics processor and MUST NOT be used for large memory or computationally intensive process)

vglconnect -s viz002
or
vglconnect -s viz003

from this connection you will want to run graphic applications (e.g., SimModeler or ParaView) prefaced by the command <code>vglrun</code>. You can test that you have it setup right
with the toy-app <code>glxgears</code> as follows

vglrun glxgears

Note that VGL uses a number of threads. If you have trouble with <code>vglrun</code> crashing with a message about <code>Thread::Start()</code> make sure you haven't set your stack size too large (remove any <code>ulimit -s</code> or <code>ulimit -n</code> calls from your shell start scripts).

NOTE ALSO: The primary purpose for <code>viz00x</code> is for visualization and for debugging. Production runs should be done elsewhere.

== Troubleshooting ==

If you have used vncserver (It doesn't matter which version) before, you will need to clear your vnc settings for the script to work. You can do this by running rm -rf ~/.vnc

stop_vnc.sh may display some errors; this is normal.

If you have trouble deleting ~/.vnc send an email to Benjamin.A.Matthews@colorado.edu

If any of these commands fail, you may need to source /etc/profile to get the necessary environment variables (this should be fixed soon)

VirtualGL has trouble with some threaded programs. If your OpenGL program exhibits segmentation faults or other issues, this could be the problem. Check back for the solution later.

If the given password is rejected you can run stop_vnc.sh and restart to get a new one. Occasionally the random password generator may generate passwords which VNC doesn't like.

If VirtualGL complains about not being able to get a 24bit FB config either vglconnect to another VirtualGL enabled server or complain to Benjamin.A.Matthews@Colorado.edu

If your VNC connection is very slow, you might want to try changing the compression and encoding options. See your vncviewer's documentation or try this
vncviewer -encodings tight -quality 6 -compresslevel 6
If you have trouble with text distortion try adding
-nojpeg

If you're running OSX and see an error about Zlib, try changing your compression settings (maximum quality usually works) or use a different client. RealVNC and certain versions of ChickenOfTheVNC both exhibit this issue. The latest build of TigerVNC should work reliably, as does the Java based TightVNC client.