PHASTA Wiki - User contributions [en]

Interpolate Solution from Different Mesh

2026-04-02T17:53:35Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and input (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr. NOTE: Even with the same data files, different versions of PV will produce different ordering so '''CHECK'''.

For primitive code turbulent:
awk '{print $8,$9,$10,$2,$4,$5,$6,$3,$1}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $13,$14,$15,$2,$3,$4,$5,$6,$9,$10,$11,$8,$7,$1}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget (which is inside 1-procs_case) and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-11-17T18:07:55Z

Conrad54418: /* Process Overview (manual) */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and input (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr. NOTE: Even with the same data files, different versions of PV will produce different ordering so '''CHECK'''.

For primitive code turbulent:
awk '{print $9,$10,$11,$3,$5,$6,$7,$4,$2}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $13,$14,$15,$2,$3,$4,$5,$6,$9,$10,$11,$8,$7,$1}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget (which is inside 1-procs_case) and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-03-29T16:22:01Z

Conrad54418: /* Turbulent, Compressible, Unstructured Mesh */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr. For primitive code turbulent:
awk '{print $9,$10,$11,$3,$5,$6,$7,$4,$2}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $13,$14,$15,$2,$3,$4,$5,$6,$9,$10,$11,$8,$7,$1}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget (which is inside 1-procs_case) and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-02-20T16:42:35Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr. For primitive code turbulent:
awk '{print $9,$10,$11,$3,$5,$6,$7,$4,$2}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $13,$14,$15,$2,$3,$4,$5,$6,$9,$10,$11,$8,$7,$1}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-02-02T13:56:27Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr. For primitive code turbulent:
awk '{print $9,$10,$11,$3,$5,$6,$7,$4,$2}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-02-02T13:54:24Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w, T, sclr.
awk '{print $9,$10,$11,$3,$5,$6,$7,$4,$2}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

SimModeler

2025-01-29T20:42:32Z

Conrad54418: /* Compressible */

[[Category:Software]]

SimModeler is a model creation program from Simmetrix. It takes the mesh and geometric model and creates the input files for PHASTA.

== Running ==
To run SimModeler, first connect via VNC, then use vglconnect to connect to one of the compute machines:

vglconnect -s viz001

Add the desired version of SimModeler to your environment (the below example will get the "default" version):

soft add +simmodeler

and lunch the GUI:

vglrun simmodeler

== Converting old files ==
This is a guide for converting old files (parasolid and .spj) to the new format (.smd).

After connecting to one of the compute machines, add the suite of tools for SimModeler to your environment:

soft add +simmodsuite

From your case, make a new directory and copy your parasolid (.x_t or .xmt_txt), and .spj file into it. Rename the parasolid file to geom.xmt_txt and the .spj file to geom.spj, if they aren't already named that way. Then from the directory just created (now holds geom.xmt_txt and geom.spj) run:

/users/matthb2/simmodelerconvert/testConvert

Your directory now contains two new files: model.smd and model.x_t

== Creating new files ==

Loading in geometry is about as intuitive as is possibly can be. Go to File -> Import Geometry, browse to the appropriate model, and select Open. Once open, it is possible to both mesh the model and to create boundary conditions for it. Because BLMesher is presently the primary meshing tool, it is only necessary to use SimModeler to create boundary conditions. Go to Analysis -> Select Solver, and select phasta. After selecting phasta, the Analysis Attributes option under Analysis becomes valid. Clicking it brings up the corresponding window. From this new window, it is possible to apply boundary conditions and initial conditions by clicking the small button next to the drop down menu [add picture]. Note you must also double click on "problem definition" which will allow you to name the case. Later post processing expects the name "geom" so always name it so.

== Boundary conditions ==

Commonly boundary conditions include:

*comp3 - Specifies a 3D velocity vector
*comp1 - Specifies a 3D vector in which the velocity is constrained. Velocity normal to this vector is not directly affected. This is useful for creating slip walls and mimicking free stream regions.
*temperature - Sets the temperature of the wall. This is only needed for compressible cases.
*scalar_1 - Sets the scalar_1 / eddy viscosity to apply at a wall. For the Spalart Allmaras models, scalar_1 should be zero at physical walls where a boundary layer develops and 3 to 5 times the molecular viscosity at free stream boundaries (http://turbmodels.larc.nasa.gov/spalart.html)
*surf ID - Associates a number with one or more faces. This can then be read by Phasta and used to apply more complicated boundary conditions in software.
*natural pressure - Apply a mean pressure over a surface. The pressure at any particular point is still allowed to vary (someone verify).
*traction vector - ??. The zero vector is typically applied at outlet.
*heat flux - Specifies the rate at which heat is injected / removed (not sure which one) into / from the fluid domain. The value is almost always set to zero to create a perfectly insulated boundary.
*scalar_1 flux - set the flux of scalar_1 / eddy viscosity into / out of the domain (not sure which one). This is typically only used at outlets where high values of eddy viscosity have been convected downstream of turbulent walls. The value is almost always set to zero.
*turbulence wall - Indicates that a surface is to be included in the calculation of d2wall files (verify) which are then used by the Spalart Allmaras turbulence model to generate more physical turbulent kinetic energy production / dissipation budgets.

=== Incompressible ===

Common BCs used for an incompressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream kinematic viscosity)
*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream kinematic viscosity)
*Outflow
**natural pressure (zero)
**scalar_1 flux (zero)
**traction vector (zero vector)
*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
*Impermeable slip walls
**Comp 1 (zero in wall-normal direction)
**scalar_1 flux (zero)
**traction vector (zero vector)

=== Compressible ===

Common BCs used for a compressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream kinematic viscosity)
**initial pressure
**initial temperature

*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream kinematic viscosity)
**temperature
**pressure

*Outflow
**scalar_1 flux (zero)
**traction vector (zero vector)
**heat flux (zero)

*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
**temperature or heat flux

*Impermeable slip walls
**Comp 1 (zero in wall-normal direction)
**scalar_1 flux (zero)
**traction vector (zero vector)
**heat flux (zero)

SimModeler

2025-01-29T20:33:34Z

Conrad54418: /* Compressible */

SimModeler

2025-01-29T20:33:23Z

Conrad54418: /* Incompressible */

[[Category:Software]]

SimModeler is a model creation program from Simmetrix. It takes the mesh and geometric model and creates the input files for PHASTA.

== Running ==
To run SimModeler, first connect via VNC, then use vglconnect to connect to one of the compute machines:

vglconnect -s viz001

Add the desired version of SimModeler to your environment (the below example will get the "default" version):

soft add +simmodeler

and lunch the GUI:

vglrun simmodeler

== Converting old files ==
This is a guide for converting old files (parasolid and .spj) to the new format (.smd).

After connecting to one of the compute machines, add the suite of tools for SimModeler to your environment:

soft add +simmodsuite

From your case, make a new directory and copy your parasolid (.x_t or .xmt_txt), and .spj file into it. Rename the parasolid file to geom.xmt_txt and the .spj file to geom.spj, if they aren't already named that way. Then from the directory just created (now holds geom.xmt_txt and geom.spj) run:

/users/matthb2/simmodelerconvert/testConvert

Your directory now contains two new files: model.smd and model.x_t

== Creating new files ==

Loading in geometry is about as intuitive as is possibly can be. Go to File -> Import Geometry, browse to the appropriate model, and select Open. Once open, it is possible to both mesh the model and to create boundary conditions for it. Because BLMesher is presently the primary meshing tool, it is only necessary to use SimModeler to create boundary conditions. Go to Analysis -> Select Solver, and select phasta. After selecting phasta, the Analysis Attributes option under Analysis becomes valid. Clicking it brings up the corresponding window. From this new window, it is possible to apply boundary conditions and initial conditions by clicking the small button next to the drop down menu [add picture]. Note you must also double click on "problem definition" which will allow you to name the case. Later post processing expects the name "geom" so always name it so.

== Boundary conditions ==

Commonly boundary conditions include:

*comp3 - Specifies a 3D velocity vector
*comp1 - Specifies a 3D vector in which the velocity is constrained. Velocity normal to this vector is not directly affected. This is useful for creating slip walls and mimicking free stream regions.
*temperature - Sets the temperature of the wall. This is only needed for compressible cases.
*scalar_1 - Sets the scalar_1 / eddy viscosity to apply at a wall. For the Spalart Allmaras models, scalar_1 should be zero at physical walls where a boundary layer develops and 3 to 5 times the molecular viscosity at free stream boundaries (http://turbmodels.larc.nasa.gov/spalart.html)
*surf ID - Associates a number with one or more faces. This can then be read by Phasta and used to apply more complicated boundary conditions in software.
*natural pressure - Apply a mean pressure over a surface. The pressure at any particular point is still allowed to vary (someone verify).
*traction vector - ??. The zero vector is typically applied at outlet.
*heat flux - Specifies the rate at which heat is injected / removed (not sure which one) into / from the fluid domain. The value is almost always set to zero to create a perfectly insulated boundary.
*scalar_1 flux - set the flux of scalar_1 / eddy viscosity into / out of the domain (not sure which one). This is typically only used at outlets where high values of eddy viscosity have been convected downstream of turbulent walls. The value is almost always set to zero.
*turbulence wall - Indicates that a surface is to be included in the calculation of d2wall files (verify) which are then used by the Spalart Allmaras turbulence model to generate more physical turbulent kinetic energy production / dissipation budgets.

=== Incompressible ===

Common BCs used for an incompressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream kinematic viscosity)
*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream kinematic viscosity)
*Outflow
**natural pressure (zero)
**scalar_1 flux (zero)
**traction vector (zero vector)
*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
*Impermeable slip walls
**Comp 1 (zero in wall-normal direction)
**scalar_1 flux (zero)
**traction vector (zero vector)

=== Compressible ===

Common BCs used for a compressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream molecular viscosity)
**initial pressure
**initial temperature

*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream molecular viscosity)
**temperature
**pressure

*Outflow
**scalar_1 flux (zero)
**traction vector (zero vector)
**heat flux (zero)

*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
**temperature or heat flux

Interpolate Solution from Different Mesh

2025-01-15T23:31:07Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$6}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-01-15T23:19:41Z

Conrad54418: /* Turbulent, Compressible, Unstructured Mesh */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
for entropy code laminar:
awk '{print $12,$13,$14,$1,$2,$3,$4,$5,$8,$9,$10,$7,$7}' test.dat > solInterp.1
for entropy code with eddy viscosity:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2025-01-15T23:09:11Z

Conrad54418: /* Process Overview (manual) */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'MergeBlocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source (new mesh file) and target (MergeBlocks) blocks accordingly
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
for entropy code:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

Interpolate Solution from Different Mesh

2024-10-26T16:42:59Z

Conrad54418: /* Alternative to Using MATLAB for Reformatting */

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'merge blocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source and target blocks accordingly (ParaView’s naming is super unclear: they want source to be the coordinates where you need the solution (new mesh) and input to be the mesh that has the solution values associated with it that you want to interpolate from (which in this case is the MergeBlocks). This is confusing because it is exactly backwards to what we use for solution interpolation where we call the mesh with a solution the source and the new mesh the target).
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
for entropy code:
awk '{print $14,$15,$16,$3,$4,$5,$6,$7,$10,$11,$12,$9,$8,$2}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

TCNEQ Version

2024-09-06T17:07:44Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs (see input.config in source code)
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity, [Pa-s]
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp

* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature (related to initial vibrational temperature, see input.config in source code), [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity, [Pa-s]
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8
Turbulence Model: No-Model # RANS-SA

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1 # laminar
# Step Construction : 0 1 0 1 10 11 10 11 # turbulent
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-09-06T17:07:34Z

Conrad54418: /* Initial Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs (see input.config in source code)
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity, [Pa-s]
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature (related to initial vibrational temperature, see input.config in source code), [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity, [Pa-s]
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8
Turbulence Model: No-Model # RANS-SA

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1 # laminar
# Step Construction : 0 1 0 1 10 11 10 11 # turbulent
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-09-06T17:02:52Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs (see input.config in source code)
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity, [Pa-s]
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8
Turbulence Model: No-Model # RANS-SA

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1 # laminar
# Step Construction : 0 1 0 1 10 11 10 11 # turbulent
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:07:33Z

Conrad54418: /* Example of solver.inp file: */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8
Turbulence Model: No-Model # RANS-SA

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1 # laminar
# Step Construction : 0 1 0 1 10 11 10 11 # turbulent
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:06:47Z

Conrad54418: /* Example of solver.inp file: */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8
Turbulence Model: No-Model # RANS-SA

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1 # 10 11 10 11
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:05:27Z

Conrad54418: /* Post-Processing */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''evisc'' and ''dwal'' fields would require turbulence modeling to be turned on for the example below to work. This is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:04:30Z

Conrad54418: /* Example of flow.pht file */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="40">
<GeometryFileNamePattern pattern="ent_from_NAS/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="ent_from_NAS/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="9"
auto_generate_indices="1"
start_index="65800"
increment_index_by="1000"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="10">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_3"
start_index_in_phasta_array="2"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_4"
start_index_in_phasta_array="3"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_5"
start_index_in_phasta_array="4"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="5"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="8"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="evisc"
start_index_in_phasta_array="10"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="dwal"
paraview_field_tag="dwal"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:01:51Z

Conrad54418: /* Simulation Inputs */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species (''nsp'') equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:01:27Z

Conrad54418: /* Example of solver.inp file: */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5.

=== Example of solver.inp file: ===

<nowiki>

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:01:13Z

Conrad54418: Undo revision 2084 by Conrad54418 (talk)

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:00:35Z

Conrad54418: /* Example of solver.inp file: */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5.

=== Example of solver.inp file: ===

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T14:00:21Z

Conrad54418: /* Simulation Inputs */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 1000
Time Step Size: 1e-8

# Limit instructions : switch min max -- change switch from zero to activate
Limit Temperature : 0 0 0 # also limits vibrational temperature
Limit u1 : 0 0 0
Limit u2 : 0 0 0
Limit u3 : 0 -1 1
Limit rho1 : 0 1e-20 0
Limit rho2 : 0 1e-20 0
Limit rho3 : 0 1e-20 0
Limit rho4 : 0 1e-20 0
Limit rho5 : 0 1e-20 0
Limit Scalar 1 : 0 0 0
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 100
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscous Control: Viscous #None
Shear Law: Wilke's Mixing Rule # ishear=1 => matflag(2,n)
Bulk Viscosity Law: Constant Bulk Viscosity # ibulk=0 => matflag(3,n)
Conductivity Law: Wilke's Mixing Rule # icond=1 => matflag(4,n)
#}

#REACTING FLOW
#{
Number of species: 1 # nsp
Species IDs: 1 2 3 4 5 # ispecIDs
# IDs numbered in order: N2=1,O2=2,NO=3,N=4,O=5
Inflow Concentrations: 0.78997 0.21 1e-5 1e-5 1e-5 # concinf
Allow reactions: False # ichem = 1 if True. right now can only be used for nsp=5
Temperature threshold: 2000 # Tth--below which, reactions ignored. 2000 K is from Chalot 1990
Equilibrium Tolerance: 1e-5 # chemtol (max species production rate for reactions to equilibrium in IC's/BC's)
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Vibrational Temperature BC: 1 # TvibBC. must be a positive number. at any BC with T set:
# if greater than 5, then Tvib = TvibBC
# else, then Tvib = TvibBC*T
Vibrational Temperature IC: -1 # TvibIC (if negative, Tvib = T initially. if positive, value here is used)
Restart from primitive file: 0 # 1 if restarting from file from primitive code
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem
#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Shock Sensor : False # i_ss_on = 1 if True
Shock Sensor Value Full Off : 0.05 # % change of temperature across element above which we ramp mu until
Shock Sensor Value Full On : 0.2 # percent change after which we hold at scale factor * mu
Shock Sensor Scale Factor : 100.0 # scale factor on mu or other sensor
Wall Distance to Shield Shock Sensor : -1 # The above won't be applied within this wall distance (-1 ignores this condition)
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:52:25Z

Conrad54418: /* Simulation Inputs */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:51:10Z

Conrad54418: /* Initial Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of turbulent eddy viscosity
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with solver.inp values to compute the mole fraction of each species

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:49:39Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures in conjunction with reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of species 2 mole fraction
* initial scalar_2 - Initial value of species 3 mole fraction
* initial scalar_3 - Initial value of species 4 mole fraction
* initial scalar_4 - Initial value of species 5 mole fraction
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with the initial scalar values to compute the mole fraction of species 1

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:49:25Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of each species of the gas with Dalton's Law of partial pressures using reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of species 2 mole fraction
* initial scalar_2 - Initial value of species 3 mole fraction
* initial scalar_3 - Initial value of species 4 mole fraction
* initial scalar_4 - Initial value of species 5 mole fraction
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with the initial scalar values to compute the mole fraction of species 1

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:49:06Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fractions of species of the gas with Dalton's Law of partial pressures using reference mole fractions specified in solver.inp
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of species 2 mole fraction
* initial scalar_2 - Initial value of species 3 mole fraction
* initial scalar_3 - Initial value of species 4 mole fraction
* initial scalar_4 - Initial value of species 5 mole fraction
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with the initial scalar values to compute the mole fraction of species 1

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:48:08Z

Conrad54418: /* Boundary Conditions */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Turbulent eddy viscosity
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fraction of species 1 of the gas with Dalton's Law of partial pressures and subtracting the summation of the other mole fractions from a value of 1
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of species 2 mole fraction
* initial scalar_2 - Initial value of species 3 mole fraction
* initial scalar_3 - Initial value of species 4 mole fraction
* initial scalar_4 - Initial value of species 5 mole fraction
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with the initial scalar values to compute the mole fraction of species 1

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

TCNEQ Version

2024-08-29T13:46:51Z

Conrad54418: /* Pre-Processing */

== Background ==
The following information relates to the use of the thermochemical nonequilibrium (TCNEQ) version of PHASTA written in terms of entropy variables. The reader is referred to the following for additional information.

* F. Chalot, T.J.R. Hughes, and F. Shakib, '''"Symmetrization of Conservation Laws with Entropy for High-Temperature Hypersonic Computations,"''' Computing Systems in Engineering, 1(2-4):495–521, 1990.

* J. Pointer, '''"Influence of Interpolation Variables and Discontinuity Capturing Operators on Inviscid Hypersonic Flow Simulations Using a Stabilized Continuous Galerkin Solver,"''' Ph.D. dissertation, University of Colorado, Boulder, CO, 2022.

== Pre-Processing ==
In this section, details of the meshing and model attributes are provided. Currently, capability exists to simulate a gas with number of species (''nsp'') ≤ 5.

=== Meshing ===
Within the Simmodeler utility, the mesh can either be created or loaded from an existing .cas file. Below are steps for loading a mesh from a .cas file:
# Launch Simmodeler (for this example, SimModeler7.0-190604 is used)
# File > Import Discrete Data > (select .cas file to import) > (keep defaults and click OK) > (select YES to keep volume mesh)
# Save .sms and .smd files
# Attributes can now be assigned to the model as normal

=== Boundary Conditions ===
Below are the recognized boundary conditions that can be applied for the current version:
* comp1/comp2/comp3 - Specification of one/two/three components of velocity, [m/s]
* temperature - Specification of translational-rotational temperature, [K]. By default, vibrational temperature is held in equilibrium with this value and nonequilibrium is controlled through simulation inputs.
* surfID - When value is set to 702, the boundary is treated as a slip wall. If using this option, include a boundary layer mesh along the surface to ensure the wall normal direction is accurately computed.
* scalar_1 - Mole fraction of species 2 of the gas
* scalar_2 - Mole fraction of species 3 of the gas
* scalar_3 - Mole fraction of species 4 of the gas
* scalar_4 - Mole fraction of species 5 of the gas
* pressure - Specification of static pressure over a surface, [Pa]
** Used to compute mole fraction of species 1 of the gas with Dalton's Law of partial pressures and subtracting the summation of the other mole fractions from a value of 1
* heat flux - set to zero for adiabatic wall boundary condition

=== Initial Conditions ===
Below are the required initial conditions for the current version:
* initial velocity - Components and magnitude of flow velocity, [m/s]
** If a supersonic outlet condition is used, set such that flow is initialized Mach > 1
* initial temperature - Value used to set translational-rotational temperature, [K]
* initial scalar_1 - Initial value of species 2 mole fraction
* initial scalar_2 - Initial value of species 3 mole fraction
* initial scalar_3 - Initial value of species 4 mole fraction
* initial scalar_4 - Initial value of species 5 mole fraction
* initial pressure - Static pressure of the gas, [Pa]
** For multi-species flows, this value is used in combination with the initial scalar values to compute the mole fraction of species 1

== Simulation Inputs ==

Below is an example of the input script for the current version of the code. Capability is included for handling multi-species flows up to number of species ''nsp'' equal to 5. This feature has been tested and shown to work for five species mass conservation equations at one time. However, when chemical species production governed by the finite-rate chemistry module is active, the solution becomes unstable. Additional work on this feature is needed.

=== Example of solver.inp file: ===

<nowiki># PHASTA_HYP Version 1 Input File

#SOLUTION CONTROL
#{
Equation of State: Compressible
Number of Timesteps: 100
Time Step Size: 3.2339656949004e-08

Limit Density: 0 0.01 0.1 # solution limiting on variables [switch, min, max]
Limit u1: 0 0. 2.8e3
Limit u2: 0 0 0
Limit u3: 0 0 0
Limit Temperature: 0 230 3500 # also limits vibrational temperature
#}

#OUTPUT CONTROL
#{
Number of Timesteps between Restarts: 250
Print Error Indicators: True # shock error stored in column 6, DC factor \nu stored in column 10
Error Indicator Threshold: 0.01 # err > thresh*err_max is flagged as 1 (i.e. identified for refinement)
# --> smaller values = narrower flagged region along shock
Number of Error Smoothing Iterations: 0 # ierrsmooth
Load and set 3D IC: False # load the flowfield from a file as the initial condition
Position Tolerance on IC Load: 1e-7 # sets the tolerance for matching node locations while loading the initial condition
#}

#MATERIAL CONTROL
#{
Viscosity: 1.95508431704028e-5 # dynamic viscosity (only used if nsp.eq.99 for air mixture)
Thermal Conductivity: 26.6843390135759e-3 # only used if nsp.eq.99 for air mixture
Viscous Control: None #Viscous #None

#}

#REACTING FLOW
#{
Number of species: 1 # nsp
#.......currently only allowing 1<=nsp<=5.
#.........and make sure scalars passed from simmodeler are in correct slots
#
Species IDs: 1 # specIDs, length of array must equal
# nsp (IDs numbered in order:
# N2,O2,NO,N,O
# ID:99 for Air molecule
# e.g. Species IDs: 1 3 <<< would give N2 and NO
Ref Entropy Conditions: 1e3 230 230 0 #[P0,T0,T0vib,S0]
Ref Entropy Mole Frac: 1 0 0 0 0 # composition of gas used as reference entropy condition
# >> must sum to zero
Allow reactions: False # chemical reactions, ichem = 1 if True
Chemical heat release: False # chemical heat release, iqtot = 1 if True
Limit on reaction step: 0.00001 # rlim (limits change in species cs per step)
Tolerance to global time: 0.01 # ttol, chem solver is advanced in time until time diff < ttol*dt_global
Temperature threshold: 500 # Tth (below which, reactions ignored)
Reaction solver MIN steps: 5 # nstepmin, minimum number of time steps
Reaction solver MAX steps: 100 # nstepmax, maximum number of time steps
Two Temperature coefficient: 0.5 # qta (Tvib**qta*T**(1-qta))
Exclude vib energy: True # ivib0 = 1 if True
Exclude vib source: True # ivibS0 = 1 if True
Tvib BC Ratio: 1.0 # at any BC with T set, Tvib = tvibBC * T
Vibrational Temperature IC: -1 # set negative value to force Tvib = T, otherwise positive value set as IC value
#}

#LINEAR SOLVER
#
Solver Type: GMRES sparse
Number of GMRES Sweeps per Solve: 1 # replaces nGMRES
Minimum Number of Iterations per Nonlinear Iteration: 10 # minIters
Number of Krylov Vectors per GMRES Sweep: 100 # replaces Kspace
Tolerance on Momentum Equations: 0.01 # epstol(1), affects etol for Hessenberg problem

#}

#DISCRETIZATION CONTROL
#{
Weak Form: SUPG # alternate is Galerkin only for compressible
Time Integration Rule: First Order # 1st Order sets rinf(1) -1
Tau Matrix: Matrix-Ent-Adv
Include Viscous Correction in Stabilization: False # if p=1 idiff=1
# if p=2 idiff=2
Tau Time Constant: 1.0
Tau C Scale Factor: 1.0 # taucfct best value depends
Number of Elements Per Block: 64 #ibksiz

#}

#DISCONTINUITY CAPTURING
#{
Discontinuity Capturing: DC-quadratic # Current Options: DC-mallet, DC-minimum, DC-quadratic, DC-yzbeta
Multiplier for DC factor: 1 # scales DC variable in e3DC
Discontinuity Capturing Scheme: 1 # 0: discontinuous, 1: continuous (L2 projection)
Include Source Term in DC: 0 # 1: sets idcSRC to 1
Write DCqpt: 0 # if > 0, writes out data at a quadrature point iDCqpt

#----Parameters for YZBeta DC operator ----
Beta Value: 1 # 1: smoother , 2: sharper, 12: compromise between 1 and 2
YZB Farfield Conditions: 1e5 2119 10 10 300 # [Pressure, X-Vel, Y-Vel, Z-Vel, Temperature]
YZB Farfield Mole Frac: 1 0 0 0 0 # mole fractions at reference condition
# [xN2,xO2,xNO,xN,xO]
# must sum to 1, must be length 5
Include Umod Term: 1 # 0: no, 1: yes
Mach Adjustment Bm Value: 1 # 0: off,1: smoother shock, 2: sharper shock
Mach Adjustment Bj Value: 6 # 0: off,1: smoother shock, 2: sharper shock
Include Time Term in Z: 1 # 0: no, 1: yes
#------------------------------------------
#}

#STEP SEQUENCE
#{
Step Construction : 0 1 0 1
#}</nowiki>

== Post-Processing ==

An example of the ''flow.pht'' file is provided to demonstrate the ordering of the variables that can be viewed in Paraview. Note that both the ''errors'' and ''DCqpt'' fields would need to be output to the restart file for the example below to work. Saving these two fields is controlled through the solver.inp options.

=== Example of flow.pht file ===

<nowiki><?xml version="1.0" ?>
<PhastaMetaFile number_of_pieces="24">
<GeometryFileNamePattern pattern="24-procs_case/geombc.dat.%d"
has_piece_entry="1"
has_time_entry="0"/>
<FieldFileNamePattern pattern="24-procs_case/restart.%d.%d"
has_piece_entry="1"
has_time_entry="1"/>
<TimeSteps number_of_steps="1"
auto_generate_indices="1"
start_index="2010"
increment_index_by="50"
start_value="0"
increment_value_by="1">
</TimeSteps>
<Fields number_of_fields="9">
<Field phasta_field_tag="solution"
paraview_field_tag="rho_N2"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="rho_O2"
start_index_in_phasta_array="1"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="velocity"
start_index_in_phasta_array="2"
number_of_components="3"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temp_vib"
start_index_in_phasta_array="5"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="solution"
paraview_field_tag="temperature"
start_index_in_phasta_array="6"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="errors"
paraview_field_tag="nu"
start_index_in_phasta_array="9"
number_of_components="1"
data_dependency="0"
data_type="double"/>
<Field phasta_field_tag="DCqpt"
paraview_field_tag="DCqpt1"
start_index_in_phasta_array="0"
number_of_components="1"
data_dependency="1"
data_type="double"/>
</Fields>
</PhastaMetaFile></nowiki>

PHASTA Group Machines

2024-08-21T19:36:29Z

Conrad54418: /* Machines */

This page documents the local machines owned by the group, logging in, and two factor authentication.

== Logging In ==

The entry point for the group machines is <code>jumpgate</code>, which is accessed publicly via <code>jumpgate-phasta.colorado.edu</code>. We access this entry point by creating an SSH connection between it and our personal machines. To access the system via command line (terminal, command prompt, etc), simply run <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>. Note: if you are using Windows 8 or older, you will need to download and install an SSH client such as [https://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY], and follow the directions specified as '''Windows 8'''.

This [https://fluid.colorado.edu/tutorials/tutorialVideos/VNC_tutorial.mov video] walks you through the multiple next steps you will take in order to access and interact with the PHASTA machines. wThe steps in the video are more thoroughly documented in the remainder of this page and the [[VNC]] page of the wiki.

To get started, open a command line terminal, enter <code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>, and the login process will look like the following:

➜ ssh USERNAME@jumpgate-phasta.colorado.edu
Password:
Verification code:

'''**Windows 8**'''
Open your PuTTY application and enter the following in the field that says "Host name:"
<code>USERNAME@jumpgate-phasta.colorado.edu</code>
Next make sure that Connection type is set to "SSH". Click on "Open". This will start a terminal window session connected to jumpgate that will have the
same output as the non-windows 8 instructions above.

where the <code>Password:</code> and <code>Verification code:</code> are prompts for you to enter in your password and 2FA pass code. Note the 2FA request for a verification code will not start occurring until after you setup 2fa as noted below.

Very little can/should be done on <code>jumpgate</code>. The most common use is to establish a tunnel for a VNC session. The second usage that must be done to set that up is connecting to <code>portal1</code>. This is done via <code>ssh portal1</code> while on <code>jumpgate</code>.

=== Setting Up Two-Factor Authentication ===
Due to recent brute force ssh attacks we are moving to using two factor authentication (2FA). Existing users will have one week to switch over to this process. New users are expected to do this within 24 hours. This is pretty easy to setup as follows (from a terminal in your mac or linux laptop (and Windows if new enough)) or using PuTTY. All commands to be run are will be in <code>code</code> block. If you have not already established a connection with jumpgate (from the previous steps above), open a new terminal on your personal computer and enter the following:

<code>ssh USERNAME@jumpgate-phasta.colorado.edu</code>

This will prompt for your ''password'' (the private password you set or, if this is your first login, the one in the account creation email). Enter your password to log in. We will refer to this terminal as "Primary terminal" for the remainder of these instructions.

Next you need to download and install an authenticator application either for your computer or phone. There are several from Google, Microsoft, Twilio, etc (Google Authenticator works great). Launch that application on your phone or computer. In whatever mode it uses to create a new token generator, do that (often it opens with a QR code scanner enabled as it knows that is the easiest way to link the phone application to the QR scan created on the machine you are trying to access).

Before moving forward, start a second terminal connection to <code>jumpgate</code> by repeating the process above for establishing an ssh connection (PuTTY steps for Windows 8 users). We will refer to this second terminal connection as our "backup terminal". If at any point you want/need to reset, simply run <code>''rm -rf ~/.google_authenticator''</code> in the backup terminal.

Now, in your primary <code>jumpgate</code> terminal on your laptop type and enter:

<code>google-authenticator</code>

If your terminal window is big enough, it should display a QR code that you can scan with your authenticator app on your phone. At this point it will ask you some questions about options (I answered yes to all). **You '''MUST''' answer yes to the time-based question, otherwise you will not be able to copy files onto ALCF (learned from experience)**

Now open a 3rd terminal and log on to <code>jumpgate</code> with ssh just as we did before. Because we have created a 2FA token, in addition to prompting for your ''password'', it will also prompt for a "Verification code:". In your authenticator app, find the auto-generated 6 digit code associated to the jumpgate machine and enter it in the "Verification code:" field. If you've logged on successfully, then you are done and can move on to setting up your VNC. Otherwise, attempt to troubleshoot or reset the process with the <code>rm -rf ~/.google_authenticator</code> command in the "backup" terminal you opened previously. If you have reset the process, close your primary and 3rd terminals, keep the backup terminal open, and open a new primary terminal and start the process over.

=== VNC - Viewing your PHASTA Machine Sessions ===
Most members of the group interact with the PHASTA machines via a VNC tool (Virtual Network Computing), which provides a graphical user interface (GUI) link between the PHASTA machines (server side) and your personal machine (client side). This connection permits you to interact with the PHASTA machines by using your personal machine as from any location with a secure internet connection. Setting up the VNC server is documented on the [[VNC]] page, and for the purpose of the On Ramp, follow the sections identified with steps 1 through 3. The remainder of the sections on the VNC page are options you can explore at a later time. Click [[VNC|here]] to set up your VNC and continue with the On Ramp.

== Machines ==

=== <code>jumpgate</code> ===
This is the machine that allows you to "jump" to the other machines in the local network via <code>ssh</code>. It is simply the public-facing machine and should only be used as such.

=== <code>portal1</code> ===
This is where most of the non-computationally intensive tasks are done, such as text editing, moving files, etc. Effectively, if it takes longer than 5 seconds to run, you should probably think about running it on one of the <code>viz*</code> nodes.

=== <code>viz003</code> ===
This is where most computationally intensive tasks are done. However, they should only be run for debugging or post-processing. Production runs should be run on servers outside of the group's local machines ([[CU Boulder RC|Summit]], [[NAS]], [[ALCF]], etc.)

=== <code>lab3</code> ===
Windows machine for using Windows programs, such as [[SolidWorks]]. Accessing <code>lab3</code> is different than with the other machines, see [[Access Lab3]]

=== <code>ciscoX</code> ===
These machines (<code>cisco1</code>, <code>cisco2</code>, etc.) are meant for computing and are accessed by submitting a job via PBS.

== Using PBS for the cisco machines ==

Jobs submitted to PBS can either be scripted or interactive.

=== Interactive Job Example ===

<nowiki>
matthb2@portal1:~$ soft add +pbs
matthb2@portal1:~$ qsub -I -l select=2:ncpus=24:mpiprocs=4 -q workq
qsub: waiting for job 1008.pbs to start
qsub: job 1008.pbs ready

matthb2@cisco1:~$ module load openmpi
matthb2@cisco1:~$ mpicc mpihello/mpihello.c
matthb2@cisco1:~$ mpirun ./a.out
Hello Parallel World
Rank: 1 Number is: 1
Rank: 2 Number is: 1
Rank: 3 Number is: 1
Rank: 0 Number is: 1
Rank: 5 Number is: 1
Rank: 4 Number is: 1
Rank: 6 Number is: 1
Rank: 7 Number is: 1
matthb2@cisco1:~$
</nowiki>

[[Category:Compute Facilities]]

Interpolate Solution from Different Mesh

2024-07-01T20:25:13Z

Conrad54418:

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/connor_primitive</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'merge blocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source and target blocks accordingly (ParaView’s naming is super unclear: they want source to be the coordinates where you need the solution (new mesh) and input to be the mesh that has the solution values associated with it that you want to interpolate from (which in this case is the MergeBlocks). This is confusing because it is exactly backwards to what we use for solution interpolation where we call the mesh with a solution the source and the new mesh the target).
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

NAS

2024-05-02T18:39:05Z

Conrad54418: /* Running TotalView on NAS */

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older TotalView interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using TotalView. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by executing this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the TotalView module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with TotalView, execute this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

A TotalView feature that is useful while running on NAS is "rescan libraries" which can be found in the "File" drop-down menu. You can select this option after recompiling the PHASTA code which will allow you to restart the job in TotalView without having to close and reopen the application, saving time.

With that, you should be all set! For more information about using TotalView on NAS you can visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:38:41Z

Conrad54418: /* Running TotalView on NAS */

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older TotalView interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using TotalView. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by executing this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the TotalView module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with TotalView, execute this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

One useful TotalView feature to use while running on NAS is "rescan libraries" which can be found in the "File" drop-down menu. You can select this option after recompiling the PHASTA code which will allow you to restart the job in TotalView without having to close and reopen the application, saving time.

With that, you should be all set! For more information about using TotalView on NAS you can visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:35:56Z

Conrad54418: /* Running TotalView on NAS */

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older TotalView interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using TotalView. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by executing this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the TotalView module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with TotalView, execute this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

With that, you should be all set! For more information about using TotalView on NAS you can visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:30:37Z

Conrad54418: /* Running TotalView on NAS */

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older TotalView interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using TotalView. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by executing this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the TotalView module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with TotalView, execute this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

With that, you should be all set! For more information about using totalview on NAS you can visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:29:18Z

Conrad54418: /* Running TotalView on NAS */

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older TotalView interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using TotalView. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by running this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the TotalView module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with TotalView, run this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

With that, you should be all set! For more information about using totalview on NAS you can visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:25:38Z

Conrad54418:

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older totalview interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using totalview. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by running this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the totalview module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with totalview, run this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

With that, you should be all set! For more information about using totalview on NAS you can watch the tutorial video I wrote this page based on here or visit this website: https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

NAS

2024-05-02T18:24:58Z

Conrad54418:

[[ Category: Compute Facilities]]
Wiki for information related to the '''NASA Advanced Supercomputing''' ('''NAS''') facility.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Pleiades
| Compute
|-
|
| Lou
| Storage and Analysis
|-
|
| Electra
| Compute
|-
|
| Endeavour
| Compute
|-
|
| Merope
| Compute
|-
|
| Aitken
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://www.nas.nasa.gov/hecc/support/kb/portable-batch-system-(pbs)-overview_126.html PBS]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.nas.nasa.gov/hecc/support/kb/ Support Knowledgebase]
|
|}

== How-To's ==

=== How-To's in Separate Wiki's ===

* [[Setting_Default_File_Permissions#NAS|Setting Default File Permissions]]

=== Backup Data from Scratch Directories===

This is done simply by copying data from the <code>/nobackup/$USER</code> directories to your home directory on Lou (<code>lfe</code>). The <code>/nobackup/$USER</code> directories are mounted onto <code>lfe</code>, so transfers should be done on <code>lfe</code>.

It is recommended to mirror the directory structure of your <code>/nobackup/$USER</code> directory on <code>lfe</code> to allow for the data to be easily recovered back to it's original state. This is especially important if you use symlinks (as they are path dependent and will break if either the source file or the symlink itself are not in the correct location).

This can be done with <code>scp</code>, but it is recommended to use NASA's in-house utility <code>shiftc</code>. <code>shiftc</code> will automatically perform parallel file transfers, data integrity checks and repairs, and syncing features similar to <code>rsync</code>.

'''Commands:'''

jrwrigh7@lfe7: shiftc -r -d --sync /nobackup/jrwrigh7/models/STGFlatPlate/STFM_Tet_dz4-10_dx15 .

This will copy the directory <code>STFM_Tet_dz4-10_dx15</code> to the current location (<code>.</code>). The flags do as follows

* <code>-r</code>: Recursively copy files from destination
* <code>-d</code>: Create required directories that don't already exist. Equivalent of the <code>-p</code> flag for <code>mkdir</code>
* <code>--sync</code>: Only copy over "new" files, where "new" are any changes to the modification time or file size.
** If a file exists on destination (<code>.</code>), but not source (<code>STFM_Tet_dz4-10_dx15</code>), it will not be copied back to source nor will it be deleted to match the state of source.

Once this command is submitted, the transfer process will be backgrounded. Progress can be viewed by running <code>shiftc --monitor</code>. Additionally, you will recieve an email with the transfer job is completed.

jrwrigh7@lfe7: shiftc --stop --id [shiftc job ID]

This will stop the given shiftc job. The <code>[shiftc job ID]</code> is the same number that appears beside the output of <code>shiftc --monitor</code>.

More documentation for <code>shiftc</code> can be found in its man page (<code>man shiftc</code>) and on [https://www.nas.nasa.gov/hecc/support/kb/shift-transfer-tool-overview_300.html NAS's documentation website].

=== Control MPI Rank Placement ===
==== Rank 1 Solo Node ====
To make the rank 1 MPI process take a node on it's own, put this in the PBS directives:

#PBS -l select=1:mpiprocs=1:model=sky_ele+1:mpiprocs=40:model=sky_ele

This will request 2 nodes: One will have the rank 1 process all by itself, and the other will have 40 MPI Processes (for all 40 CPU cores available on <code>sky_ele</code> nodes).

====Distribute Non-First Rank MPI Processes====
For controlling the placement of non-first rank MPI processes, use the <code>mbind.x</code> utility.

For example, if we have requested 4 nodes and want 10 MPI processes per node, the <code>mpiexec</code> command needs to be modified to the following:

mpiexec -np 40 /u/scicon/tools/bin/mbind.x -n10 [executable]

Note that <code>mbind.x</code> is also socket aware, so it will distribute nodes evenly between nodes ''and'' between CPU's in each node (NAS nodes have 2 CPU's per node).
For more information on <code>mbind.x</code>, see it's help flag (<code>mbind.x -help</code>) or [https://www.nas.nasa.gov/hecc/support/kb/using-the-mbind-tool-for-pinning_288.html NAS's documentation website].

=== Common commands ===

* <code>node_stats.sh</code>: Displays how many nodes are available or actively running jobs
* <code>tracejobssh</code>: Helps to answer "Why isn't my job running?". Part of the [https://github.com/PHASTA/utilities git repo].

=== See Priority "Score" in Queue ===

To see what your priority "score" in PBS is use the <code>qstat -W o=+pri</code> to add the "Priority" column to the output of <code>qstat</code>.

==== Priority Scoring (as of 2021-01-22) ====

* Job priority score grows by 1 every 12 hours
* We are capped at a max score of 20 per job
** Note that other users/groups using NAS may start with higher priority and grow higher than 20
** Result is that it's quite difficult to get large jobs running
* If you don't have any jobs running, you get an addition +10 to the score
** This score bump is removed as soon as you have a running job

=== Compiling ===
* Generally will want use <code>module load hpe-mpi/mpt comp-intel</code> for compiling
* Sometimes, <code>mpi{cc,cxx,f90}</code> will not pick the Intel compilers by default. You can check this by running <code>mpi{cc,cxx,f90} --version</code> to verify the compiler it links to.
** To fix this, you can set <code>export MPICC_CC=icc MPICXX_CXX=icpc MPIF90_F90=ifort</code> to force it to use the Intel compilers

=== Running TotalView on NAS ===

Using TotalView on NAS starts with adding a configuration file which will enable port forwarding on the viz nodes. On the viz nodes, run these commands:

cd ~/.ssh/
cp ~kjansen/sshconfig config

Now, login to NAS sfe with X forwarding:

ssh -X [nas username here]@sfe6.nas.nasa.gov

You'll need to enter your password and NAS secure passcode. Next transfer to NAS pfe, also with X forwarding:

ssh -X pfe

You'll need to again enter a NAS secure passcode. The next step is optional. If you want to use the older totalview interface that looks like the one on the viz nodes, run this command:

echo false > ~/.totalview/.tvnewui

Now start an interactive job on NAS pfe with the following command:

qsub -X -I -q devel -lselect=1:ncpus=8:model=sky_ele,walltime=2:00:00

The number of cpus (8) should match the number of processes for the case you'll be debugging. Now on the interactive session on NAS pfe, load the correct environment that matches the build of your code. In my case it was:

module load mpi-hpe/mpt
module load comp-intel/2020.4.304

It is important to note that the build of PHASTA on NAS that you want to debug must be built as the debug version. This will allow you to place breakpoints in the code using totalview. PHASTA can be built as the debug version by setting the "-DCMAKE_BUILD_TYPE=Debug \" flag in the cmake file. Now, navigate to the directory in nobackup where you have your case setup to run. Define the path to the PHASTA build by running this command:

export PHASTA_CONFIG=[path to PHASTA build]

Now load the totalview module:

module load totalview/2023.4.16

You may need to manually remove the "doubleRun-check" folder from the "procs_case" folder before running PHASTA. Now, to run PHASTA and debug with totalview, run this command:

totalview mpiexec_mpt.real -a -np 8 $PHASTA_CONFIG/bin/phastaC.exe

With that, you should be all set! For more information about using totalview on NAS you can watch the tutorial video I wrote this page based on here or visit this website:

https://www.nas.nasa.gov/hecc/support/kb/totalview_95.html

ALCF

2024-04-05T16:56:43Z

Conrad54418:

Wiki page for information related to the '''Argonne Leadership Computing Facility''' ('''ALCF''') located at Argonne National Labs.

==Overview==
{| class="wikitable" style="text-align:center;"
! Key
! Value
! Notes
|-
| style="font-weight:bold; text-align:left;" | Machines
| Cooley
| Visualization
|-
| style="text-align:left;" |
| Theta
| Compute
|-
| style="font-weight:bold; text-align:left;" | Job Submission System
| [https://xgitlab.cels.anl.gov/aig-public/cobalt Cobalt]
|
|-
| style="font-weight:bold; text-align:left;" | Facility Documentation
| [https://www.alcf.anl.gov/support-center ALCF Support Center]
|
|}

The availability of the Cooley nodes can be found [https://status.alcf.anl.gov/cooley/activity here]. The first sixteen nodes are available for checkout by PHASTA users.

The availability of the Theta nodes can be found [https://status.alcf.anl.gov/theta/activity here].

== How-To's ==

=== Login ===
From the Viz nodes, connect to either Theta or Cooley at ALCF with the command(s):

ssh <username>@theta.alcf.anl.gov

ssh <username>@cooley.alcf.anl.gov

It asks for a password; this is the 8-digit code in the MobilePASS+ app on your phone (instructions on how to set that up are given by ALCF when you create an account).

=== Start Interactive Session on Cooley ===

To start an interactive session, run this script:

/projects/cfdml_aesp/Viz/cooley/Viz-SyncIO/subIntNodesTime.sh 1 120

This will reserve you one node for 120 minutes (this is the maximum time allowed by ALCF). You may need to edit the script to include the correct filesystem you are working with. This script includes "home", "grand", and "theta-fs0".

=== Copy Files From Viz Nodes To Cooley===

scp -r [your username here]@jumpgate-phasta.colorado.edu:[insert file(s)/folder here] .

It will ask you for your password and verification; this is the same password used to login to jumpgate, not the 8-digit code from MobilePASS+.

=== Copy Files From Cooley To Viz Nodes ===

scp -r [insert file(s)/folder here] [your username here]@jumpgate-phasta.colorado.edu:[insert destination folder here]

It will ask you for your password and verification; this is the same password used to login to jumpgate, not the 8-digit code from MobilePASS+.

=== Archiving Data to ALCF's Tape Drives ===
See [[ALCF/Archiving_Data_at_ALCF|Archiving Data at ALCF ]].

== Subpages ==
{{Special:PrefixIndex/ALCF/}}

[[Category:Compute Facilities]]

Interpolate Solution from Different Mesh

2024-01-21T21:32:59Z

Conrad54418:

This will be a general page for how to interpolate solutions from different meshes onto a new mesh.
Those meshes are assumed to be of the same domain.

The generic terms for the two meshes are "source" and "target", where source has the desired solution data and target is the mesh that will be receiving.

== Laminar, Incompressible, Semi-structured Mesh ==
This section will assume that the source mesh is in a structured ijk form. In the future, this may be expanded to meshes created by MGEN.

=== Process Overview ===

# A CSV of the solution must be created. This must be done on a serial running version of ParaView '''and''' must be done after a MergeBlocks filter is done.
# A special solution file is created via <code>Sort2StructuredGrid</code>, which will take in the csv from step 1 and the ordered coordinates of the source file from Matlab.
#* Effectively, it creates a single solution file in the same ordering as the Matlab points. The importance of this is significant as the executable used in the next step depends on some expected ordering. If the ordering is different, a new executable will need to be used/created.
# The interpolation is performed onto the new grid via <code>par3DInterp3</code>, which creates <code>solInterp.<1-nparts></code> files in the <code>solnTarget</code> directory.
# The interpolation is then used by PHASTA by setting <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>.
#*This should only be done for 1 timestep, as it will continue to reset the IC for all proceeding timesteps. The <code>solnTarget</code> directory needs to be symlinked into the <code>-procs_case</code> directory for this to work.

==== 1. Create CSV ====

* Created using ParaView
* PV must be running in Serial mode
** Otherwise the CSV will not be in the correct order and possibly have duplicated points
# Load in source dataset
# Apply Mergeblocks filter
# Save dataset as a csv with 12 digits of precision in scientific notation
#* Make sure the csv is in "pressure, u0, u1, u2, x0, x1, x2" format
#* This can be done by only loading the pressure and velocity fields into Paraview (either by editing the <code>.phts</code> or in the data load menu in Paraview).
# Replace the commas with spaces
#* Can use <code>vim</code> or run <code>sed -i 's/,/\ /g' test.csv</code>
#* Though the next step looks for a .csv extension, it is a fortran formatted read and actually needs those commas replaced by spaces
# Remove the first line of the csv file
#* Done in vi or sed (<code>sed -i 1,1d test.csv</code>) or tail (<code>tail -n +2 test.csv > trimmedLine1.csv</code>)
#* Needed for the next program
#* Better yet, we should change the next code to read past that header line and then delete this line when that is complete. We should also consider the solution in [https://stackoverflow.com/a/46451049/7564988 this StackOverflow answer] as it shows how to make a data structure that could read the csv lines directly in the next program and avoid ALL this file manipulation with modern fortran (see HighPerformanceMark's answer).

==== 2. Create Structured Solution File ====

'''Note:''' These instructions will be for the <code>parallelSortDNSzBinJames</code> executable, which has some highly specific requirements and command inputs.

This step will take the data from the source solution file and put it in an format/order that will make the interpolation process work much faster.

# Symlink the source mesh's ordered coordinate file as <code>ordered.crd</code>
#* This may come from the files used to create the mesh (ie. for [[Tutorial_Video_Overviews#MatLabMeshAndConvert.mov|matchedNodeElementReader]])
#* ''(Untested)''This may also be created using the coordinates from the solution file
# Rename/symlink csv to be the correct file name (in my specific case, it was <code>dnsSolution1procLongFort.csv</code>)
# Create an interactive job on whatever machine you're needing to run on (ALCF Cooley in this case)
# Load approriate MPI environment variables (<code>soft add +mvapich2</code> for Cooley)
# Run the executable via <code>mpirun -np [nprocs] [executable path] [executable inputs]</code>
#* This will produce <code>source.sln.{1..nprocs}</code> files
# Concatenate <code>source.sln.{1..nprocs}</code> files '''in order''' into single <code>source.sln</code> file
#* The individual <code>source.sln.{1..nprocs}</code> files need to be concatenated into a single <code>solution.sln</code> file, which can be done (in zsh at least) via <code>echo source.sln.{1..[MPIRanks]}| xargs cat > source.sln</code> (or probably equivalent <code>cat source.sln.{1..[MPIRanks]} > source.sln</code>). Note these files '''must''' be concatenated in order of rank, otherwise it will be out of sequence.

'''Example Command:''' <code>mpirun -np 24 /lus/theta-fs0/projects/PHASTA_aesp/Utilities/Sort2StructuredGrid/parallelSortDNSzBinJames 47822547 47822547 212 0.0291</code>

* The inputs for this command are <code>[nlines of csv] [nlines of ordered.crd] [number of element in z] [z domain width]</code>
* '''Note:''' These inputs are specfic to this executable. Changing executable will change which is used
* Also note that the <code>[number of elements in z]</code> is equivalent to <code>nsons</code> - 1 or the number of nodes in z - 1.

==== 3. Create Interpolated files ====

This step will create the <code>solInterp.[target nprocs]</code> files used by PHASTA to perform the interpolation.

# Create <code>Interpolate.../[target nprocs]-procs_case</code> directory in the target's Chef directory and move to that directory
# Symlink the target's POSIX <code>geombc.[target nprocs]</code> files (that were created by Chef) to the <code>[target nprocs]-procs_case</code> directory
#* The <code>geombc.[target nprocs]</code> files should be copied in the exact fashion that they are in the Chef created <code>[target nprocs]-procs_case</code> directory, including if they're "fanned out"
# Create a directory called <code>solnTarget</code>
#* This may be corrected in the future, but currently if <code>solnTarget</code> is not present the job will fail
# Symlink the <code>source.sln</code> to the directory and the <code>ordered.crd</code> file as <code>source.crd</code>
# Run <code>phInterp</code> via mpirun on an interactive job.
# This creates a series of <code>solInterp.[target nprocs]</code> files in the <code>solnTarget</code> directory

The file format for <code>solInterp.N</code> is quite simple. Each line corresponds to the node number in the partition and the file itself has 7 columns:

coord_x coord_y coord_z pressure velocity_x velocity_y velocity_z

'''Example Command:''' <code>mpirun -np 64 /path/to/phInterp 16 799 281 213 0.452</code>

* The inputs for this command are <code>[target parts per MPIProc] [source nx] [source ny] [source nz] [z Length]</code>
* Note that the number of processes given to <code>mpirun</code> times the <code>[target parts per MPIProc]</code> must be equal to the number of target partitions. In this case, the target partition was 1024, so 64*16 = 1024
* The way it works is that each of the 64 MPIProcs is given 16 partitions to interpolate.

==== 4. Interpolate the solution in PHASTA ====

This step will take the <code>solInterp.[target nprocs]</code> files and load them as initial conditions.

# Symlink the <code>solnTarget</code> directory into the <code>[target nprocs]-procs_case</code> directory.
# Add/uncomment <code>Load and set 3D IC: True</code> in the <code>solver.inp</code>
# Run PHASTA for a few timesteps and write out <code>restart-dat.[target nprocs]</code> files
# Remove/comment out the <code>Load and set 3D IC: True</code> line from the <code>solver.inp</code>
# The new restart files have the interpolated solution
#* Note that if you forget to remove the <code>Load and set 3D IC: True</code> statement from <code>solver.inp</code>, PHASTA will overwrite the existing solution in the restart files

== Turbulent, Compressible, Unstructured Mesh ==

Currently, the only version of PHASTA that is set up to handle this type of solution transfer is the <code>conrad54418/ViscousHypersonics</code> branch (as of 6/22/22). Creating the <code>solInterp.1</code> file can be automated via a script or done manually (see below).

=== Process Overview (scripted) ===

Scripts have been made for use with the compressible, but not turbulent, version of the code. An example folder with the scripts included can be found at <code>/project/tutorials/ParaviewSolutionTransfer</code>. The three scripts needed are called interpolateSol.py, pvCSV2customSLN_Nproc_prim.m, and parRunAll.sh. The only script of those three you need to run is parRunAll.sh, which can be found in <code>targetFolder/solutionInterp</code>. You then need to run PHASTA via the runPhasta.sh script, which will produce a restart.1.1 file that contains the transferred solution on the new mesh. The manual section below provides insight about what the scripts are doing.

=== Process Overview (manual) ===

# Load existing solution into ParaView. Use the 'merge blocks' filter to convert to serial case.
# Load the target case .pht file into ParaView
# Use the 'Resample from dataset' filter and select the source and target blocks accordingly (ParaView’s naming is super unclear: they want source to be the coordinates where you need the solution (new mesh) and input to be the mesh that has the solution values associated with it that you want to interpolate from (which in this case is the MergeBlocks). This is confusing because it is exactly backwards to what we use for solution interpolation where we call the mesh with a solution the source and the new mesh the target).
# Save the output as a .csv file. Write a single time step and select scientific notation to 12 decimals. NOTE: sometimes ParaView will make an error and write zeros where it can't quite find the closest point when doing the solution transfer. In this case, the .csv file needs to be manually edited to replace the zeros for pressure and temperature with realistic values.
# The .csv file can now be reformatted and renamed with MATLAB to match the expected form of solInterp.1.
# Advance PHASTA one step in serial, then convert to desired number of processors using Chef.

==== Alternative to Using MATLAB for Reformatting ====

For those wanting to skip the MATLAB step, <code>awk</code> can also do the necessary column manipulation.

Copy your file (in case you goof up):
cp PVinterp0.csv test.dat
Remove the header line:
sed -i 1,1d test.dat
Replace the commas with spaces:
sed -i 's/,/\ /g' test.dat
Rearrange the columns to be what solInterp.1 wants. It is probably a good idea to do a <code>head -1 PVinterp0.csv</code> to be sure as you might have more or less fields than I did but use that header to find the column numbers (starting from 1, not 0) to write x, y, z, p, u, v, w.
awk '{print $6,$7,$8,$1,$2,$3,$4}' test.dat > solInterp.1
and don’t forget to put this into a directory called solnTarget and also to turn on the flag in solver.inp <code>Load and set 3D IC: True</code> for the 1 step that Joe mentioned. Finally, if you are worried about that one step messing up your solution recent versions of the code can take
iexec : 0
or
Number of Timesteps: 0
to not take any actual steps. Note however that only very recent version of the code have the iexec conditional moved AFTER the loading of the interpolated solution but it should be pretty easy to figure out where to move that conditional. Alternatively, the second option also skips over the time stepping and writes the solution AFTER applying the boundary conditions which can be useful to see as well to confirm you have the intended BC’s set (iexec :0 won’t detect this)

SimModeler

2023-08-14T16:21:55Z

Conrad54418:

[[Category:Software]]

SimModeler is a model creation program from Simmetrix. It takes the mesh and geometric model and creates the input files for PHASTA.

== Running ==
To run SimModeler, first connect via VNC, then use vglconnect to connect to one of the compute machines:

vglconnect -s viz001

Add the desired version of SimModeler to your environment (the below example will get the "default" version):

soft add +simmodeler

and lunch the GUI:

vglrun simmodeler

== Converting old files ==
This is a guide for converting old files (parasolid and .spj) to the new format (.smd).

After connecting to one of the compute machines, add the suite of tools for SimModeler to your environment:

soft add +simmodsuite

From your case, make a new directory and copy your parasolid (.x_t or .xmt_txt), and .spj file into it. Rename the parasolid file to geom.xmt_txt and the .spj file to geom.spj, if they aren't already named that way. Then from the directory just created (now holds geom.xmt_txt and geom.spj) run:

/users/matthb2/simmodelerconvert/testConvert

Your directory now contains two new files: model.smd and model.x_t

== Creating new files ==

Loading in geometry is about as intuitive as is possibly can be. Go to File -> Import Geometry, browse to the appropriate model, and select Open. Once open, it is possible to both mesh the model and to create boundary conditions for it. Because BLMesher is presently the primary meshing tool, it is only necessary to use SimModeler to create boundary conditions. Go to Analysis -> Select Solver, and select phasta. After selecting phasta, the Analysis Attributes option under Analysis becomes valid. Clicking it brings up the corresponding window. From this new window, it is possible to apply boundary conditions and initial conditions by clicking the small button next to the drop down menu [add picture]. Note you must also double click on "problem definition" which will allow you to name the case. Later post processing expects the name "geom" so always name it so.

== Boundary conditions ==

Commonly boundary conditions include:

*comp3 - Specifies a 3D velocity vector
*comp1 - Specifies a 3D vector in which the velocity is constrained. Velocity normal to this vector is not directly affected. This is useful for creating slip walls and mimicking free stream regions.
*temperature - Sets the temperature of the wall. This is only needed for compressible cases.
*scalar_1 - Sets the scalar_1 / eddy viscosity to apply at a wall. For the Spalart Allmaras models, scalar_1 should be zero at physical walls where a boundary layer develops and 3 to 5 times the molecular viscosity at free stream boundaries (http://turbmodels.larc.nasa.gov/spalart.html)
*surf ID - Associates a number with one or more faces. This can then be read by Phasta and used to apply more complicated boundary conditions in software.
*natural pressure - Apply a mean pressure over a surface. The pressure at any particular point is still allowed to vary (someone verify).
*traction vector - ??. The zero vector is typically applied at outlet.
*heat flux - Specifies the rate at which heat is injected / removed (not sure which one) into / from the fluid domain. The value is almost always set to zero to create a perfectly insulated boundary.
*scalar_1 flux - set the flux of scalar_1 / eddy viscosity into / out of the domain (not sure which one). This is typically only used at outlets where high values of eddy viscosity have been convected downstream of turbulent walls. The value is almost always set to zero.
*turbulence wall - Indicates that a surface is to be included in the calculation of d2wall files (verify) which are then used by the Spalart Allmaras turbulence model to generate more physical turbulent kinetic energy production / dissipation budgets.

=== Incompressible ===

Common BCs used for an incompressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream molecular viscosity)
*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream molecular viscosity)
*Outflow
**natural pressure (zero)
**scalar_1 flux (zero)
**traction vector (zero vector)
*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
*Impermeable slip walls
**Comp 1 (zero in wall-normal direction)
**scalar_1 flux (zero)
**traction vector (zero vector)

=== Compressible ===

Common BCs used for a compressible case with the S-A turbulence model

*Initial conditions
**initial velocity (nonzero, typically small)
**initial scalar_1 (3-5 times free-stream molecular viscosity)
**initial pressure
**initial temperature

*Inflow
**Comp 3
**scalar_1 (also 3-5 times free-stream molecular viscosity)
**temperature
**pressure

*Outflow
**scalar_1 flux (zero)
**traction vector (zero vector)
**heat flux (zero)

*Solid physical walls
**Comp 3 (zero vector)
**scalar_1 (zero)
**turbulence wall (value unimportant; use zero)
**temperature or heat flux

Getting Started with Simmodeler

2023-02-22T03:20:11Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select and set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file of this example, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file to grow the mesh at a desired rate.

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20) 
(3) 6.92E-06*(1.0+($x-0.25)*20) 
... 
(49) 1.00E-04*(1.0+($x-0.25)*20) 
(50) 1.00E-04*(1.0+($x-0.25)*20) 

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].

Getting Started with Simmodeler

2023-02-22T03:19:15Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select the set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file of this example, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file to grow the mesh at a desired rate.

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20) 
(3) 6.92E-06*(1.0+($x-0.25)*20) 
... 
(49) 1.00E-04*(1.0+($x-0.25)*20) 
(50) 1.00E-04*(1.0+($x-0.25)*20) 

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].

Getting Started with Simmodeler

2023-02-22T03:18:31Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select the set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file of this example, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file;

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20) 
(3) 6.92E-06*(1.0+($x-0.25)*20) 
... 
(49) 1.00E-04*(1.0+($x-0.25)*20) 
(50) 1.00E-04*(1.0+($x-0.25)*20) 

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].

Getting Started with Simmodeler

2023-02-22T03:17:37Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select the set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file;

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20) 
(3) 6.92E-06*(1.0+($x-0.25)*20) 
... 
(49) 1.00E-04*(1.0+($x-0.25)*20) 
(50) 1.00E-04*(1.0+($x-0.25)*20) 

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].

Getting Started with Simmodeler

2023-02-22T03:17:19Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select the set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file;

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20) 
(3) 6.92E-06*(1.0+($x-0.25)*20) 
. 
. 
. 
(49) 1.00E-04*(1.0+($x-0.25)*20) 
(50) 1.00E-04*(1.0+($x-0.25)*20) 

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].

Getting Started with Simmodeler

2023-02-22T03:17:00Z

Conrad54418: /* Boundary Layer Mesh Height Grows in X */

==Goal of using SimModeler==
In this section you will learn how to create a mesh and apply boundary and initial conditions to your <code>translated-model.smd</code> model file from the previous step. You will save both the model and mesh containing the boundary/initial conditions under new file names, which we will later prepare to be passed into '''Chef'''.

==Launching the Software ==
First, tunnel to viz002 or viz003 using <code>vglconnect -s viz00x</code>.

Then, set your environment as shown in the [https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4 convert step video].

Finally, run <code>vglrun simmodeler</code> in your terminal to launch simmodeler. Once SimModeler is opened, select File > Open Model. If you ran <code>vglrun simmodeler</code> while in your working directory, the <code>translated-model.smd</code> file should be immediately available to select and open.

==Accessing the User Manual==
When launching simmodeler, there is a blue question mark at the top right of the GUI. Click it and then click "launch manual" to open the user manual associated with the version of simmodeler you are using. This gives detailed descriptions of the various attributes and how they are defined to generate the desired mesh.

==Surface Meshing==

===2D and 1D Boundary Layers===

To generate a proper surface mesh, it is important that both 2D and 1D boundary layers are implemented. Note that a 2D Boundary layer is defined on a surface and a 1D Boundary layer is defined on a line. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_1D_2D_BLs.mp4 video] shows both 2D and 1D boundary layers being applied to an airfoil and the mesh that results from these applied attributes.

===Boundary Layer Mesh Height Grows in X===

An advanced technique for boundary layer meshing utilizes the "Specific Thicknesses" option of the 2D Boundary Layer attribute in SimModeler. This option can prescribe the boundary layer mesh to grow as it develops along the surface, mimicking the behavior of the boundary layer itself.

# Create a 2D Boundary Layer attribute on the edge where you expect the boundary layer to grow. Select the "Specific Thicknesses" type.
# Select the set the face where you want the boundary layer mesh to propagate into (it should be connected to the selected edge).
# Edit "Layer Thicknesses" and select "Import" in the pop-up window.
#* The imported file should have the thicknesses of each layer of the mesh, growing as a function of x. The example below shows the first and last few lines of such a setup. In this case, the first layer of the mesh is 5e-6 m tall at the beginning of the selected edge. In the geometry file, the beginning of the selected edge starts at 0.25 meters. As you traverse in x, the thickness of the mesh is scaled by a factor of 20. This pattern can be easily extrapolated in an excel file;

(1) 5.00E-06*(1.0+($x-0.25)*20) 
(2) 5.88E-06*(1.0+($x-0.25)*20)
(3) 6.92E-06*(1.0+($x-0.25)*20)
.
.
.
(49) 1.00E-04*(1.0+($x-0.25)*20)
(50) 1.00E-04*(1.0+($x-0.25)*20)

===Mesh Size and Face Extrusions===

Two more useful attributes when generating proper surface meshes are defining Mesh Sizes and Face Extrusions on your desired surfaces. Face Extrusions are useful when adding refinement over curved surfaces. This process is well covered in this [https://fluid.colorado.edu/tutorials/tutorialVideos/MeshingWingInRoom_MeshSize_FaceExt.mp4 video].

==Volume Meshing==

===3D Boundary Layers===
One of the most important aspects of volume mesh development is generating proper 3D boundary layers. This process is well outlined [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 here] from around 8:30 to 16:00. Note that all mesh attributes are set up under the "Meshing" tab.

===Mesh Refinement Zones===
Mesh refinement zones are useful for increasing the grid density within certain volumes of your simulation domain. The linked [https://fluid.colorado.edu/tutorials/tutorialVideos/Mesh_refinement_zone_tutorial.mp4 mesh refinement tutorial] will briefly walk you through how these are implemented.

==Applying Boundary and Initial Conditions==
A nice description of each of the common BCs used in both the Compressible and Incompressible builds of PHASTA is provided [[SimModeler|here]]. The same video as the [https://fluid.colorado.edu/tutorials/tutorialVideos/RajDemo.mp4 volume meshing step] shows how to apply boundary conditions for an incompressible case starting at around 16:20. This linked [https://fluid.colorado.edu/tutorials/tutorialVideos/BC_tutorial.mp4 video] also shows how the BCs are applied for an incompressible case. Note that the case needs to be called ''geom'' as is done and explained in the tutorial video. Once you are done applying the BC's, it is very important that you save out the model <code>.smd</code> file with these applied boundary conditions.

==Saving Out the Mesh==
Once you are satisfied with your mesh refinement and have applied boundary/initial conditions to your model, you are ready to generate and save out the final mesh. In the meshing tab, you will want to click ''Generate Mesh''. The default settings will suffice for proper mesh generation. If your grid is large, this step can take some time. Once the grid is finished generating, you will want to save out the generated <code>.sms</code> file as you will need it for the next step in the process.

==Next Steps - Preping for Chef==
Once the BCs are successfully implemented, you have saved out a model file with these BCs (<code>.smd</code>), and have generated and saved your mesh grid (<code>.sms</code> file), you are ready to prepare the grid for Chef. To get started on this step, head over to [[Prepping the Grid for Chef]].