https://fluid.colorado.edu/wiki/api.php?action=feedcontributions&feedformat=atom&user=Prte0550 2026-04-29T03:14:08Z User contributions MediaWiki 1.30.0 https://fluid.colorado.edu/wiki/index.php?title=PHASTA/Restart_Ordering&diff=2047 2024-03-20T19:39:11Z

<p>Prte0550: Change to update to what solution phasta writes under non-pressure primitive solves</p> <hr /> <div>Knowledge of the ordering of restarts is most useful for the creation of &lt;code&gt;.pht&lt;/code&gt; or &lt;code&gt;.phts&lt;/code&gt; files (used to tell ParaView what to read in from restarts in POSIX or SyncIO format respectively). There are both standard outputs from PHASTA that will be written no matter what, and optional ones that may depend on the simulation type or what type of analysis is planned on being done. <br /> <br /> In the following, index numbering will be in index-by-1 format, or Fortran format. When creating a &lt;code&gt;.pht&lt;/code&gt; or &lt;code&gt;.phts&lt;/code&gt; file and filling in the &lt;code&gt;start_index_in_phasta_array=&quot; &quot;&lt;/code&gt; section for each field, note that you should subtract one as ParaView reads this file using index-by-0.<br /> <br /> == Standard Outputs ==<br /> <br /> The output flow variables written to restarts are NOT dependent on the choice of variables used to solve a given setup. PHASTA will always write pressure, velocity, and temperature (in that order) for both compressible and incompressible for the sake of consistency. If the incompressible formulation is being used without a temperature solve, the temperature field will still exist but will simply be all zeros. These output flow variables are stored under the header<br /> &lt;code&gt;solution&lt;/code&gt;<br /> <br /> In the case where turbulence models are being used or species are being solved for, &lt;code&gt;solution&lt;/code&gt; will also populate scalar fields starting in the 6th field in sequential ordering:<br /> * Scalar 1<br /> * Scalar 2<br /> <br /> The final ordering is then:<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Pressure Primitive Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Pressure ||<br /> |-<br /> | 2:4 || Velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 5 || Temperature || This field can be ignored if incompressible<br /> |-<br /> | 6 || Scalar 1 || Only written if scalar 1 exists<br /> |-<br /> | 7 || Scalar 2 || Only written if scalar 2 exists<br /> |}<br /> <br /> The exact quantity that any scalar represents depends on the models being used, though as some examples, for the Spalart-Allmaras (SA) 1-equation RANS model or DES/DDES formulations using the SA model, scalar 1 will be &amp;nu &lt;sub&gt;t&lt;/sub&gt;. Some branches of the code may have the ability to solve more scalar equations and those extra scalars will be appended, in order, to the end of the &lt;code&gt;solution&lt;/code&gt; field.<br /> <br /> === Time Derivatives ===<br /> The time derivatives of all of the above fields are also present in the restarts, under the header &lt;code&gt;time derivative of solution&lt;/code&gt;. These fields are in the exact same order as in &lt;code&gt;solution&lt;/code&gt;, and are subject to the same caveats about how many scalars there may be and what they actually represent for any given branch of the code and simulation setup.<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Pressure Primitive Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Time derivative of pressure ||<br /> |-<br /> | 2:4 || Time derivative of velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 5 || Time derivative of temperature || This field can be ignored if incompressible<br /> |-<br /> | 6 || Time derivative of scalar 1 || Only written if scalar 1 exists<br /> |-<br /> | 7 || Time derivative of scalar 2 || Only written if scalar 2 exists<br /> |}<br /> <br /> == Optional Outputs ==<br /> <br /> === Wall Distance ===<br /> This quanity is simply a measure of the distance from any given node to the nearest wall point in the simulation. This quantity is mostly used in turbulence models but can be useful for the post-processing of complex domains<br /> *Header: &lt;code&gt;dwal&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;placeholder&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Wall Distance || <br /> |}<br /> <br /> === Vorticity ===<br /> *Header: &lt;code&gt;vorticity&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print vorticity: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Vorticity || Vector quantity, ordered &amp;omega;&lt;sub&gt;x&lt;/sub&gt;, &amp;omega;&lt;sub&gt;y&lt;/sub&gt;, &amp;omega;&lt;sub&gt;z&lt;/sub&gt;<br /> |-<br /> | 4 || Magnitude of vorticity || <br /> |-<br /> | 5 || Q || Defined as the second invariant of the velocity gradient tensor<br /> |}<br /> <br /> <br /> === Time Averaged Statistics (point-wise) ===<br /> Point-wise time averaged statistics are useful for problems where there are no homogeneous directions in the flow to accumulate an average along. Instead, PHASTA can accumulate averages at each individual node. Note that this formulation only accumulates per-run, so an average is only computed from the start step of the current run, and total averages must be computed by adding successive averages with the appropriate weighting. <br /> *Header: &lt;code&gt;ybar&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print ybar: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields (incompressible)<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Average velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 2:4 || Average pressure || <br /> |-<br /> | 5 || Average speed || <br /> |-<br /> | 6:8 || Average of the square of velocity || Vector quantity, ordered u&lt;sup&gt;2&lt;/sup&gt;, v&lt;sup&gt;2&lt;/sup&gt;, w&lt;sup&gt;2&lt;/sup&gt;<br /> |-<br /> | 9 || Average of the square of pressure ||<br /> |-<br /> | 10:12 || Average of velocity cross-components || Vector quantity, ordered uv, uw, vw<br /> |-<br /> | 13 || Average of scalar 1 ||<br /> |-<br /> | 14:16 || Average of vorticity || Vector quantity, ordered &amp;omega;&lt;sub&gt;x&lt;/sub&gt;, &amp;omega;&lt;sub&gt;y&lt;/sub&gt;, &amp;omega;&lt;sub&gt;z&lt;/sub&gt;<br /> |-<br /> | 17 || Average vorticity magnitude ||<br /> |-<br /> | 18 || Average of scalar 2 ||<br /> |}<br /> <br /> Average of Q may be in position 19 depending on the branch<br /> <br /> <br /> === Wall Shear Stress ===<br /> This field is only defined at wall points and is used to get the most accurate measure of the wall shear stress possible as otherwise a finite gradient using the first point off of the wall must be implemented, which may or may not be normal to a wall point underneath of it. <br /> *Header: &lt;code&gt;wss&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print Wall Fluxes: True&lt;/code&gt; (check this)<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Wall shear stress || Vector quantity, ordered &amp;tau;&lt;sub&gt;wall&lt;sub&gt;x&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;y&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;z&lt;/sub&gt;&lt;/sub&gt;<br /> |}<br /> <br /> === Time Averaged Wall Shear Stress ===<br /> *Header: &lt;code&gt;wssbar&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print Wall Fluxes: True&lt;/code&gt; and &lt;code&gt;Print ybar: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Average wall shear stress || Vector quantity, ordered &amp;tau;&lt;sub&gt;wall&lt;sub&gt;x&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;y&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;z&lt;/sub&gt;&lt;/sub&gt;<br /> |}<br /> <br /> === Pressure Projection Vectors ===<br /> *Header: &lt;code&gt;pressure projection vectors&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;placeholder&lt;/code&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=PHASTA/Restart_Ordering&diff=2045 2024-03-12T17:29:17Z

<p>Prte0550: Initial creation</p> <hr /> <div>Knowledge of the ordering of restarts is most useful for the creation of &lt;code&gt;.pht&lt;/code&gt; or &lt;code&gt;.phts&lt;/code&gt; files (used to tell ParaView what to read in from restarts in POSIX or SyncIO format respectively). There are both standard outputs from PHASTA that will be written no matter what, and optional ones that may depend on the simulation type or what type of analysis is planned on being done. <br /> <br /> In the following, index numbering will be in index-by-1 format, or Fortran format. When creating a &lt;code&gt;.pht&lt;/code&gt; or &lt;code&gt;.phts&lt;/code&gt; file and filling in the &lt;code&gt;start_index_in_phasta_array=&quot; &quot;&lt;/code&gt; section for each field, note that you should subtract one as ParaView reads this file using index-by-0.<br /> <br /> NOTE: <br /> This page is a work in progress and none of this information should be taken as fully accurate while this warning persists<br /> <br /> == Standard Outputs ==<br /> <br /> The output flow variables written to restarts are dependent on the choice of variables used to solve a given setup. These output flow variables are stored under the header<br /> &lt;code&gt;solution&lt;/code&gt;<br /> And for pressure primitive, have ordering:<br /> * Pressure<br /> * Velocity (vector quantity, ordered x, y, z)<br /> * Temperature (if solving the compressible equations, this field can be ignored if incompressible)<br /> <br /> In the case where turbulence models are being used or species are being solved for, &lt;code&gt;solution&lt;/code&gt; will also populate scalar fields starting in the 6th field in sequential ordering:<br /> * Scalar 1<br /> * Scalar 2<br /> <br /> The final ordering is then:<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Pressure Primitive Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Pressure ||<br /> |-<br /> | 2:4 || Velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 5 || Temperature || This field can be ignored if incompressible<br /> |-<br /> | 6 || Scalar 1 || Only written if scalar 1 exists<br /> |-<br /> | 7 || Scalar 2 || Only written if scalar 2 exists<br /> |}<br /> <br /> The exact quantity that any scalar represents depends on the models being used, though as some examples, for the Spalart-Allmaras (SA) 1-equation RANS model or DES/DDES formulations using the SA model, scalar 1 will be &amp;nu &lt;sub&gt;t&lt;/sub&gt;. Some branches of the code may have the ability to solve more scalar equations and those extra scalars will be appended, in order, to the end of the &lt;code&gt;solution&lt;/code&gt; field.<br /> <br /> === Time Derivatives ===<br /> The time derivatives of all of the above fields are also present in the restarts, under the header &lt;code&gt;time derivative of solution&lt;/code&gt;. These fields are in the exact same order as in &lt;code&gt;solution&lt;/code&gt;, and are subject to the same caveats about how many scalars there may be and what they actually represent for any given branch of the code and simulation setup.<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Pressure Primitive Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Time derivative of pressure ||<br /> |-<br /> | 2:4 || Time derivative of velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 5 || Time derivative of temperature || This field can be ignored if incompressible<br /> |-<br /> | 6 || Time derivative of scalar 1 || Only written if scalar 1 exists<br /> |-<br /> | 7 || Time derivative of scalar 2 || Only written if scalar 2 exists<br /> |}<br /> <br /> == Optional Outputs ==<br /> <br /> === Wall Distance ===<br /> This quanity is simply a measure of the distance from any given node to the nearest wall point in the simulation. This quantity is mostly used in turbulence models but can be useful for the post-processing of complex domains<br /> *Header: &lt;code&gt;dwal&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;placeholder&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Wall Distance || <br /> |}<br /> <br /> === Vorticity ===<br /> *Header: &lt;code&gt;vorticity&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print vorticity: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Vorticity || Vector quantity, ordered &amp;omega;&lt;sub&gt;x&lt;/sub&gt;, &amp;omega;&lt;sub&gt;y&lt;/sub&gt;, &amp;omega;&lt;sub&gt;z&lt;/sub&gt;<br /> |-<br /> | 4 || Magnitude of vorticity || <br /> |-<br /> | 5 || Q || Defined as the second invariant of the velocity gradient tensor<br /> |}<br /> <br /> <br /> === Time Averaged Statistics (point-wise) ===<br /> Point-wise time averaged statistics are useful for problems where there are no homogeneous directions in the flow to accumulate an average along. Instead, PHASTA can accumulate averages at each individual node. Note that this formulation only accumulates per-run, so an average is only computed from the start step of the current run, and total averages must be computed by adding successive averages with the appropriate weighting. <br /> *Header: &lt;code&gt;ybar&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print ybar: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields (incompressible)<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1 || Average velocity || Vector quantity, ordered u, v, w<br /> |-<br /> | 2:4 || Average pressure || <br /> |-<br /> | 5 || Average speed || <br /> |-<br /> | 6:8 || Average of the square of velocity || Vector quantity, ordered u&lt;sup&gt;2&lt;/sup&gt;, v&lt;sup&gt;2&lt;/sup&gt;, w&lt;sup&gt;2&lt;/sup&gt;<br /> |-<br /> | 9 || Average of the square of pressure ||<br /> |-<br /> | 10:12 || Average of velocity cross-components || Vector quantity, ordered uv, uw, vw<br /> |-<br /> | 13 || Average of scalar 1 ||<br /> |-<br /> | 14:16 || Average of vorticity || Vector quantity, ordered &amp;omega;&lt;sub&gt;x&lt;/sub&gt;, &amp;omega;&lt;sub&gt;y&lt;/sub&gt;, &amp;omega;&lt;sub&gt;z&lt;/sub&gt;<br /> |-<br /> | 17 || Average vorticity magnitude ||<br /> |-<br /> | 18 || Average of scalar 2 ||<br /> |}<br /> <br /> Average of Q may be in position 19 depending on the branch<br /> <br /> <br /> === Wall Shear Stress ===<br /> This field is only defined at wall points and is used to get the most accurate measure of the wall shear stress possible as otherwise a finite gradient using the first point off of the wall must be implemented, which may or may not be normal to a wall point underneath of it. <br /> *Header: &lt;code&gt;wss&lt;/code&gt;<br /> *&lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print Wall Fluxes: True&lt;/code&gt; (check this)<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Wall shear stress || Vector quantity, ordered &amp;tau;&lt;sub&gt;wall&lt;sub&gt;x&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;y&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;z&lt;/sub&gt;&lt;/sub&gt;<br /> |}<br /> <br /> === Time Averaged Wall Shear Stress ===<br /> Header: &lt;code&gt;wssbar&lt;/code&gt;<br /> &lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;Print Wall Fluxes: True&lt;/code&gt; and &lt;code&gt;Print ybar: True&lt;/code&gt;<br /> <br /> {| class=&quot;wikitable&quot;<br /> |+ Fields<br /> |-<br /> ! Field Number !! Description !! Notes<br /> |-<br /> | 1:3 || Average wall shear stress || Vector quantity, ordered &amp;tau;&lt;sub&gt;wall&lt;sub&gt;x&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;y&lt;/sub&gt;&lt;/sub&gt;, &amp;tau;&lt;sub&gt;wall&lt;sub&gt;z&lt;/sub&gt;&lt;/sub&gt;<br /> |}<br /> <br /> === Pressure Projection Vectors ===<br /> Header: &lt;code&gt;pressure projection vectors&lt;/code&gt;<br /> &lt;code&gt;solver.inp&lt;/code&gt; flag: &lt;code&gt;placeholder&lt;/code&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1957 2023-08-18T21:32:58Z

<p>Prte0550: Updated location of latest version</p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;/nobackup/uncompressed/Models/GustWing/2dOTS/SymmetricRoom/Mesh/MGEN_NR&lt;/code&gt; as of January 2023. This version can read in model tags, handle multiple model regions, and arbitrary domain widths. <br /> <br /> <br /> == Basic Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Convert]] tool for unstructured grids.<br /> <br /> <br /> Once the mesh files are in place, MGEN can be run with &lt;code&gt;./tm3Extrude&lt;/code&gt; as usual. The code will ask for inputs for zmin, zmax, numelz, and npart. These should be entered in a single string with spaces in between the values before hitting in order to continue code execution.<br /> <br /> == Advanced Usage ==<br /> For more complex geometries, complete information about the model cannot be assumed and must instead be given to MGEN. In order to prepare for this, we will need an additional form of the mesh and to make changes to the MGEN code itself in order to tell the program where geometric features are. <br /> <br /> A model first needs to be converted into a &lt;code&gt;.dmg&lt;/code&gt; file. This can be created with &lt;code&gt;mdlConvert&lt;/code&gt;. Example usage of this is &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert &lt;simmetixMesh&gt;.xmt_txt outModel.dmg&lt;/code&gt;. This captures only information about model points, edges, and faces and their relationships to each other but does not capture information about physical location.<br /> <br /> == Outputs ==<br /> MGEN will write its outputs to the same working directory that the executable and source mesh files are in. There are multiple file types written, most with a suffix of a number to denote the part number of that file. The different parted files and their purposes are as follows:<br /> <br /> ;geom3D.class :Classification file describing what type of geometric entity each point lies on (vertex, edge, face, volume)<br /> ;geom3D.cnndt :Connectivity of the elements <br /> ;geom3D.coord :Node coordinates<br /> ;geom3D.fathr :Parent vertex from the 2D source mesh<br /> ;geom3D.match :Contains periodic partners<br /> <br /> <br /> There is also one more file:<br /> <br /> ; geom3DHead.cnn<br /> <br /> Which lists the headers containing information on the size of the file each of the above connectivity files.<br /> <br /> == Using the outputted files ==<br /> The outputted files from MGEN now need to be prepared for Chef, this is done via &lt;code&gt;matchedNodeElmReader&lt;/code&gt;. The provided example will be for a build on Cooley.<br /> <br /> First, the environment needs to be prepared via setting &lt;code&gt;SIM_LICENSE_FILE&lt;/code&gt; and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt;. Examples of this are:<br /> <br /> <br /> &lt;code&gt;<br /> export SIM_LICENSE_FILE=/eagle/PHASTA_aesp/SCOREC-CORE/deps/Simmetrix/UCBoulder<br /> <br /> export LD_LIBRARY_PATH=/eagle/PHASTA_aesp/SCOREC-CORE/deps/16.0-220326/lib/x64_rhel_gcc48/psKrnl/:$LD_LIBRARY_PATH<br /> &lt;/code&gt;<br /> <br /> <br /> From here, &lt;code&gt;matchedNodeElmReader&lt;/code&gt; can be run with:<br /> <br /> &lt;code&gt;<br /> mpirun -f /var/tmp/cobalt.2137783 -np &lt;np&gt; -genvall /eagle/PHASTA_aesp/SCOREC-CORE/build_gtvertCorruption/test/matchedNodeElmReader ../geom3D.cnndt ../geom3D.coord ../geom3D.match ../geom3D.class ../geom3D.fathr NULL ../geom3DHead.cnn outModel.dmg outModel/<br /> &lt;/code&gt;<br /> <br /> Where &lt;np&gt; should be replaced by the same number as used for npart when running MGEN.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Paraview_Trace&diff=1948 2023-06-16T18:38:32Z

<p>Prte0550: </p> <hr /> <div>Paraview traces are ways to create a python script of a set of actions that can be later applied to different datasets or over a loop of datasets automatically. The following will brefly explain the creation, cleaning, and running of a python trace in Paraview (which will often be shortened to pvTrace or something similar).<br /> <br /> === Creation ===<br /> <br /> === Basic Changes to the Python Script ===<br /> <br /> === Running a Trace ===<br /> Running a pvTrace, whether on the same or a different dataset requires only a few key steps. For now, it will be assumed that the trace is being run on Cooley at ALCF, though many of the steps should be shared for other machines. <br /> <br /> If you are running the trace on a dataset that is not the same as the one for which you created the trace, it is good practice to always check your script and inputs. Make sure you have all of the restart and geombc files that you will need, and that you are pointing to the correct locations in the python script (it is recommended that you use absolute paths to reduce the chances for error). Also check your output file name and location.<br /> <br /> Once your python script is ready, you need to get an interactive allocation on Cooley and load the same version of paraview with which the trace was created:<br /> <br /> &lt;code&gt;soft add +paraview-5.5.2&lt;/code&gt;<br /> <br /> This loads &lt;code&gt;pvpython&lt;/code&gt; which is what is used to run the python trace script. To run this scrip is simply a modification of a standard python run command. <br /> <br /> &lt;code&gt;pvpython Trace_Script.py&lt;/code&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Paraview_Trace&diff=1947 2023-06-16T18:37:48Z

<p>Prte0550: Initial creation, only added running for now because I need to do other things and those are the steps I just forgot and had to re-learn. Hoping to circle back later</p> <hr /> <div>== Paraview traces are ways to create a python script of a set of actions that can be later applied to different datasets or over a loop of datasets automatically. The following will brefly explain the creation, cleaning, and running of a python trace in Paraview (which will often be shortened to pvTrace or something similar). ==<br /> <br /> === Creation ===<br /> <br /> === Basic Changes to the Python Script ===<br /> <br /> === Running a Trace ===<br /> Running a pvTrace, whether on the same or a different dataset requires only a few key steps. For now, it will be assumed that the trace is being run on Cooley at ALCF, though many of the steps should be shared for other machines. <br /> <br /> If you are running the trace on a dataset that is not the same as the one for which you created the trace, it is good practice to always check your script and inputs. Make sure you have all of the restart and geombc files that you will need, and that you are pointing to the correct locations in the python script (it is recommended that you use absolute paths to reduce the chances for error). Also check your output file name and location.<br /> <br /> Once your python script is ready, you need to get an interactive allocation on Cooley and load the same version of paraview with which the trace was created:<br /> <br /> &lt;code&gt;soft add +paraview-5.5.2&lt;/code&gt;<br /> <br /> This loads &lt;code&gt;pvpython&lt;/code&gt; which is what is used to run the python trace script. To run this scrip is simply a modification of a standard python run command. <br /> <br /> &lt;code&gt;pvpython Trace_Script.py&lt;/code&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Chef/Mesh_Partitioning&diff=1935 2023-02-03T20:37:34Z

<p>Prte0550: /* Documentation */</p> <hr /> <div>This webpage is first inspired from a tutorial provided to Igor and his team at NCSU in order to set up two phase flow test cases on a local cluster named Firebird at NCSU and Cetus/Mira at ALCF. At this time, this tutorial includes copy-paste materials from old emails. <br /> <br /> The code has evolved since then! If you scroll down, you will also find critical updates since the first tutorial was written. Please do not ignore them or there is 100% chance your mesh partitioning/refinement will fail.<br /> <br /> Please update this page for our viz nodes when you get a chance. <br /> <br /> Thanks, <br /> <br /> - Michel<br /> <br /> <br /> <br /> <br /> <br /> Here is a tutorial about how to respectively partition the initial mesh and generate the phasta files on firebird (and other platforms including Cetus/Mira) using Chef. This tutorial is rather long but should include everything you need.<br /> The testcase to demonstrate the workflow is the familiar 3-way subchannel flow. The root path of this test case is /sgidata2/mrasquin/Models/subchannel. The parasolid model is located in /sgidata2/mrasquin/Models/subchannel/convertParasolid2ParasolidNative/geomFromSimmodeler_nat.xmt_txt.<br /> The workflow that describes how to use Chef is now explained in the next sections.<br /> <br /> = Documentation =<br /> General documentation of Chef that may be used in supplement to this page is available [https://github.com/SCOREC/core/wiki/chef-partition-control here]<br /> <br /> = Initial tutorial =<br /> <br /> == Env variables==<br /> <br /> All the subsequent tools need<br /> * The fresh version of openmpi I built on firebird<br /> * The latest Simmetrix library I installed in /Install on firebird.<br /> <br /> To update your paths, source the following file:<br /> &lt;code&gt;/Install/SCOREC.develop/envLinux2014.sh&lt;/code&gt;.<br /> <br /> The env variables defined or updated in this env script include PATH and LD_LIBRARY_PATH. What is defined in this script should prevail on your settings but I strongly suggest removing any redundancy that you may have, for instance, in your .basrc. Note that I actually source this env file directly in my .bashrc so that I do not have to do it manually every time I log in to firebird. When you source it, it will also print the version of gcc, openmpi and simmodsuite lib that are set up.<br /> <br /> == BLMesherParallel ==<br /> <br /> Note that Simmetrix only supports matched faces for single part mesh so that the mesh must be built with one core. However the initial mesh must already include some information related to the partitioning, even if the mesh only includes a single part for format reasons. This additional information about the partitioning is required for conversion of the mesh file from the Simmetrix format to the SCOREC MDS format that Chef can read.<br /> <br /> The initial mesh for the 3-way subchannel was built in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0&lt;/code&gt;. Check the script named &lt;code&gt;runBLMesherParallel.sh&lt;/code&gt; in this directory.<br /> <br /> Running &lt;code&gt;./runBLMesherParallel.sh&lt;/code&gt; with no arguments will tell you the usage, that is:<br /> Usage: ./runBLMesherParallel.sh &lt;X&gt; &lt;Y&gt; &lt;Z&gt;<br /> <br /> The arguments are as follows.<br /> * &lt;X&gt; (geometric model) should be the parasolid model geomFromSimmodeler_nat.xmt_txt.<br /> * &lt;Y&gt; (attribute file) should be BLattr.inp.<br /> * &lt;Z&gt; (number of processors) should be 1 here since we need to generate a single part mesh using a single core.<br /> <br /> The BLattr.inp input file is the same as the one read by the old serial version of BLMesher. But BLMesherParallel can do whatever the old version of BLMesher can do. In addition, if your test case does not include any matched face, you may try to mesh in parallel by specifying &lt;Z&gt; to be larger than 1. However, some meshing features are available only when BLMesherParallel is used with a single core so it is always important to check the resulting mesh.<br /> <br /> BLMesherParallel outputs the following files.<br /> <br /> * &lt;code&gt;mesh.sms&lt;/code&gt; --- The resulting mesh is stored in a directory named mesh.sms, which is a parameter hardcoded in the runBLMesherParallel.sh script.<br /> * &lt;code&gt;BLMesher.log&lt;/code&gt; --- The log from BLMesherParallel is saved in BMesher.log, whereas the Simmetrix log is saved in mesh.log. Both filenames are also hardcoded in the script.<br /> <br /> I also mentioned in previous discussions that Simmetrix has developed its own model format called geomsim. However, the boundary layer collapses near matched faces with this model format, which is not the case when we use the parasolid format. This issue has been reported to Simmetrix but until they can provide a fix, we are forced to start with the parasolid format when our test cases include matched faces.<br /> <br /> == Mesh conversion==<br /> <br /> Chef can read only the MDS format developed at SCOREC. Therefore, the Simmetrix mesh mush first be converted to this format.<br /> <br /> This operation was carried out for the 3-way channel in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/simMeshToMdsMesh&lt;/code&gt;. Simply run the script &lt;code&gt;./simMeshToMdsMesh.sh&lt;/code&gt;, which executes the &quot;convert&quot; executable. In the script, you can see that the convert executable reads 3 arguments:<br /> # The '''input parasolid model''' named named geom.xmt_txt, which points to geomFromSimmodeler_nat.x_t. Note that convert expects an .xmt_txt extension (or .smd extension for the complete geomsim format).<br /> # The '''input Simmetrix mesh''' named here parts.sms (for historical reason but can be renamed).<br /> # The '''name of the output mds mesh directory''', which is mdsMesh_bz2 here. Note that this name is prepended by &quot;bz2:&quot;, which means that the output mds mesh file is compressed using bzip2. &quot;bz2:&quot; will not be part of the name of the output directory. If you do not specify &quot;bz2:&quot;, the mds mesh file will be saved in ascii format, which is a waste of space so I suggest to always prepend your directory name by &quot;bz2:&quot;. This will also apply later to the output mesh directory generated by Chef (see below).<br /> <br /> Note that convert needs to run with a number of processes (-np ##) equal to the number of input parts in the Simmetrix mesh. For cases that include match faces, the Simmetrix mesh must include only one part, which is the reason why convert runs here with -np 1. But in other circumstances, convert can run in parallel if the Simmetrix mesh has already been partitioned in n parts with n&gt;1 (for instance mesh generated in parallel with BLMesherParallel and/or partitioned with phParAdapt-Simmetrix).<br /> <br /> == Boundary and initial conditions (spj file)==<br /> <br /> Before running Chef for mesh operations such as uniform refinement, tetrahedronization and partitioning, we need to define the BCs and ICs for the generation of the phasta files. These BCs and ICs are defined in an spj file, which is in ASCII to facilitate scripting of BCs/ICs. Most of the attributes you are familiar with from the Simmodeler GUI can be specified in the spj file.<br /> <br /> For the 3-way channel flow, see the spj file located in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Simplified_SPJ_file/geom.spj&lt;/code&gt;. Each line corresponds to one attribute that applies to one face.<br /> <br /> The structure of the spj file is:<br /> # Optional comments anywhere preceded by the pound symbol (#).<br /> # For each boundary or initial condition a line as follows:<br /> &lt;attribute_name&gt;: &lt;face_id&gt; &lt;dimension&gt; &lt;attribute list&gt;<br /> <br /> Note the following.<br /> * &lt;code&gt;&lt;dimension&gt;&lt;/code&gt;: 2 for a face attribute in 2D, 3 for the initial conditions that applies to the 3D domain. 1D and 0D attributes are also allowed for lines and vertices if needed.<br /> * &lt;code&gt;&lt;attribute list&gt;&lt;/code&gt;: typically magnitude and direction if this applies&gt;.<br /> <br /> Syntax is strict.<br /> * No empty line. Each line should be either a comment which starts with the # character, or an attribute.<br /> * There must be one single space after the semicolon character.<br /> * There must be one single space between any numbers.<br /> <br /> In this example, a zero &quot;traction vector&quot; attribute is specified on the periodic faces parallel to the length of the channel. It is wrong to specify such an attribute on these periodic faces for a 3-way channel, but this was inherited from the 1-way periodic channel where these faces were slip walls instead of periodic faces. I will try to update my test cases in the future. But because we have now continuous integration tools that run every night to verify the Chef code, I will need to update all the cases if I modify the spj file now. So double check the attributes that you need for this model and consider the existing spj file as a source of inspiration rather than the correct spj file for production runs.<br /> <br /> == Chef ==<br /> <br /> A few rules must be followed to run Chef.<br /> <br /> First, the number of mpi processes must be equal to the number of input parts (''this has changed in the newest version of Chef, as described below'').<br /> <br /> Second, Chef is threaded with openmp and the total number of output parts after partitioning should be at most equal to the total number of available hardware threads of your machine/allocation. On BGQ, there are 4 hardware threads per core. On Linux platform such as firebird, the number of hardware threads corresponds to the number of available cores. That said, we have observed that if the number of output parts is equal to the total number of available hardware threads, Chef can hang. Therefore, it is safer to limit the number of output parts to a lower number than the number of available hardware threads. Therefore, on firebird, we should not try to partition a mesh to more than 16 parts.<br /> <br /> The next mesh operations will have to take place on Tukey and Cetus/Mira.<br /> <br /> The first example of a partitioning with Chef can be found in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch&lt;/code&gt;. With my naming convention, &lt;code&gt;4-1-Chef-PartLocal-Scratch&lt;/code&gt; can be decomposed as follows:<br /> * The first number (4) corresponds to the number of output parts<br /> * The second number (1) correspond to the number of input parts<br /> * &quot;Chef&quot; means this mesh was treated with this program (in opposition to phParAdapt, phTest, etc which are previous executables that we used for similar purpose).<br /> * &quot;PartLocal&quot; means the mesh is partitioned locally.<br /> * &quot;Scratch&quot; means that the initial solution in the resulting phasta files is generated entirely from the spj file defined in a previous section of this tutorial. That is, we are starting a simulation &quot;from scratch,&quot; using the spj file's initial conditions as opposed to a solution migrated from a previous run.<br /> <br /> In summary, Chef was used in this directory to partition a single part mesh into 4 parts and the solution in the phasta files was generated directly from scratch using the spj file.<br /> <br /> === Chef's input files ===<br /> <br /> The script to run Chef is named runChef.sh in this directory and simply call the executable. Chef reads all it needs from two input files called numstart.dat and adapt.inp.<br /> <br /> ==== numstart.dat ====<br /> <br /> Instead of building the initial solution from scratch using the initial conditions defined in the spj file, the user can migrate an existing solution stored in a set of restart files that were saved from a previous phasta simulation. Numstart.dat contains the time step stamp of the input restart files to read in order to migrate a solution.<br /> <br /> ==== adapt.inp ====<br /> <br /> This input file contains all the other parameters Chef expects. Note that many of these parameters have been inherited from the old phParAdapt, and are currently obsolete or unused. In what follows, all the parameters available in adapt.inp are listed and the critical parameters are in bold. Any line that starts with # is ignored.<br /> <br /> * '''globalP''': obsolete/unused.<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Note that adaptation only works with a mixed mesh (with wedges in the BL) and not with an all-tet mesh. Tetrahedronization should therefore take place after uniform refinement. Right now, the mixed mesh gets uniformly refined everywhere including the BL but it is possible to refine uniformly outside the BL only with some light modifications of the code. In the future, we hope to have other adaptation strategies in place in Chef based on local error indicator. If interested in these strategy, then phParAdapt-Simmetrix must be used. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * rRead: obsolete/unused.<br /> <br /> * rStart: obsolete/unused.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested in Chef. If interested in other more sophisticated adaptation strategies, phParAdapt-Simmetrix must be used for now.<br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * Periodic: obsolete. Periodicity in the mesh and in the solution is not treated automatically as long as i) the mesh built with BLMesher is periodic (i.e. location of the mesh vertices on periodic faces in the same) and ii) the spj file contains the correct &quot;periodic slave&quot; attributes.<br /> <br /> * prCD: obsolete/unused.<br /> <br /> * timing: obsolete/unused.<br /> <br /> * outputFormat: obsolete. Phasta files are saved by default in binary format.<br /> <br /> * internalBCNodes: obsolete/unused.<br /> <br /> * WRITEASC: obsolete/unused.<br /> <br /> * phastaIO: obsolete/unused.<br /> <br /> * '''numTotParts''': Final number of parts. If numTotParts is larger than the number of Chef processes which is equal to the number of input parts, the mesh will be partitioned.<br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * isReorder: obsolete/unused. Reordering for better cache performance is now applied by default to both the phasta files and mesh files.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * numSplit: obsolete/unused.<br /> <br /> * '''LocalPtn''': local partitioning if set to 1, set global partitioning if set to 0. Currently, only local partitioning is implemented in Chef and has been shown to be sufficient so far.<br /> <br /> * '''RecursivePtn''': should always be set to 1. In the past, this parameter allowed recursive partitioning steps in phParAdapt. The code will stop or crash if this parameter is not 1.<br /> <br /> * RecursivePtnStep: obsolete/unused.<br /> <br /> * '''partitionMethod''': Currently, the GRAPH method for local partitioning is hard coded in one of the Chef routine.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> The second example of a partitioning with Chef can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-Tet-PartLocal-SolMgr. For this case, based on the naming convention of 8-4-Chef-Tet-PartLocal-SolMgr (and the parameters specified in adapt.inp and numstart.dat),<br /> * the number of output parts requested is 8, <br /> * the number of input parts is 4 (note &quot;-np 4&quot; in the runChef.sh script),<br /> * the input mixed mesh is first tetrahedronized before being partitioned. <br /> * the solution in the resulting phasta files is migrated from the previous Chef run. <br /> Note that the spj file is different for this second example and the initial conditions have been commented out in order not to overwrite the solution that is migrated from the previous Chef run.<br /> <br /> The third and final example can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-UR2-Tet-PartLocal-SolMgr. In this directory 8-4-Chef-UR2-Tet-PartLocal-SolMgr, Chef <br /> * reads a four part mesh, <br /> * applies a double recursive uniform refinement, <br /> * tetrahedronize the resulting mixed mesh that has been uniformly refined twice, <br /> * partition the resulting 4 part all-tet uniformly refined mesh into 8 parts,<br /> * migrate and interpolate the solution read from existing restart files coming from the first example.<br /> <br /> As a final comment, note that the restart files are always read directly from a procs_case directory. However, when the number of output restart files exceeds 2048, the restart files are then saved in subdirectories of the root procs_case directory in order to reduce file contention, in the same (but still different) way as what you have implemented at some point in your version of phasta. The best strategy would be to write phasta files using mpi_io for instance so that we can store more than one part in a single file and avoid large number of phasta files.<br /> <br /> For further partitioning on BG/Q machines a conversion to the native Parasolid model is required. The tool is located in: /Install/SCOREC.develop/scorec/test/cadToSim/cadToSim <br /> and should be run from [Case directory]/convertParasolid2ParasolidNative/ on firebird.<br /> <br /> = Updated Chef version (2015/03/26)=<br /> <br /> <br /> 1) MPI implementation<br /> <br /> A new version of chef has been implemented and does not rely on threads any more.<br /> Instead, it is now based on a pure MPI implementation. <br /> That means that there is an important change in how chef is called at runtime.<br /> <br /> With the previous threaded version, the number of MPI processes had to be equal to the number of input parts. <br /> Chef was then in charge of starting a number of threads equal to the number of output parts, which was automatic.<br /> <br /> Since the pure MPI version of chef does not start thread any more, it now requires a number of MPI processes equal to the final number of output parts, and not input parts.<br /> <br /> <br /> 2) adapt.inp<br /> <br /> In the new version of chef, &quot;numTotParts&quot; in adapt.inp (which was used to specify the final number of output parts) has been replaced by &quot;splitFactor&quot;, which corresponds to the ratio of the number of output parts with the number of input parts. <br /> If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> If you set this parameter to 2, each part of your input mesh will be split in 2 new sub-parts, etc<br /> Keep in mind that the number of MPI processes that needs to be requested for chef must therefore be equal to (number of input parts times) * (splitFactor).<br /> <br /> I have also removed the obsolete parameter in adapt.inp and saved a representative version of this file in /projects/tools/SCOREC.develop/runscripts/adapt.inp<br /> <br /> <br /> 3) Paths<br /> <br /> I have updated chef on the VIz nodes, Mira and Tukey so that it only relies on the more robust pure MPI implementation.<br /> <br /> On the viz nodes, use /projects/tools/SCOREC.develop/build-chefMPI-GNU-*/test/chef<br /> For simplicity, this is the default version of the master branch coming directly from our github repository.<br /> <br /> On Tukey, use /home/mrasquin/SCOREC.develop/build-tukey-GNU-OptG-c2c360bc-mpi-*<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35-noblsnap means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is off during uniform refinement (UR).<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35 means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is on during UR.<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol33 means that the target imbalance for both the vtx and elem is 3%, and BL snapping is on during UR.<br /> Note that these versions have been slightly modified w.r.t. the master branch. In particular, the imbalance target is not a parameter yet. Also, in Parma, HPS (Heavy Part Splitting) and FixDisconnectedPart are not called here because the latest version of the diffusion algorithm with improved selection of (i) target parts for element exchange and (ii) elements.<br /> <br /> On Mira, use /home/mrasquin/SCOREC.develop/build-XL-OptG-c2c360bc-mpi-*<br /> Similar comments applies to build-XL-OptG-c2c360bc-mpi-tol33, build-XL-OptG-c2c360bc-mpi-tol35 and build-XL-OptG-c2c360bc-mpi-tol35-noblsnap.<br /> <br /> Note that BL snapping is not called for a repartitioning of the mesh. It can only play a role during uniform refinement.<br /> Consequently, if you do not request a UR in adapt.inp, then build-*-tol35 and build-*-tol35-noblsnap will behave the same way.<br /> <br /> In case you are wondering what the weird numbers are in the name of the build directory, this comes from the git log hash, which is a unique number associated with a git commit (easier to couple an executable with a version of the code).<br /> <br /> <br /> = Updated Chef version (2015/05/29 and 2016/04/05)=<br /> <br /> Updated list of useful parameters in adapt.inp<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Now, mixed mesh can be refined uniformly either everywhere including the BL, or only outside the BL (see parameter splitAllLayerEdges below). Other adaptation strategies based on local error indicators are being developed in Chef and will be complimentary to the existing strategies available in phParAdapt-Simmetrix. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested and validated in Chef. <br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * '''SplitAllLayerEdges''': This parameter is only applicable to mixed meshes during uniform refinement. If set to 1, uniform refinement of all mesh edges including the edges along normal growth curve (wedges) of the BL. If set to 0, uniform refinement except for edges along normal growth curve (tets only). For all tet meshes, this parameter is ignored and all the tets get split.<br /> <br /> * '''Snap''': If set to 1 during a uniform refinement, Chef will attempt snapping to model surface. Use with caution and check the resulting mesh: if the input mixed mesh is too coarse, snapping can be partially ignored. Non valid meshes have also sometimes been observed (phasta crash).<br /> <br /> * '''splitFactor''':, ratio of the number of output parts with the number of input parts. If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * '''partitionMethod''': graph or zrib (for Zoltan Recursive Inertia Bisection) are both available options so far. Graph should be preferred choice for now.<br /> <br /> * '''LocalPtn''': 0 for global partitioning, 1 for local partitioning. Global partitioning coupled with graph as the partition method requires a lot of memory and time and is limited to coarse meshes with a small number of mesh parts. Global RIB is more robust but can lead to larger vertex imbalance. If starting from a well balanced mesh with few or no disconnected parts, local graph is the recommended choice so far.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''elementImbalance''': target element imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''vertexImbalance''': target vertex imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> * '''filterMatches''': If set to 0, the matching between periodic entities are imposed from the mesh file regardless of the periodic faces defined in the attributes that could potentially differ. When enabled, this code derives the period associations between all model entities based on the &quot;periodic slave&quot; attribute set in the attribute file. Note that the mesh must be periodic to support this feature. This can be useful for instance for a three-way periodic channel mesh that can support both 1-way and 3-way periodic attributes without the need to build two distinct meshes for that purpose.<br /> <br /> * '''axisymmetry''': When enabled, this parameter supports axisymmetric periodic mesh and attributes for instance for an annular flow.<br /> <br /> = Words of caution=<br /> When working with large meshes occasionally the executable will crash by no fault of the user. This has happened in the past when (adapt.inp set parameter: ensa_ndof)*(number of nodes in the mesh)&gt;(maximum value of the 4-byte integer 2^31). This was debugged by using the addr2line command. Information about this command can be found at https://fluid.colorado.edu/wiki/index.php/Debugging.<br /> <br /> [[Category:Chef]]</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Chef/Mesh_Partitioning&diff=1934 2023-02-03T20:35:56Z

<p>Prte0550: /* Documentation */</p> <hr /> <div>This webpage is first inspired from a tutorial provided to Igor and his team at NCSU in order to set up two phase flow test cases on a local cluster named Firebird at NCSU and Cetus/Mira at ALCF. At this time, this tutorial includes copy-paste materials from old emails. <br /> <br /> The code has evolved since then! If you scroll down, you will also find critical updates since the first tutorial was written. Please do not ignore them or there is 100% chance your mesh partitioning/refinement will fail.<br /> <br /> Please update this page for our viz nodes when you get a chance. <br /> <br /> Thanks, <br /> <br /> - Michel<br /> <br /> <br /> <br /> <br /> <br /> Here is a tutorial about how to respectively partition the initial mesh and generate the phasta files on firebird (and other platforms including Cetus/Mira) using Chef. This tutorial is rather long but should include everything you need.<br /> The testcase to demonstrate the workflow is the familiar 3-way subchannel flow. The root path of this test case is /sgidata2/mrasquin/Models/subchannel. The parasolid model is located in /sgidata2/mrasquin/Models/subchannel/convertParasolid2ParasolidNative/geomFromSimmodeler_nat.xmt_txt.<br /> The workflow that describes how to use Chef is now explained in the next sections.<br /> <br /> = Documentation =<br /> General documentation of Chef that may be used in supplement to this page is available [https://github.com/SCOREC/core/wiki/chef-partition-control|here]<br /> <br /> = Initial tutorial =<br /> <br /> == Env variables==<br /> <br /> All the subsequent tools need<br /> * The fresh version of openmpi I built on firebird<br /> * The latest Simmetrix library I installed in /Install on firebird.<br /> <br /> To update your paths, source the following file:<br /> &lt;code&gt;/Install/SCOREC.develop/envLinux2014.sh&lt;/code&gt;.<br /> <br /> The env variables defined or updated in this env script include PATH and LD_LIBRARY_PATH. What is defined in this script should prevail on your settings but I strongly suggest removing any redundancy that you may have, for instance, in your .basrc. Note that I actually source this env file directly in my .bashrc so that I do not have to do it manually every time I log in to firebird. When you source it, it will also print the version of gcc, openmpi and simmodsuite lib that are set up.<br /> <br /> == BLMesherParallel ==<br /> <br /> Note that Simmetrix only supports matched faces for single part mesh so that the mesh must be built with one core. However the initial mesh must already include some information related to the partitioning, even if the mesh only includes a single part for format reasons. This additional information about the partitioning is required for conversion of the mesh file from the Simmetrix format to the SCOREC MDS format that Chef can read.<br /> <br /> The initial mesh for the 3-way subchannel was built in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0&lt;/code&gt;. Check the script named &lt;code&gt;runBLMesherParallel.sh&lt;/code&gt; in this directory.<br /> <br /> Running &lt;code&gt;./runBLMesherParallel.sh&lt;/code&gt; with no arguments will tell you the usage, that is:<br /> Usage: ./runBLMesherParallel.sh &lt;X&gt; &lt;Y&gt; &lt;Z&gt;<br /> <br /> The arguments are as follows.<br /> * &lt;X&gt; (geometric model) should be the parasolid model geomFromSimmodeler_nat.xmt_txt.<br /> * &lt;Y&gt; (attribute file) should be BLattr.inp.<br /> * &lt;Z&gt; (number of processors) should be 1 here since we need to generate a single part mesh using a single core.<br /> <br /> The BLattr.inp input file is the same as the one read by the old serial version of BLMesher. But BLMesherParallel can do whatever the old version of BLMesher can do. In addition, if your test case does not include any matched face, you may try to mesh in parallel by specifying &lt;Z&gt; to be larger than 1. However, some meshing features are available only when BLMesherParallel is used with a single core so it is always important to check the resulting mesh.<br /> <br /> BLMesherParallel outputs the following files.<br /> <br /> * &lt;code&gt;mesh.sms&lt;/code&gt; --- The resulting mesh is stored in a directory named mesh.sms, which is a parameter hardcoded in the runBLMesherParallel.sh script.<br /> * &lt;code&gt;BLMesher.log&lt;/code&gt; --- The log from BLMesherParallel is saved in BMesher.log, whereas the Simmetrix log is saved in mesh.log. Both filenames are also hardcoded in the script.<br /> <br /> I also mentioned in previous discussions that Simmetrix has developed its own model format called geomsim. However, the boundary layer collapses near matched faces with this model format, which is not the case when we use the parasolid format. This issue has been reported to Simmetrix but until they can provide a fix, we are forced to start with the parasolid format when our test cases include matched faces.<br /> <br /> == Mesh conversion==<br /> <br /> Chef can read only the MDS format developed at SCOREC. Therefore, the Simmetrix mesh mush first be converted to this format.<br /> <br /> This operation was carried out for the 3-way channel in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/simMeshToMdsMesh&lt;/code&gt;. Simply run the script &lt;code&gt;./simMeshToMdsMesh.sh&lt;/code&gt;, which executes the &quot;convert&quot; executable. In the script, you can see that the convert executable reads 3 arguments:<br /> # The '''input parasolid model''' named named geom.xmt_txt, which points to geomFromSimmodeler_nat.x_t. Note that convert expects an .xmt_txt extension (or .smd extension for the complete geomsim format).<br /> # The '''input Simmetrix mesh''' named here parts.sms (for historical reason but can be renamed).<br /> # The '''name of the output mds mesh directory''', which is mdsMesh_bz2 here. Note that this name is prepended by &quot;bz2:&quot;, which means that the output mds mesh file is compressed using bzip2. &quot;bz2:&quot; will not be part of the name of the output directory. If you do not specify &quot;bz2:&quot;, the mds mesh file will be saved in ascii format, which is a waste of space so I suggest to always prepend your directory name by &quot;bz2:&quot;. This will also apply later to the output mesh directory generated by Chef (see below).<br /> <br /> Note that convert needs to run with a number of processes (-np ##) equal to the number of input parts in the Simmetrix mesh. For cases that include match faces, the Simmetrix mesh must include only one part, which is the reason why convert runs here with -np 1. But in other circumstances, convert can run in parallel if the Simmetrix mesh has already been partitioned in n parts with n&gt;1 (for instance mesh generated in parallel with BLMesherParallel and/or partitioned with phParAdapt-Simmetrix).<br /> <br /> == Boundary and initial conditions (spj file)==<br /> <br /> Before running Chef for mesh operations such as uniform refinement, tetrahedronization and partitioning, we need to define the BCs and ICs for the generation of the phasta files. These BCs and ICs are defined in an spj file, which is in ASCII to facilitate scripting of BCs/ICs. Most of the attributes you are familiar with from the Simmodeler GUI can be specified in the spj file.<br /> <br /> For the 3-way channel flow, see the spj file located in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Simplified_SPJ_file/geom.spj&lt;/code&gt;. Each line corresponds to one attribute that applies to one face.<br /> <br /> The structure of the spj file is:<br /> # Optional comments anywhere preceded by the pound symbol (#).<br /> # For each boundary or initial condition a line as follows:<br /> &lt;attribute_name&gt;: &lt;face_id&gt; &lt;dimension&gt; &lt;attribute list&gt;<br /> <br /> Note the following.<br /> * &lt;code&gt;&lt;dimension&gt;&lt;/code&gt;: 2 for a face attribute in 2D, 3 for the initial conditions that applies to the 3D domain. 1D and 0D attributes are also allowed for lines and vertices if needed.<br /> * &lt;code&gt;&lt;attribute list&gt;&lt;/code&gt;: typically magnitude and direction if this applies&gt;.<br /> <br /> Syntax is strict.<br /> * No empty line. Each line should be either a comment which starts with the # character, or an attribute.<br /> * There must be one single space after the semicolon character.<br /> * There must be one single space between any numbers.<br /> <br /> In this example, a zero &quot;traction vector&quot; attribute is specified on the periodic faces parallel to the length of the channel. It is wrong to specify such an attribute on these periodic faces for a 3-way channel, but this was inherited from the 1-way periodic channel where these faces were slip walls instead of periodic faces. I will try to update my test cases in the future. But because we have now continuous integration tools that run every night to verify the Chef code, I will need to update all the cases if I modify the spj file now. So double check the attributes that you need for this model and consider the existing spj file as a source of inspiration rather than the correct spj file for production runs.<br /> <br /> == Chef ==<br /> <br /> A few rules must be followed to run Chef.<br /> <br /> First, the number of mpi processes must be equal to the number of input parts (''this has changed in the newest version of Chef, as described below'').<br /> <br /> Second, Chef is threaded with openmp and the total number of output parts after partitioning should be at most equal to the total number of available hardware threads of your machine/allocation. On BGQ, there are 4 hardware threads per core. On Linux platform such as firebird, the number of hardware threads corresponds to the number of available cores. That said, we have observed that if the number of output parts is equal to the total number of available hardware threads, Chef can hang. Therefore, it is safer to limit the number of output parts to a lower number than the number of available hardware threads. Therefore, on firebird, we should not try to partition a mesh to more than 16 parts.<br /> <br /> The next mesh operations will have to take place on Tukey and Cetus/Mira.<br /> <br /> The first example of a partitioning with Chef can be found in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch&lt;/code&gt;. With my naming convention, &lt;code&gt;4-1-Chef-PartLocal-Scratch&lt;/code&gt; can be decomposed as follows:<br /> * The first number (4) corresponds to the number of output parts<br /> * The second number (1) correspond to the number of input parts<br /> * &quot;Chef&quot; means this mesh was treated with this program (in opposition to phParAdapt, phTest, etc which are previous executables that we used for similar purpose).<br /> * &quot;PartLocal&quot; means the mesh is partitioned locally.<br /> * &quot;Scratch&quot; means that the initial solution in the resulting phasta files is generated entirely from the spj file defined in a previous section of this tutorial. That is, we are starting a simulation &quot;from scratch,&quot; using the spj file's initial conditions as opposed to a solution migrated from a previous run.<br /> <br /> In summary, Chef was used in this directory to partition a single part mesh into 4 parts and the solution in the phasta files was generated directly from scratch using the spj file.<br /> <br /> === Chef's input files ===<br /> <br /> The script to run Chef is named runChef.sh in this directory and simply call the executable. Chef reads all it needs from two input files called numstart.dat and adapt.inp.<br /> <br /> ==== numstart.dat ====<br /> <br /> Instead of building the initial solution from scratch using the initial conditions defined in the spj file, the user can migrate an existing solution stored in a set of restart files that were saved from a previous phasta simulation. Numstart.dat contains the time step stamp of the input restart files to read in order to migrate a solution.<br /> <br /> ==== adapt.inp ====<br /> <br /> This input file contains all the other parameters Chef expects. Note that many of these parameters have been inherited from the old phParAdapt, and are currently obsolete or unused. In what follows, all the parameters available in adapt.inp are listed and the critical parameters are in bold. Any line that starts with # is ignored.<br /> <br /> * '''globalP''': obsolete/unused.<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Note that adaptation only works with a mixed mesh (with wedges in the BL) and not with an all-tet mesh. Tetrahedronization should therefore take place after uniform refinement. Right now, the mixed mesh gets uniformly refined everywhere including the BL but it is possible to refine uniformly outside the BL only with some light modifications of the code. In the future, we hope to have other adaptation strategies in place in Chef based on local error indicator. If interested in these strategy, then phParAdapt-Simmetrix must be used. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * rRead: obsolete/unused.<br /> <br /> * rStart: obsolete/unused.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested in Chef. If interested in other more sophisticated adaptation strategies, phParAdapt-Simmetrix must be used for now.<br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * Periodic: obsolete. Periodicity in the mesh and in the solution is not treated automatically as long as i) the mesh built with BLMesher is periodic (i.e. location of the mesh vertices on periodic faces in the same) and ii) the spj file contains the correct &quot;periodic slave&quot; attributes.<br /> <br /> * prCD: obsolete/unused.<br /> <br /> * timing: obsolete/unused.<br /> <br /> * outputFormat: obsolete. Phasta files are saved by default in binary format.<br /> <br /> * internalBCNodes: obsolete/unused.<br /> <br /> * WRITEASC: obsolete/unused.<br /> <br /> * phastaIO: obsolete/unused.<br /> <br /> * '''numTotParts''': Final number of parts. If numTotParts is larger than the number of Chef processes which is equal to the number of input parts, the mesh will be partitioned.<br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * isReorder: obsolete/unused. Reordering for better cache performance is now applied by default to both the phasta files and mesh files.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * numSplit: obsolete/unused.<br /> <br /> * '''LocalPtn''': local partitioning if set to 1, set global partitioning if set to 0. Currently, only local partitioning is implemented in Chef and has been shown to be sufficient so far.<br /> <br /> * '''RecursivePtn''': should always be set to 1. In the past, this parameter allowed recursive partitioning steps in phParAdapt. The code will stop or crash if this parameter is not 1.<br /> <br /> * RecursivePtnStep: obsolete/unused.<br /> <br /> * '''partitionMethod''': Currently, the GRAPH method for local partitioning is hard coded in one of the Chef routine.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> The second example of a partitioning with Chef can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-Tet-PartLocal-SolMgr. For this case, based on the naming convention of 8-4-Chef-Tet-PartLocal-SolMgr (and the parameters specified in adapt.inp and numstart.dat),<br /> * the number of output parts requested is 8, <br /> * the number of input parts is 4 (note &quot;-np 4&quot; in the runChef.sh script),<br /> * the input mixed mesh is first tetrahedronized before being partitioned. <br /> * the solution in the resulting phasta files is migrated from the previous Chef run. <br /> Note that the spj file is different for this second example and the initial conditions have been commented out in order not to overwrite the solution that is migrated from the previous Chef run.<br /> <br /> The third and final example can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-UR2-Tet-PartLocal-SolMgr. In this directory 8-4-Chef-UR2-Tet-PartLocal-SolMgr, Chef <br /> * reads a four part mesh, <br /> * applies a double recursive uniform refinement, <br /> * tetrahedronize the resulting mixed mesh that has been uniformly refined twice, <br /> * partition the resulting 4 part all-tet uniformly refined mesh into 8 parts,<br /> * migrate and interpolate the solution read from existing restart files coming from the first example.<br /> <br /> As a final comment, note that the restart files are always read directly from a procs_case directory. However, when the number of output restart files exceeds 2048, the restart files are then saved in subdirectories of the root procs_case directory in order to reduce file contention, in the same (but still different) way as what you have implemented at some point in your version of phasta. The best strategy would be to write phasta files using mpi_io for instance so that we can store more than one part in a single file and avoid large number of phasta files.<br /> <br /> For further partitioning on BG/Q machines a conversion to the native Parasolid model is required. The tool is located in: /Install/SCOREC.develop/scorec/test/cadToSim/cadToSim <br /> and should be run from [Case directory]/convertParasolid2ParasolidNative/ on firebird.<br /> <br /> = Updated Chef version (2015/03/26)=<br /> <br /> <br /> 1) MPI implementation<br /> <br /> A new version of chef has been implemented and does not rely on threads any more.<br /> Instead, it is now based on a pure MPI implementation. <br /> That means that there is an important change in how chef is called at runtime.<br /> <br /> With the previous threaded version, the number of MPI processes had to be equal to the number of input parts. <br /> Chef was then in charge of starting a number of threads equal to the number of output parts, which was automatic.<br /> <br /> Since the pure MPI version of chef does not start thread any more, it now requires a number of MPI processes equal to the final number of output parts, and not input parts.<br /> <br /> <br /> 2) adapt.inp<br /> <br /> In the new version of chef, &quot;numTotParts&quot; in adapt.inp (which was used to specify the final number of output parts) has been replaced by &quot;splitFactor&quot;, which corresponds to the ratio of the number of output parts with the number of input parts. <br /> If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> If you set this parameter to 2, each part of your input mesh will be split in 2 new sub-parts, etc<br /> Keep in mind that the number of MPI processes that needs to be requested for chef must therefore be equal to (number of input parts times) * (splitFactor).<br /> <br /> I have also removed the obsolete parameter in adapt.inp and saved a representative version of this file in /projects/tools/SCOREC.develop/runscripts/adapt.inp<br /> <br /> <br /> 3) Paths<br /> <br /> I have updated chef on the VIz nodes, Mira and Tukey so that it only relies on the more robust pure MPI implementation.<br /> <br /> On the viz nodes, use /projects/tools/SCOREC.develop/build-chefMPI-GNU-*/test/chef<br /> For simplicity, this is the default version of the master branch coming directly from our github repository.<br /> <br /> On Tukey, use /home/mrasquin/SCOREC.develop/build-tukey-GNU-OptG-c2c360bc-mpi-*<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35-noblsnap means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is off during uniform refinement (UR).<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35 means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is on during UR.<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol33 means that the target imbalance for both the vtx and elem is 3%, and BL snapping is on during UR.<br /> Note that these versions have been slightly modified w.r.t. the master branch. In particular, the imbalance target is not a parameter yet. Also, in Parma, HPS (Heavy Part Splitting) and FixDisconnectedPart are not called here because the latest version of the diffusion algorithm with improved selection of (i) target parts for element exchange and (ii) elements.<br /> <br /> On Mira, use /home/mrasquin/SCOREC.develop/build-XL-OptG-c2c360bc-mpi-*<br /> Similar comments applies to build-XL-OptG-c2c360bc-mpi-tol33, build-XL-OptG-c2c360bc-mpi-tol35 and build-XL-OptG-c2c360bc-mpi-tol35-noblsnap.<br /> <br /> Note that BL snapping is not called for a repartitioning of the mesh. It can only play a role during uniform refinement.<br /> Consequently, if you do not request a UR in adapt.inp, then build-*-tol35 and build-*-tol35-noblsnap will behave the same way.<br /> <br /> In case you are wondering what the weird numbers are in the name of the build directory, this comes from the git log hash, which is a unique number associated with a git commit (easier to couple an executable with a version of the code).<br /> <br /> <br /> = Updated Chef version (2015/05/29 and 2016/04/05)=<br /> <br /> Updated list of useful parameters in adapt.inp<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Now, mixed mesh can be refined uniformly either everywhere including the BL, or only outside the BL (see parameter splitAllLayerEdges below). Other adaptation strategies based on local error indicators are being developed in Chef and will be complimentary to the existing strategies available in phParAdapt-Simmetrix. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested and validated in Chef. <br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * '''SplitAllLayerEdges''': This parameter is only applicable to mixed meshes during uniform refinement. If set to 1, uniform refinement of all mesh edges including the edges along normal growth curve (wedges) of the BL. If set to 0, uniform refinement except for edges along normal growth curve (tets only). For all tet meshes, this parameter is ignored and all the tets get split.<br /> <br /> * '''Snap''': If set to 1 during a uniform refinement, Chef will attempt snapping to model surface. Use with caution and check the resulting mesh: if the input mixed mesh is too coarse, snapping can be partially ignored. Non valid meshes have also sometimes been observed (phasta crash).<br /> <br /> * '''splitFactor''':, ratio of the number of output parts with the number of input parts. If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * '''partitionMethod''': graph or zrib (for Zoltan Recursive Inertia Bisection) are both available options so far. Graph should be preferred choice for now.<br /> <br /> * '''LocalPtn''': 0 for global partitioning, 1 for local partitioning. Global partitioning coupled with graph as the partition method requires a lot of memory and time and is limited to coarse meshes with a small number of mesh parts. Global RIB is more robust but can lead to larger vertex imbalance. If starting from a well balanced mesh with few or no disconnected parts, local graph is the recommended choice so far.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''elementImbalance''': target element imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''vertexImbalance''': target vertex imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> * '''filterMatches''': If set to 0, the matching between periodic entities are imposed from the mesh file regardless of the periodic faces defined in the attributes that could potentially differ. When enabled, this code derives the period associations between all model entities based on the &quot;periodic slave&quot; attribute set in the attribute file. Note that the mesh must be periodic to support this feature. This can be useful for instance for a three-way periodic channel mesh that can support both 1-way and 3-way periodic attributes without the need to build two distinct meshes for that purpose.<br /> <br /> * '''axisymmetry''': When enabled, this parameter supports axisymmetric periodic mesh and attributes for instance for an annular flow.<br /> <br /> = Words of caution=<br /> When working with large meshes occasionally the executable will crash by no fault of the user. This has happened in the past when (adapt.inp set parameter: ensa_ndof)*(number of nodes in the mesh)&gt;(maximum value of the 4-byte integer 2^31). This was debugged by using the addr2line command. Information about this command can be found at https://fluid.colorado.edu/wiki/index.php/Debugging.<br /> <br /> [[Category:Chef]]</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Chef/Mesh_Partitioning&diff=1933 2023-02-03T19:11:01Z

<p>Prte0550: Added link to documentation</p> <hr /> <div>This webpage is first inspired from a tutorial provided to Igor and his team at NCSU in order to set up two phase flow test cases on a local cluster named Firebird at NCSU and Cetus/Mira at ALCF. At this time, this tutorial includes copy-paste materials from old emails. <br /> <br /> The code has evolved since then! If you scroll down, you will also find critical updates since the first tutorial was written. Please do not ignore them or there is 100% chance your mesh partitioning/refinement will fail.<br /> <br /> Please update this page for our viz nodes when you get a chance. <br /> <br /> Thanks, <br /> <br /> - Michel<br /> <br /> <br /> <br /> <br /> <br /> Here is a tutorial about how to respectively partition the initial mesh and generate the phasta files on firebird (and other platforms including Cetus/Mira) using Chef. This tutorial is rather long but should include everything you need.<br /> The testcase to demonstrate the workflow is the familiar 3-way subchannel flow. The root path of this test case is /sgidata2/mrasquin/Models/subchannel. The parasolid model is located in /sgidata2/mrasquin/Models/subchannel/convertParasolid2ParasolidNative/geomFromSimmodeler_nat.xmt_txt.<br /> The workflow that describes how to use Chef is now explained in the next sections.<br /> <br /> = Documentation =<br /> General documentation of Chef that may be used in supplement to this page is available [[https://github.com/SCOREC/core/wiki/chef-partition-control|here]]<br /> <br /> <br /> = Initial tutorial =<br /> <br /> == Env variables==<br /> <br /> All the subsequent tools need<br /> * The fresh version of openmpi I built on firebird<br /> * The latest Simmetrix library I installed in /Install on firebird.<br /> <br /> To update your paths, source the following file:<br /> &lt;code&gt;/Install/SCOREC.develop/envLinux2014.sh&lt;/code&gt;.<br /> <br /> The env variables defined or updated in this env script include PATH and LD_LIBRARY_PATH. What is defined in this script should prevail on your settings but I strongly suggest removing any redundancy that you may have, for instance, in your .basrc. Note that I actually source this env file directly in my .bashrc so that I do not have to do it manually every time I log in to firebird. When you source it, it will also print the version of gcc, openmpi and simmodsuite lib that are set up.<br /> <br /> == BLMesherParallel ==<br /> <br /> Note that Simmetrix only supports matched faces for single part mesh so that the mesh must be built with one core. However the initial mesh must already include some information related to the partitioning, even if the mesh only includes a single part for format reasons. This additional information about the partitioning is required for conversion of the mesh file from the Simmetrix format to the SCOREC MDS format that Chef can read.<br /> <br /> The initial mesh for the 3-way subchannel was built in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0&lt;/code&gt;. Check the script named &lt;code&gt;runBLMesherParallel.sh&lt;/code&gt; in this directory.<br /> <br /> Running &lt;code&gt;./runBLMesherParallel.sh&lt;/code&gt; with no arguments will tell you the usage, that is:<br /> Usage: ./runBLMesherParallel.sh &lt;X&gt; &lt;Y&gt; &lt;Z&gt;<br /> <br /> The arguments are as follows.<br /> * &lt;X&gt; (geometric model) should be the parasolid model geomFromSimmodeler_nat.xmt_txt.<br /> * &lt;Y&gt; (attribute file) should be BLattr.inp.<br /> * &lt;Z&gt; (number of processors) should be 1 here since we need to generate a single part mesh using a single core.<br /> <br /> The BLattr.inp input file is the same as the one read by the old serial version of BLMesher. But BLMesherParallel can do whatever the old version of BLMesher can do. In addition, if your test case does not include any matched face, you may try to mesh in parallel by specifying &lt;Z&gt; to be larger than 1. However, some meshing features are available only when BLMesherParallel is used with a single core so it is always important to check the resulting mesh.<br /> <br /> BLMesherParallel outputs the following files.<br /> <br /> * &lt;code&gt;mesh.sms&lt;/code&gt; --- The resulting mesh is stored in a directory named mesh.sms, which is a parameter hardcoded in the runBLMesherParallel.sh script.<br /> * &lt;code&gt;BLMesher.log&lt;/code&gt; --- The log from BLMesherParallel is saved in BMesher.log, whereas the Simmetrix log is saved in mesh.log. Both filenames are also hardcoded in the script.<br /> <br /> I also mentioned in previous discussions that Simmetrix has developed its own model format called geomsim. However, the boundary layer collapses near matched faces with this model format, which is not the case when we use the parasolid format. This issue has been reported to Simmetrix but until they can provide a fix, we are forced to start with the parasolid format when our test cases include matched faces.<br /> <br /> == Mesh conversion==<br /> <br /> Chef can read only the MDS format developed at SCOREC. Therefore, the Simmetrix mesh mush first be converted to this format.<br /> <br /> This operation was carried out for the 3-way channel in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/simMeshToMdsMesh&lt;/code&gt;. Simply run the script &lt;code&gt;./simMeshToMdsMesh.sh&lt;/code&gt;, which executes the &quot;convert&quot; executable. In the script, you can see that the convert executable reads 3 arguments:<br /> # The '''input parasolid model''' named named geom.xmt_txt, which points to geomFromSimmodeler_nat.x_t. Note that convert expects an .xmt_txt extension (or .smd extension for the complete geomsim format).<br /> # The '''input Simmetrix mesh''' named here parts.sms (for historical reason but can be renamed).<br /> # The '''name of the output mds mesh directory''', which is mdsMesh_bz2 here. Note that this name is prepended by &quot;bz2:&quot;, which means that the output mds mesh file is compressed using bzip2. &quot;bz2:&quot; will not be part of the name of the output directory. If you do not specify &quot;bz2:&quot;, the mds mesh file will be saved in ascii format, which is a waste of space so I suggest to always prepend your directory name by &quot;bz2:&quot;. This will also apply later to the output mesh directory generated by Chef (see below).<br /> <br /> Note that convert needs to run with a number of processes (-np ##) equal to the number of input parts in the Simmetrix mesh. For cases that include match faces, the Simmetrix mesh must include only one part, which is the reason why convert runs here with -np 1. But in other circumstances, convert can run in parallel if the Simmetrix mesh has already been partitioned in n parts with n&gt;1 (for instance mesh generated in parallel with BLMesherParallel and/or partitioned with phParAdapt-Simmetrix).<br /> <br /> == Boundary and initial conditions (spj file)==<br /> <br /> Before running Chef for mesh operations such as uniform refinement, tetrahedronization and partitioning, we need to define the BCs and ICs for the generation of the phasta files. These BCs and ICs are defined in an spj file, which is in ASCII to facilitate scripting of BCs/ICs. Most of the attributes you are familiar with from the Simmodeler GUI can be specified in the spj file.<br /> <br /> For the 3-way channel flow, see the spj file located in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Simplified_SPJ_file/geom.spj&lt;/code&gt;. Each line corresponds to one attribute that applies to one face.<br /> <br /> The structure of the spj file is:<br /> # Optional comments anywhere preceded by the pound symbol (#).<br /> # For each boundary or initial condition a line as follows:<br /> &lt;attribute_name&gt;: &lt;face_id&gt; &lt;dimension&gt; &lt;attribute list&gt;<br /> <br /> Note the following.<br /> * &lt;code&gt;&lt;dimension&gt;&lt;/code&gt;: 2 for a face attribute in 2D, 3 for the initial conditions that applies to the 3D domain. 1D and 0D attributes are also allowed for lines and vertices if needed.<br /> * &lt;code&gt;&lt;attribute list&gt;&lt;/code&gt;: typically magnitude and direction if this applies&gt;.<br /> <br /> Syntax is strict.<br /> * No empty line. Each line should be either a comment which starts with the # character, or an attribute.<br /> * There must be one single space after the semicolon character.<br /> * There must be one single space between any numbers.<br /> <br /> In this example, a zero &quot;traction vector&quot; attribute is specified on the periodic faces parallel to the length of the channel. It is wrong to specify such an attribute on these periodic faces for a 3-way channel, but this was inherited from the 1-way periodic channel where these faces were slip walls instead of periodic faces. I will try to update my test cases in the future. But because we have now continuous integration tools that run every night to verify the Chef code, I will need to update all the cases if I modify the spj file now. So double check the attributes that you need for this model and consider the existing spj file as a source of inspiration rather than the correct spj file for production runs.<br /> <br /> == Chef ==<br /> <br /> A few rules must be followed to run Chef.<br /> <br /> First, the number of mpi processes must be equal to the number of input parts (''this has changed in the newest version of Chef, as described below'').<br /> <br /> Second, Chef is threaded with openmp and the total number of output parts after partitioning should be at most equal to the total number of available hardware threads of your machine/allocation. On BGQ, there are 4 hardware threads per core. On Linux platform such as firebird, the number of hardware threads corresponds to the number of available cores. That said, we have observed that if the number of output parts is equal to the total number of available hardware threads, Chef can hang. Therefore, it is safer to limit the number of output parts to a lower number than the number of available hardware threads. Therefore, on firebird, we should not try to partition a mesh to more than 16 parts.<br /> <br /> The next mesh operations will have to take place on Tukey and Cetus/Mira.<br /> <br /> The first example of a partitioning with Chef can be found in &lt;code&gt;/sgidata2/mrasquin/Models/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140927/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch&lt;/code&gt;. With my naming convention, &lt;code&gt;4-1-Chef-PartLocal-Scratch&lt;/code&gt; can be decomposed as follows:<br /> * The first number (4) corresponds to the number of output parts<br /> * The second number (1) correspond to the number of input parts<br /> * &quot;Chef&quot; means this mesh was treated with this program (in opposition to phParAdapt, phTest, etc which are previous executables that we used for similar purpose).<br /> * &quot;PartLocal&quot; means the mesh is partitioned locally.<br /> * &quot;Scratch&quot; means that the initial solution in the resulting phasta files is generated entirely from the spj file defined in a previous section of this tutorial. That is, we are starting a simulation &quot;from scratch,&quot; using the spj file's initial conditions as opposed to a solution migrated from a previous run.<br /> <br /> In summary, Chef was used in this directory to partition a single part mesh into 4 parts and the solution in the phasta files was generated directly from scratch using the spj file.<br /> <br /> === Chef's input files ===<br /> <br /> The script to run Chef is named runChef.sh in this directory and simply call the executable. Chef reads all it needs from two input files called numstart.dat and adapt.inp.<br /> <br /> ==== numstart.dat ====<br /> <br /> Instead of building the initial solution from scratch using the initial conditions defined in the spj file, the user can migrate an existing solution stored in a set of restart files that were saved from a previous phasta simulation. Numstart.dat contains the time step stamp of the input restart files to read in order to migrate a solution.<br /> <br /> ==== adapt.inp ====<br /> <br /> This input file contains all the other parameters Chef expects. Note that many of these parameters have been inherited from the old phParAdapt, and are currently obsolete or unused. In what follows, all the parameters available in adapt.inp are listed and the critical parameters are in bold. Any line that starts with # is ignored.<br /> <br /> * '''globalP''': obsolete/unused.<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Note that adaptation only works with a mixed mesh (with wedges in the BL) and not with an all-tet mesh. Tetrahedronization should therefore take place after uniform refinement. Right now, the mixed mesh gets uniformly refined everywhere including the BL but it is possible to refine uniformly outside the BL only with some light modifications of the code. In the future, we hope to have other adaptation strategies in place in Chef based on local error indicator. If interested in these strategy, then phParAdapt-Simmetrix must be used. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * rRead: obsolete/unused.<br /> <br /> * rStart: obsolete/unused.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested in Chef. If interested in other more sophisticated adaptation strategies, phParAdapt-Simmetrix must be used for now.<br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * Periodic: obsolete. Periodicity in the mesh and in the solution is not treated automatically as long as i) the mesh built with BLMesher is periodic (i.e. location of the mesh vertices on periodic faces in the same) and ii) the spj file contains the correct &quot;periodic slave&quot; attributes.<br /> <br /> * prCD: obsolete/unused.<br /> <br /> * timing: obsolete/unused.<br /> <br /> * outputFormat: obsolete. Phasta files are saved by default in binary format.<br /> <br /> * internalBCNodes: obsolete/unused.<br /> <br /> * WRITEASC: obsolete/unused.<br /> <br /> * phastaIO: obsolete/unused.<br /> <br /> * '''numTotParts''': Final number of parts. If numTotParts is larger than the number of Chef processes which is equal to the number of input parts, the mesh will be partitioned.<br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * isReorder: obsolete/unused. Reordering for better cache performance is now applied by default to both the phasta files and mesh files.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * numSplit: obsolete/unused.<br /> <br /> * '''LocalPtn''': local partitioning if set to 1, set global partitioning if set to 0. Currently, only local partitioning is implemented in Chef and has been shown to be sufficient so far.<br /> <br /> * '''RecursivePtn''': should always be set to 1. In the past, this parameter allowed recursive partitioning steps in phParAdapt. The code will stop or crash if this parameter is not 1.<br /> <br /> * RecursivePtnStep: obsolete/unused.<br /> <br /> * '''partitionMethod''': Currently, the GRAPH method for local partitioning is hard coded in one of the Chef routine.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> The second example of a partitioning with Chef can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-Tet-PartLocal-SolMgr. For this case, based on the naming convention of 8-4-Chef-Tet-PartLocal-SolMgr (and the parameters specified in adapt.inp and numstart.dat),<br /> * the number of output parts requested is 8, <br /> * the number of input parts is 4 (note &quot;-np 4&quot; in the runChef.sh script),<br /> * the input mixed mesh is first tetrahedronized before being partitioned. <br /> * the solution in the resulting phasta files is migrated from the previous Chef run. <br /> Note that the spj file is different for this second example and the initial conditions have been commented out in order not to overwrite the solution that is migrated from the previous Chef run.<br /> <br /> The third and final example can be found in /sgidata2/mrasquin/Models/TwoPhase/subchannel/subchannel_3way/Mixed-Parallel1-parasolid-9.0-140906/2-A0/1PFPP-phPA/4-1-Chef-PartLocal-Scratch/8-4-Chef-UR2-Tet-PartLocal-SolMgr. In this directory 8-4-Chef-UR2-Tet-PartLocal-SolMgr, Chef <br /> * reads a four part mesh, <br /> * applies a double recursive uniform refinement, <br /> * tetrahedronize the resulting mixed mesh that has been uniformly refined twice, <br /> * partition the resulting 4 part all-tet uniformly refined mesh into 8 parts,<br /> * migrate and interpolate the solution read from existing restart files coming from the first example.<br /> <br /> As a final comment, note that the restart files are always read directly from a procs_case directory. However, when the number of output restart files exceeds 2048, the restart files are then saved in subdirectories of the root procs_case directory in order to reduce file contention, in the same (but still different) way as what you have implemented at some point in your version of phasta. The best strategy would be to write phasta files using mpi_io for instance so that we can store more than one part in a single file and avoid large number of phasta files.<br /> <br /> For further partitioning on BG/Q machines a conversion to the native Parasolid model is required. The tool is located in: /Install/SCOREC.develop/scorec/test/cadToSim/cadToSim <br /> and should be run from [Case directory]/convertParasolid2ParasolidNative/ on firebird.<br /> <br /> = Updated Chef version (2015/03/26)=<br /> <br /> <br /> 1) MPI implementation<br /> <br /> A new version of chef has been implemented and does not rely on threads any more.<br /> Instead, it is now based on a pure MPI implementation. <br /> That means that there is an important change in how chef is called at runtime.<br /> <br /> With the previous threaded version, the number of MPI processes had to be equal to the number of input parts. <br /> Chef was then in charge of starting a number of threads equal to the number of output parts, which was automatic.<br /> <br /> Since the pure MPI version of chef does not start thread any more, it now requires a number of MPI processes equal to the final number of output parts, and not input parts.<br /> <br /> <br /> 2) adapt.inp<br /> <br /> In the new version of chef, &quot;numTotParts&quot; in adapt.inp (which was used to specify the final number of output parts) has been replaced by &quot;splitFactor&quot;, which corresponds to the ratio of the number of output parts with the number of input parts. <br /> If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> If you set this parameter to 2, each part of your input mesh will be split in 2 new sub-parts, etc<br /> Keep in mind that the number of MPI processes that needs to be requested for chef must therefore be equal to (number of input parts times) * (splitFactor).<br /> <br /> I have also removed the obsolete parameter in adapt.inp and saved a representative version of this file in /projects/tools/SCOREC.develop/runscripts/adapt.inp<br /> <br /> <br /> 3) Paths<br /> <br /> I have updated chef on the VIz nodes, Mira and Tukey so that it only relies on the more robust pure MPI implementation.<br /> <br /> On the viz nodes, use /projects/tools/SCOREC.develop/build-chefMPI-GNU-*/test/chef<br /> For simplicity, this is the default version of the master branch coming directly from our github repository.<br /> <br /> On Tukey, use /home/mrasquin/SCOREC.develop/build-tukey-GNU-OptG-c2c360bc-mpi-*<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35-noblsnap means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is off during uniform refinement (UR).<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol35 means that the target imbalance for the vtx and elem is 3% and 5% respectively, and BL snapping is on during UR.<br /> - build-tukey-GNU-OptG-c2c360bc-mpi-tol33 means that the target imbalance for both the vtx and elem is 3%, and BL snapping is on during UR.<br /> Note that these versions have been slightly modified w.r.t. the master branch. In particular, the imbalance target is not a parameter yet. Also, in Parma, HPS (Heavy Part Splitting) and FixDisconnectedPart are not called here because the latest version of the diffusion algorithm with improved selection of (i) target parts for element exchange and (ii) elements.<br /> <br /> On Mira, use /home/mrasquin/SCOREC.develop/build-XL-OptG-c2c360bc-mpi-*<br /> Similar comments applies to build-XL-OptG-c2c360bc-mpi-tol33, build-XL-OptG-c2c360bc-mpi-tol35 and build-XL-OptG-c2c360bc-mpi-tol35-noblsnap.<br /> <br /> Note that BL snapping is not called for a repartitioning of the mesh. It can only play a role during uniform refinement.<br /> Consequently, if you do not request a UR in adapt.inp, then build-*-tol35 and build-*-tol35-noblsnap will behave the same way.<br /> <br /> In case you are wondering what the weird numbers are in the name of the build directory, this comes from the git log hash, which is a unique number associated with a git commit (easier to couple an executable with a version of the code).<br /> <br /> <br /> = Updated Chef version (2015/05/29 and 2016/04/05)=<br /> <br /> Updated list of useful parameters in adapt.inp<br /> <br /> * '''timeStepNumber''': this is the time step of the output phasta files that will be generated by Chef. This stamp can be different from the number specified in numstart.dat which can be practical in some situations. But most of the time, this number is set equal to what is specified in numstart.dat<br /> <br /> * '''ensa_dof''': this corresponds to the number of degrees of freedom in the solution field of the output restart file. Note that it should correspond to the number of initial conditions specified in the spj file if the solution is built from scratch. When the solution is migrated from existing restart files, it should also correspond to the number of dof in the existing solution field. Here, this number is set to 5 for single phase flow with no turbulence model.<br /> <br /> * '''attributeFileName''': path to the spj file for the boundary and potentially initial conditions<br /> <br /> * '''modelFileName''': path to the geometric model (can be a parasolid or geomsim model on Linux but only geomsim is available on BGQ).<br /> <br /> * '''meshFileName''': path to the directory that includes the input mesh files under the SCOREC MDS format. Note that the path must end with a /. This path can also be prepended by &quot;bz2:&quot; to tell the mesh file reader that the files have been compressed. This follows the same convention as mentioned in 3)<br /> <br /> * '''outMeshFileName''': obviously the name of the directory that will include the resulting output mesh files. Note again the trailing / character. The same convention with &quot;bz2:&quot; keyword applies.<br /> <br /> * '''restartFileName''': this gives the path to the restart files that needs to be read in when solution migration is activated. In this case, the path should look for instance like &quot;../4-procs_case/restart&quot;. The phasta reader will then add the time step stamp to the name of this restartFileName variable, as well as the file #. When there is no solution migration like in this example, this parameter can be commented out for the sake of clarity.<br /> <br /> * '''adaptFlag''': if 0, no mesh adaptation will take place. But if set to 1 and if AdaptStrategy is set to 7, then the mesh will be uniformly refined. Now, mixed mesh can be refined uniformly either everywhere including the BL, or only outside the BL (see parameter splitAllLayerEdges below). Other adaptation strategies based on local error indicators are being developed in Chef and will be complimentary to the existing strategies available in phParAdapt-Simmetrix. If adaptFlags is set to 1, note also that solutionMigration must be also set to 1 (see below for this parameter) and the path to the restart files specified.<br /> <br /> * '''AdaptStrategy''': This parameter is read if adaptFlat is 1. When set up to 7, uniform refinement of a mixed mesh can take place. This is currently the only strategy tested and validated in Chef. <br /> <br /> * '''RecursiveUR''': if AdaptStrategy is set to 7, Chef offers the possibility to do recursive uniform refinement within the same job. Beware of the memory consumption if you set this value to more than 1, since the mesh can grow quickly.<br /> <br /> * '''SplitAllLayerEdges''': This parameter is only applicable to mixed meshes during uniform refinement. If set to 1, uniform refinement of all mesh edges including the edges along normal growth curve (wedges) of the BL. If set to 0, uniform refinement except for edges along normal growth curve (tets only). For all tet meshes, this parameter is ignored and all the tets get split.<br /> <br /> * '''Snap''': If set to 1 during a uniform refinement, Chef will attempt snapping to model surface. Use with caution and check the resulting mesh: if the input mixed mesh is too coarse, snapping can be partially ignored. Non valid meshes have also sometimes been observed (phasta crash).<br /> <br /> * '''splitFactor''':, ratio of the number of output parts with the number of input parts. If you set this parameter to 1, the mesh will not be split and the number of output parts will be equal to the number of input parts. <br /> <br /> * '''elementsPerMigration''': In order to reduce the memory foot print of Chef, the user can reduce the default number of elements that can be migrated at a time during partitioning or partition improvement.<br /> <br /> * '''SolutionMigration''': Activates the migration of the solution from an existing set of restart files. In this case, the path to the phasta files that contain the solution to migrate must be specified through the restartFileName parameter (see above). If the mesh is refined, the solution that is migrated will be interpolated to the new vertices of the mesh. Note also that if the solution is migrated, then the spj file should contain NO information about the initial condition. Indeed any information mentioned in the spj file will prevail. Therefore, if the spj file contains information about the initial conditions, the solution migrated from existing restart files will be overwritten and the resulting phasta files will include again the scratch solution specified in the spj file.<br /> <br /> * '''DisplacementMigration''': Migrates also the displacement field along with with solution field for other adaptation strategies. Not used for AdaptStrategy 7 so can be ignored for now.<br /> <br /> * '''Tetrahedronize''': tetrahedronize a mixed mesh if set to 1. Note that if both AdaptFlag and Tetrahedronize are set to 1, adaptation of the input mixed mesh will take place before tetrahedronization. In all cases, partitioning is always the last mesh operation. But again, an all tet mesh cannot be further refined so tetrahedronization should not take place too early in the partitioning workflow in order to get enough aggregated memory for potential future adaptation.<br /> <br /> * '''partitionMethod''': graph or zrib (for Zoltan Recursive Inertia Bisection) are both available options so far. Graph should be preferred choice for now.<br /> <br /> * '''LocalPtn''': 0 for global partitioning, 1 for local partitioning. Global partitioning coupled with graph as the partition method requires a lot of memory and time and is limited to coarse meshes with a small number of mesh parts. Global RIB is more robust but can lead to larger vertex imbalance. If starting from a well balanced mesh with few or no disconnected parts, local graph is the recommended choice so far.<br /> <br /> * '''ParmaPtn''': If set to 1, the load balance in terms of both elements and vertices per part is improved further after the partitioning with Parma. It is strongly suggested to keep ParmaPtn set to 1.<br /> <br /> * '''elementImbalance''': target element imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''vertexImbalance''': target vertex imbalance for Parma. Use 1.01 to 1.05, which corresponds respectively to 1% and 5%.<br /> <br /> * '''dwalMigration''': This parameter is useful in case the distance to the wall for a turbulence model such as RANS or DDES has already been computed by phasta. In this case, it is possible to migrate also this field along with the solution field. SolutionMigration must therefore be set to 1 for that purpose, since the dwal field cannot be migrated alone without the solution field.<br /> <br /> * '''buildMapping''': This computes the vertex mapping between the input and output mesh. It is strongly suggested to keep this parameter always set to 1. Otherwise, you will not be able to reduce your solution from your final partitioning down to the initial or any intermediate mesh (we have developed a tool for that purpose), which can be catastrophic if you are interested in local adaptation based on an error indicator. Note that building the mapping does not make sense if the mesh is uniformly refined so it should be set to 0 in this case.<br /> <br /> * '''initBubbles''': The Chef will use the external bubble information file 'bubbles.inp' to initialize the level set distance field if this flag is detected to be activated.<br /> <br /> * '''filterMatches''': If set to 0, the matching between periodic entities are imposed from the mesh file regardless of the periodic faces defined in the attributes that could potentially differ. When enabled, this code derives the period associations between all model entities based on the &quot;periodic slave&quot; attribute set in the attribute file. Note that the mesh must be periodic to support this feature. This can be useful for instance for a three-way periodic channel mesh that can support both 1-way and 3-way periodic attributes without the need to build two distinct meshes for that purpose.<br /> <br /> * '''axisymmetry''': When enabled, this parameter supports axisymmetric periodic mesh and attributes for instance for an annular flow.<br /> <br /> = Words of caution=<br /> When working with large meshes occasionally the executable will crash by no fault of the user. This has happened in the past when (adapt.inp set parameter: ensa_ndof)*(number of nodes in the mesh)&gt;(maximum value of the 4-byte integer 2^31). This was debugged by using the addr2line command. Information about this command can be found at https://fluid.colorado.edu/wiki/index.php/Debugging.<br /> <br /> [[Category:Chef]]</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1932 2023-02-02T22:24:43Z

<p>Prte0550: /* Model Convert */ Know issue addition</p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> === Other Functionality ===<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; or &lt;code&gt;.smd&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;<br /> <br /> Please note that there is a known issue with simmodler where performing certain geometric transformations and adjustments can cause the &lt;code&gt;.smd&lt;/code&gt; and &lt;code&gt;_nat.x_t&lt;/code&gt; to have disagreeing model tags for the same model entity. In this scinario other tools will lean on the information in the &lt;code&gt;.smd&lt;/code&gt; so it is best to use mdlConvert with a &lt;code&gt;.smd&lt;/code&gt; file as input.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1931 2023-02-02T22:17:18Z

<p>Prte0550: /* Model Convert */</p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> === Other Functionality ===<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; or &lt;code&gt;.smd&lt;/code&gt;file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Exporting_Parasolid_from_SolidWorks&diff=1930 2023-01-12T21:53:36Z

<p>Prte0550: </p> <hr /> <div>Save out your Model as a parasolid from [[SolidWorks]]. Note that you will want the geometry as close to ready for meshing as possible, as performing model &quot;surgery&quot; in SimModeler is not always straight forward. The outputted file from SolidWorks will have the format &lt;code&gt;&lt;file_name&gt;.x_t&lt;/code&gt;.<br /> <br /> If you do not have a parasolid model of your own, you may use the On Ramp example file located at:<br /> <br /> /projects/tutorials/OnRamp/example_geom.x_t<br /> <br /> Ensure that you are on one of the viznodes and not portal1. You may tunnel to viz003 by opening a terminal and running:<br /> <br /> vglconnect -s viz003<br /> <br /> Navigate to and copy your file into your working directory. Typically, we create a folder where all the simulation files are stored. For example, after opening a terminal I could run the commands:<br /> <br /> mkdir Demo<br /> cd Demo<br /> <br /> This would place me in my working directory 'Demo'. <br /> <br /> Next, you'll want to change the parasolid file extension from &lt;code&gt;.x_t&lt;/code&gt; to &lt;code&gt;.xmt_txt&lt;/code&gt;. To do this, run &lt;code&gt;mv &lt;file_name&gt;.x_t &lt;file_name&gt;.xmt_txt&lt;/code&gt; from your terminal. From here, you are ready for the ''convert'' step. Copy over the convert script by running:<br /> <br /> cp /projects/tutorials/OnRamp/convertParasolid2Sim.sh .<br /> <br /> The convert step is documented here: https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4<br /> <br /> <br /> '''Summary of video:''' <br /> 1. Ensure &lt;code&gt;convertParasolid2Sim.sh&lt;/code&gt; and &lt;code&gt;&lt;file name&gt;.xmt_txt&lt;/code&gt; are in your working directory. <br /> <br /> 2. Set environment with soft adds found in &lt;code&gt;more ~kjansen/soft-core.sh&lt;/code&gt;<br /> <br /> 3. Run &lt;code&gt;./convertParasolid2sim.sh &lt;file name&gt;.xmt_txt&lt;/code&gt; in your terminal<br /> <br /> 4. Convert step is complete and your directory now contains 3 new files: &lt;code&gt;model.smd&lt;/code&gt; &lt;code&gt;relations.log&lt;/code&gt; &amp; &lt;code&gt;translated-model.smd&lt;/code&gt;. The &lt;code&gt;translated-model.smd&lt;/code&gt; file is the one we need moving forward in this tutorial.<br /> <br /> <br /> Once the convert step is complete, you are ready to move onto the next step and use [[Getting Started with Simmodeler| SimModeler]] to create a mesh for the new &lt;code&gt;translated-model.smd&lt;/code&gt; file we created!<br /> <br /> == Geometries with Standalone Surfaces ==<br /> <br /> There are sometimes cases where models will contain not only three dimensional solid bodies but also surfaces. Surfaces are treated differently by Solidworks and parasolid files than solid bodies, so they need to be exported separately. In order to do this, you select all entities except the surface and save as a parasolid, after you hit save, Solidworks will ask if you want to save all of the geometry or only the selected geometry, the second option being the one we want. Now do the same but only select the surface. In newer versions of Simmodler, only the domain needs to be run though the conversion and the surfaces can be added as parasolid files. In older versions, these two separate parasolid files needed to be converted to &lt;code&gt;.smd&lt;/code&gt; separately (remember to rename files after they are converted to avoid overwriting) and recombined in Simmodeler. The toolchain seems to fail for multiple unconnected surfaces, so if this is the case, take care to export them separately.<br /> <br /> To recombine, simply open the solid body file in Simmodeler, under the &quot;Modeling&quot; tab select &quot;Add Parts&quot;, add the surfaces file, then select &quot;Make New Manifold Model&quot; which will combine the files into one model that is suitable for PHASTA.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Exporting_Parasolid_from_SolidWorks&diff=1929 2023-01-12T21:50:43Z

<p>Prte0550: </p> <hr /> <div>Save out your Model as a parasolid from [[SolidWorks]]. Note that you will want the geometry as close to ready for meshing as possible, as performing model &quot;surgery&quot; in SimModeler is not always straight forward. The outputted file from SolidWorks will have the format &lt;code&gt;&lt;file_name&gt;.x_t&lt;/code&gt;.<br /> <br /> If you do not have a parasolid model of your own, you may use the On Ramp example file located at:<br /> <br /> /projects/tutorials/OnRamp/example_geom.x_t<br /> <br /> Ensure that you are on one of the viznodes and not portal1. You may tunnel to viz003 by opening a terminal and running:<br /> <br /> vglconnect -s viz003<br /> <br /> Navigate to and copy your file into your working directory. Typically, we create a folder where all the simulation files are stored. For example, after opening a terminal I could run the commands:<br /> <br /> mkdir Demo<br /> cd Demo<br /> <br /> This would place me in my working directory 'Demo'. <br /> <br /> Next, you'll want to change the parasolid file extension from &lt;code&gt;.x_t&lt;/code&gt; to &lt;code&gt;.xmt_txt&lt;/code&gt;. To do this, run &lt;code&gt;mv &lt;file_name&gt;.x_t &lt;file_name&gt;.xmt_txt&lt;/code&gt; from your terminal. From here, you are ready for the ''convert'' step. Copy over the convert script by running:<br /> <br /> cp /projects/tutorials/OnRamp/convertParasolid2Sim.sh .<br /> <br /> The convert step is documented here: https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4<br /> <br /> <br /> '''Summary of video:''' <br /> 1. Ensure &lt;code&gt;convertParasolid2Sim.sh&lt;/code&gt; and &lt;code&gt;&lt;file name&gt;.xmt_txt&lt;/code&gt; are in your working directory. <br /> <br /> 2. Set environment with soft adds found in &lt;code&gt;more ~kjansen/soft-core.sh&lt;/code&gt;<br /> <br /> 3. Run &lt;code&gt;./convertParasolid2sim.sh &lt;file name&gt;.xmt_txt&lt;/code&gt; in your terminal<br /> <br /> 4. Convert step is complete and your directory now contains 3 new files: &lt;code&gt;model.smd&lt;/code&gt; &lt;code&gt;relations.log&lt;/code&gt; &amp; &lt;code&gt;translated-model.smd&lt;/code&gt;. The &lt;code&gt;translated-model.smd&lt;/code&gt; file is the one we need moving forward in this tutorial.<br /> <br /> <br /> Once the convert step is complete, you are ready to move onto the next step and use [[Getting Started with Simmodeler| SimModeler]] to create a mesh for the new &lt;code&gt;translated-model.smd&lt;/code&gt; file we created!<br /> <br /> == Geometries with Standalone Surfaces ==<br /> <br /> There are sometimes cases where models will contain not only three dimensional solid bodies but also surfaces. Surfaces are treated differently by Solidworks and parasolid files than solid bodies, so they need to be exported separately. In order to do this, you select all entities except the surface and save as a parasolid, after you hit save, Solidworks will ask if you want to save all of the geometry or only the selected geometry, the second option being the one we want. Now do the same but only select the surface. These two separate files can then be converted to &lt;code&gt;.smd&lt;/code&gt; separately (remember to rename files after they are converted to avoid overwriting) and recombined in Simmodeler. The toolchain seems to fail for multiple unconnected surfaces, so if this is the case, take care to export them separately.<br /> <br /> To recombine, simply open the solid body file in Simmodeler, under the &quot;Modeling&quot; tab select &quot;Add Parts&quot;, add the surfaces file, then select &quot;Make New Manifold Model&quot; which will combine the files into one model that is suitable for PHASTA.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1928 2023-01-05T21:38:12Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> === Other Functionality ===<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1927 2023-01-05T21:37:32Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> === Other Functionality ===<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1926 2023-01-05T21:37:08Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Basic Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> = Other Functionality =<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1925 2023-01-05T21:36:56Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Basic Overview =<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;code&gt;.crd&lt;/code&gt; files, though there are also versions that will push the model to &lt;code&gt;.dmg&lt;/code&gt; format.<br /> <br /> For all of the options below, also note that you should always use a version of convert built for the specific version of Simmodeler used to generate the mesh. This is most easily achieved by using up-to-date versions.<br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in Simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> = Other Functionality =<br /> <br /> Note that Convert can intake multiple model regions if that is required for a geometry. This is available in the main bran and simply takes in more than one model root face arguments in a separate file. This file should have all of the root faces in a new-line delimited format, and the call to convert changes to:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/buildTestMergeEWT/test/convert --model-face-root=ExtruRootID.txt --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where &lt;code&gt;ExtruRootID.txt&lt;/code&gt; is the file containing the root face ID's.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1924 2023-01-05T21:29:49Z

<p>Prte0550: </p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;(location)&lt;/code&gt; as of (date)<br /> <br /> <br /> == Basic Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Convert]] tool for unstructured grids.<br /> <br /> <br /> Once the mesh files are in place, MGEN can be run with &lt;code&gt;./tm3Extrude&lt;/code&gt; as usual. The code will ask for inputs for zmin, zmax, numelz, and npart. These should be entered in a single string with spaces in between the values before hitting in order to continue code execution.<br /> <br /> == Advanced Usage ==<br /> For more complex geometries, complete information about the model cannot be assumed and must instead be given to MGEN. In order to prepare for this, we will need an additional form of the mesh and to make changes to the MGEN code itself in order to tell the program where geometric features are. <br /> <br /> A model first needs to be converted into a &lt;code&gt;.dmg&lt;/code&gt; file. This can be created with &lt;code&gt;mdlConvert&lt;/code&gt;. Example usage of this is &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert &lt;simmetixMesh&gt;.xmt_txt outModel.dmg&lt;/code&gt;. This captures only information about model points, edges, and faces and their relationships to each other but does not capture information about physical location.<br /> <br /> == Outputs ==<br /> MGEN will write its outputs to the same working directory that the executable and source mesh files are in. There are multiple file types written, most with a suffix of a number to denote the part number of that file. The different parted files and their purposes are as follows:<br /> <br /> ;geom3D.class :Classification file describing what type of geometric entity each point lies on (vertex, edge, face, volume)<br /> ;geom3D.cnndt :Connectivity of the elements <br /> ;geom3D.coord :Node coordinates<br /> ;geom3D.fathr :Parent vertex from the 2D source mesh<br /> ;geom3D.match :Contains periodic partners<br /> <br /> <br /> There is also one more file:<br /> <br /> ; geom3DHead.cnn<br /> <br /> Which lists the headers containing information on the size of the file each of the above connectivity files.<br /> <br /> == Using the outputted files ==<br /> The outputted files from MGEN now need to be prepared for Chef, this is done via &lt;code&gt;matchedNodeElmReader&lt;/code&gt;. The provided example will be for a build on Cooley.<br /> <br /> First, the environment needs to be prepared via setting &lt;code&gt;SIM_LICENSE_FILE&lt;/code&gt; and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt;. Examples of this are:<br /> <br /> <br /> &lt;code&gt;<br /> export SIM_LICENSE_FILE=/eagle/PHASTA_aesp/SCOREC-CORE/deps/Simmetrix/UCBoulder<br /> <br /> export LD_LIBRARY_PATH=/eagle/PHASTA_aesp/SCOREC-CORE/deps/16.0-220326/lib/x64_rhel_gcc48/psKrnl/:$LD_LIBRARY_PATH<br /> &lt;/code&gt;<br /> <br /> <br /> From here, &lt;code&gt;matchedNodeElmReader&lt;/code&gt; can be run with:<br /> <br /> &lt;code&gt;<br /> mpirun -f /var/tmp/cobalt.2137783 -np &lt;np&gt; -genvall /eagle/PHASTA_aesp/SCOREC-CORE/build_gtvertCorruption/test/matchedNodeElmReader ../geom3D.cnndt ../geom3D.coord ../geom3D.match ../geom3D.class ../geom3D.fathr NULL ../geom3DHead.cnn outModel.dmg outModel/<br /> &lt;/code&gt;<br /> <br /> Where &lt;np&gt; should be replaced by the same number as used for npart when running MGEN.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Exporting_Parasolid_from_SolidWorks&diff=1923 2023-01-05T21:28:13Z

<p>Prte0550: </p> <hr /> <div>Save out your Model as a parasolid from [[SolidWorks]]. Note that you will want the geometry as close to ready for meshing as possible, as performing model &quot;surgery&quot; in SimModeler is not always straight forward. The outputted file from SolidWorks will have the format &lt;code&gt;&lt;file_name&gt;.x_t&lt;/code&gt;.<br /> <br /> If you do not have a parasolid model of your own, you may use the On Ramp example file located at:<br /> <br /> /projects/tutorials/OnRamp/example_geom.x_t<br /> <br /> Ensure that you are on one of the viznodes and not portal1. You may tunnel to viz003 by opening a terminal and running:<br /> <br /> vglconnect -s viz003<br /> <br /> Navigate to and copy your file into your working directory. Typically, we create a folder where all the simulation files are stored. For example, after opening a terminal I could run the commands:<br /> <br /> mkdir Demo<br /> cd Demo<br /> <br /> This would place me in my working directory 'Demo'. <br /> <br /> Next, you'll want to change the parasolid file extension from &lt;code&gt;.x_t&lt;/code&gt; to &lt;code&gt;.xmt_txt&lt;/code&gt;. To do this, run &lt;code&gt;mv &lt;file_name&gt;.x_t &lt;file_name&gt;.xmt_txt&lt;/code&gt; from your terminal. From here, you are ready for the ''convert'' step. Copy over the convert script by running:<br /> <br /> cp /projects/tutorials/OnRamp/convertParasolid2Sim.sh .<br /> <br /> The convert step is documented here: https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4<br /> <br /> <br /> '''Summary of video:''' <br /> 1. Ensure &lt;code&gt;convertParasolid2Sim.sh&lt;/code&gt; and &lt;code&gt;&lt;file name&gt;.xmt_txt&lt;/code&gt; are in your working directory. <br /> <br /> 2. Set environment with soft adds found in &lt;code&gt;more ~kjansen/soft-core.sh&lt;/code&gt;<br /> <br /> 3. Run &lt;code&gt;./convertParasolid2sim.sh &lt;file name&gt;.xmt_txt&lt;/code&gt; in your terminal<br /> <br /> 4. Convert step is complete and your directory now contains 3 new files: &lt;code&gt;model.smd&lt;/code&gt; &lt;code&gt;relations.log&lt;/code&gt; &amp; &lt;code&gt;translated-model.smd&lt;/code&gt;. The &lt;code&gt;translated-model.smd&lt;/code&gt; file is the one we need moving forward in this tutorial.<br /> <br /> <br /> Once the convert step is complete, you are ready to move onto the next step and use [[Getting Started with Simmodeler| SimModeler]] to create a mesh for the new &lt;code&gt;translated-model.smd&lt;/code&gt; file we created!<br /> <br /> == Geometries with Standalone Surfaces ==<br /> <br /> There are sometimes cases where models will contain not only three dimensional solid bodies but also surfaces. Surfaces are treated differently by Solidworks and parasolid files than solid bodies, so they need to be exported separately. IN order to do this, you select all entities except the surface and save as a parasolid, after you hit save, Solidworks will ask if you want to save all of the geometry or only the selected geometry, the second option being the one we want. Now do the same but only select the surfaces. These two separate files can then be converted to &lt;code&gt;.smd&lt;/code&gt; separately (remember to rename files after they are converted to avoid overwriting) and recombined in Simmodeler.<br /> <br /> To recombine, simply open the solid body file in Simmodeler, under the &quot;Modeling&quot; tab select &quot;Add Parts&quot;, add the surfaces file, then select &quot;Make New Manifold Model&quot; which will combine the files into one model that is suitable for PHASTA.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Exporting_Parasolid_from_SolidWorks&diff=1922 2023-01-05T18:02:00Z

<p>Prte0550: Added notes for mixed solid body / surface models</p> <hr /> <div>Save out your Model as a parasolid from [[SolidWorks]]. Note that you will want the geometry as close to ready for meshing as possible, as performing model &quot;surgery&quot; in SimModeler is not always straight forward. The outputted file from SolidWorks will have the format &lt;code&gt;&lt;file_name&gt;.x_t&lt;/code&gt;.<br /> <br /> If you do not have a parasolid model of your own, you may use the On Ramp example file located at:<br /> <br /> /projects/tutorials/OnRamp/example_geom.x_t<br /> <br /> Ensure that you are on one of the viznodes and not portal1. You may tunnel to viz003 by opening a terminal and running:<br /> <br /> vglconnect -s viz003<br /> <br /> Navigate to and copy your file into your working directory. Typically, we create a folder where all the simulation files are stored. For example, after opening a terminal I could run the commands:<br /> <br /> mkdir Demo<br /> cd Demo<br /> <br /> This would place me in my working directory 'Demo'. <br /> <br /> Next, you'll want to change the parasolid file extension from &lt;code&gt;.x_t&lt;/code&gt; to &lt;code&gt;.xmt_txt&lt;/code&gt;. To do this, run &lt;code&gt;mv &lt;file_name&gt;.x_t &lt;file_name&gt;.xmt_txt&lt;/code&gt; from your terminal. From here, you are ready for the ''convert'' step. Copy over the convert script by running:<br /> <br /> cp /projects/tutorials/OnRamp/convertParasolid2Sim.sh .<br /> <br /> The convert step is documented here: https://fluid.colorado.edu/tutorials/tutorialVideos/Convert2Sim_Tutorial.mp4<br /> <br /> <br /> '''Summary of video:''' <br /> 1. Ensure &lt;code&gt;convertParasolid2Sim.sh&lt;/code&gt; and &lt;code&gt;&lt;file name&gt;.xmt_txt&lt;/code&gt; are in your working directory. <br /> <br /> 2. Set environment with soft adds found in &lt;code&gt;more ~kjansen/soft-core.sh&lt;/code&gt;<br /> <br /> 3. Run &lt;code&gt;./convertParasolid2sim.sh &lt;file name&gt;.xmt_txt&lt;/code&gt; in your terminal<br /> <br /> 4. Convert step is complete and your directory now contains 3 new files: &lt;code&gt;model.smd&lt;/code&gt; &lt;code&gt;relations.log&lt;/code&gt; &amp; &lt;code&gt;translated-model.smd&lt;/code&gt;. The &lt;code&gt;translated-model.smd&lt;/code&gt; file is the one we need moving forward in this tutorial.<br /> <br /> <br /> Once the convert step is complete, you are ready to move onto the next step and use [[Getting Started with Simmodeler| SimModeler]] to create a mesh for the new &lt;code&gt;translated-model.smd&lt;/code&gt; file we created!<br /> <br /> == Geometries with Standalone Surfaces ==<br /> <br /> There are sometimes cases where models will contain not only three dimensional solid bodies but also surfaces. Surfaces are treated differently by Solidworks and parasolid files than solid bodies, so they need to be exported separately. IN order to do this, you select all entities except the surface and save as a parasolid, after you hit save, Solidworks will ask if you want to save all of the geometry or only the selected geometry, the second option being the one we want. Now do the same but only select the surfaces. These two separate files can then be converted to &lt;code&gt;.smd&lt;/code&gt; separately (remember to rename files after they are converted to avoid overwriting) and recombined in Symmodeler.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1921 2023-01-04T19:11:44Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Basic Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;/code&gt;.crd&lt;/code&gt; files for use with &lt;/code&gt;.crd&lt;/code&gt;.<br /> <br /> <br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/build16_Opt/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> Note that there is a second version of convert which can intake multiple model regions if that is required for a geometry. This is available at &lt;code&gt;/projects/tools/SCOREC-core/build16_Opt/test/convert&lt;/code&gt; and simply takes in more than one (comma delimited) model root face arguments.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1920 2023-01-04T18:33:52Z

<p>Prte0550: </p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Basic Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;/code&gt;.crd&lt;/code&gt; files for use with &lt;/code&gt;.crd&lt;/code&gt;.<br /> <br /> <br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.smd&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;code&gt;.crd&lt;/code&gt; files, and a &lt;code&gt;./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/build16_Opt/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> Note that there is a second version of convert which can intake multiple model regions if that is required for a geometry. This is available at &lt;code&gt;/projects/tools/SCOREC-core/build16_Opt/test/convert&lt;/code&gt; and simply takes in more than one (space delimited) model root face arguments.<br /> <br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ParaView/Run_on_Remote_Machine&diff=1919 2022-11-11T19:28:56Z

<p>Prte0550: </p> <hr /> <div>Here are instructions for running ParaView on some selected remote machines. <br /> <br /> In general, this involves:<br /> # Launching an interactive job using the machine's job scheduler <br /> # Loading whatever required software is needed on the session<br /> # Launching the Paraview server on the session<br /> # Connecting to the remote Paraview server through a local Paraview instance<br /> <br /> == Current Machines ==<br /> <br /> === Cooley (ALCF) ===<br /> ==== Connect to Cooley ====<br /> *From the command line on the Viz Nodes,<br /> ssh '''''username'''''@cooley.alcf.anl.gov<br /> *Enter the Mobile Pass + passcode from your phone<br /> <br /> ==== Submit Interactive Job ====<br /> There are multiple versions of interactive submission scripts used by the group, but most take in two inputs, number of nodes and run time. It is recommended that you check the contents of any given &lt;code&gt;submitInteractive.sh&lt;/code&gt; script you receive to look for the argument order. As an example though, the scripts being used for Gust AFOSR work (at &lt;code&gt;/projects/PHASTA_aesp/Models/GustWing/OTS/PastRuns &lt;/code&gt; is used as: &lt;code&gt; ./submitInteractive.sh &lt;runtime in minutes&gt; &lt;nodes&gt; &lt;/code&gt;.<br /> <br /> Once the interactive job is started, you will need to start a ParaView server using a &lt;code&gt;pvserverLaunch.&lt;version&gt;.sh&lt;/code&gt; script. Note that the version of the server needs to match the version you will use on the Viz Nodes. ParaView 5.5.2 is the most recent version fully supported on both the Viz Nodes and Cooley. The script available at he same directory as above takes in the number of processes per node as an input, which to use all of Cooley's resources should be 12, which is ran as: &lt;code&gt;./pvserverLaunch.5.5.2.sh 12&lt;/code&gt;. <br /> <br /> Once the server is running and &quot;Waiting for client...&quot; you should launch ParaView on the Viz Nodes and select the connect to server icon (just to the right of the open file icon at the top of the screen). From here, using the listed &lt;code&gt;cc***&lt;/code&gt; number, you will need to configure the server connection if you do not have one configured for the &lt;code&gt;cc***&lt;/code&gt; number. This is done by selecting &quot;Add Server&quot; and then populating the &quot;Edit Server Configuration&quot; page. The name of the server should be the &lt;code&gt;cc***&lt;/code&gt; number, the host field will be &lt;code&gt;cc***.cooley.pub.alcf.anl.gov&lt;code&gt; and the port is &lt;code&gt;8000&lt;/code&gt;. From here you can select configure and then connect.<br /> <br /> If, upon connection, you need to load the SyncIO remote plugin, these can be found for multiple versions of ParaView at &lt;code&gt;/lus/theta-fs0/projects/PHASTA_aesp/ParaView/ParaViewSyncIOReaderPlugin&lt;/code&gt;.<br /> <br /> <br /> == Retired Machines ==<br /> <br /> === Eureka (ALCF) ===<br /> ==== Connect to Eureka ====<br /> *From the command line on the Colorado machine,<br /> ssh '''''username'''''@eureka.alcf.anl.gov<br /> *Enter 4 digit pin, followed by the number on the CRYPTOCard<br /> <br /> ==== Submit Interactive Job ====<br /> *From the command line on Eureka, &lt;code&gt;cp /home/jmartin/qsub_interactive_command.sh&lt;/code&gt; to your home directory. Open &lt;code&gt;qsub_interactive_command.sh&lt;/code&gt; and change the total time you want to run paraview (time), and the allocation your account is under (account).<br /> ./qsub_interactive_command.sh '''''nodes'''''<br /> where '''''nodes''''' is the total number of nodes you want. Each node has 8 cores, and 32 GB of memory.<br /> <br /> === Janus (CUBoulder Research Computing) ===<br /> Video Tutorial (use the paths below, NOT the ones in the Video)<br /> http://fluid.colorado.edu/~matthb2/janus/pv_on_janus.html<br /> <br /> ==== Running the UI on Portal0 ====<br /> soft add @paraview-3.8.0<br /> soft add +paraview-3.8.0-gnu-ompi-covis<br /> vglrun paraview<br /> <br /> and then start and connect to the server:<br /> <br /> ==== Starting the Server on Janus ====<br /> . /projects/jansenke/matthb2/env-gnu.sh<br /> qsub -q janus-debug /projects/jansenke/matthb2/pvserver-gnu_runscript-sysgl.sh<br /> <br /> and use checkjob or look at the output files to figure out which node your rank0 is on and connect ParaView to that (or change it to use reverse connections if you prefer).<br /> <br /> === Tukey (ALCF) ===<br /> ==== ParaView GUI running on portal0 at Colorado ====<br /> <br /> Video Tutorial about how to run a pvserver-syncio in parallel on the Tukey visualization nodes and connect the pvserver to a ParaView Gui running on portal0 at Colorado <br /> http://fluid.colorado.edu/~mrasquin/Documents_HIDE/Tukey/ParaviewOnTukeyFromPortal0/index.html<br /> <br /> This video can be copied from /users/mrasquin/public_html/Documents_HIDE/Tukey/ParaviewOnTukeyFromPortal0 on the viz nodes.<br /> <br /> ==== ParaView GUI running on the Tukey login node ====<br /> Video Tutorial about how to run a pvserver-syncio in parallel on the Tukey visualization nodes and connect the pvserver to a ParaView Gui running on the Tukey login node<br /> https://fluid.colorado.edu/~mrasquin/phasta/ParaViewOnTukey/index.html<br /> <br /> This video can be copied from /users/mrasquin/public_html/Tukey/ParaviewOnTukeyThroughVNC on the viz nodes.<br /> <br /> Note that because vncserver on the Tukey head node does not support OpenGL, this method does not allow the export of png pictures from the ParaView Gui. Indeed, the result will be completely fuzzy. The first method is therefore strongly recommended.<br /> <br /> [[Category:Paraview]]</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ParaView/Run_on_Remote_Machine&diff=1918 2022-11-11T17:43:37Z

<p>Prte0550: Added Cooley documentation</p> <hr /> <div>Here are instructions for running ParaView on some selected remote machines. <br /> <br /> In general, this involves:<br /> # Launching an interactive job using the machine's job scheduler <br /> # Loading whatever required software is needed on the session<br /> # Launching the Paraview server on the session<br /> # Connecting to the remote Paraview server through a local Paraview instance<br /> <br /> == Current Machines ==<br /> <br /> === Cooley (ALCF) ===<br /> ==== Connect to Cooley ====<br /> *From the command line on the Viz Nodes,<br /> ssh '''''username'''''@cooley.alcf.anl.gov<br /> *Enter the Mobile Pass + passcode from your phone<br /> <br /> ==== Submit Interactive Job ====<br /> There are multiple versions of interactive submission scripts used by the group, but most take in two inputs, number of nodes and run time. It is recommended that you check the contents of any given &lt;code&gt;submitInteractive.sh&lt;/code&gt; script you receive to look for the argument order. As an example though, the scripts being used for Gust AFOSR work (at &lt;code&gt;/projects/PHASTA_aesp/Models/GustWing/OTS/PastRuns &lt;/code&gt; is used as: &lt;code&gt; ./submitInteractive.sh &lt;runtime in minutes&gt; &lt;nodes&gt; &lt;/code&gt;.<br /> <br /> Once the interactive job is started, you will need to start a ParaView server using a &lt;code&gt;pvserverLaunch.&lt;version&gt;.sh&lt;/code&gt; script. Note that the version of the server needs to match the version you will use on the Viz Nodes. ParaView 5.5.2 is the most recent version fully supported on both the Viz Nodes and Cooley. The script available at he same directory as above takes in the number of processes per node as an input, which to use all of Cooley's resources should be 12, which is ran as: &lt;code&gt;./pvserverLaunch.5.5.2.sh 12&lt;/code&gt;. <br /> <br /> Once the server is running and &quot;Waiting for client...&quot; you should launch ParaView on the Viz Nodes and select the connect to server icon (just to the right of the open file icon at the top of the screen). From here, using the listed &lt;code&gt;cc***&lt;/code&gt; number, you will need to configure the server connection if you do not have one configured for the &lt;code&gt;cc***&lt;/code&gt; number. This is done by selecting &quot;Add Server&quot; and then populating the &quot;Edit Server Configuration&quot; page. The name of the server should be the &lt;code&gt;cc***&lt;/code&gt; number, the host field will be &lt;code&gt;cc***.cooley.pub.alcf.anl.gov&lt;code&gt; and the port is &lt;code&gt;8000&lt;/code&gt;. From here you can select configure and then connect.<br /> <br /> If, upon connection, you need to load the SyncIO plugin, these can be found for multiple versions of ParaView at &lt;code&gt;/lus/theta-fs0/projects/PHASTA_aesp/ParaView/ParaViewSyncIOReaderPlugin&lt;/code&gt;.<br /> <br /> <br /> == Retired Machines ==<br /> <br /> === Eureka (ALCF) ===<br /> ==== Connect to Eureka ====<br /> *From the command line on the Colorado machine,<br /> ssh '''''username'''''@eureka.alcf.anl.gov<br /> *Enter 4 digit pin, followed by the number on the CRYPTOCard<br /> <br /> ==== Submit Interactive Job ====<br /> *From the command line on Eureka, &lt;code&gt;cp /home/jmartin/qsub_interactive_command.sh&lt;/code&gt; to your home directory. Open &lt;code&gt;qsub_interactive_command.sh&lt;/code&gt; and change the total time you want to run paraview (time), and the allocation your account is under (account).<br /> ./qsub_interactive_command.sh '''''nodes'''''<br /> where '''''nodes''''' is the total number of nodes you want. Each node has 8 cores, and 32 GB of memory.<br /> <br /> === Janus (CUBoulder Research Computing) ===<br /> Video Tutorial (use the paths below, NOT the ones in the Video)<br /> http://fluid.colorado.edu/~matthb2/janus/pv_on_janus.html<br /> <br /> ==== Running the UI on Portal0 ====<br /> soft add @paraview-3.8.0<br /> soft add +paraview-3.8.0-gnu-ompi-covis<br /> vglrun paraview<br /> <br /> and then start and connect to the server:<br /> <br /> ==== Starting the Server on Janus ====<br /> . /projects/jansenke/matthb2/env-gnu.sh<br /> qsub -q janus-debug /projects/jansenke/matthb2/pvserver-gnu_runscript-sysgl.sh<br /> <br /> and use checkjob or look at the output files to figure out which node your rank0 is on and connect ParaView to that (or change it to use reverse connections if you prefer).<br /> <br /> === Tukey (ALCF) ===<br /> ==== ParaView GUI running on portal0 at Colorado ====<br /> <br /> Video Tutorial about how to run a pvserver-syncio in parallel on the Tukey visualization nodes and connect the pvserver to a ParaView Gui running on portal0 at Colorado <br /> http://fluid.colorado.edu/~mrasquin/Documents_HIDE/Tukey/ParaviewOnTukeyFromPortal0/index.html<br /> <br /> This video can be copied from /users/mrasquin/public_html/Documents_HIDE/Tukey/ParaviewOnTukeyFromPortal0 on the viz nodes.<br /> <br /> ==== ParaView GUI running on the Tukey login node ====<br /> Video Tutorial about how to run a pvserver-syncio in parallel on the Tukey visualization nodes and connect the pvserver to a ParaView Gui running on the Tukey login node<br /> https://fluid.colorado.edu/~mrasquin/phasta/ParaViewOnTukey/index.html<br /> <br /> This video can be copied from /users/mrasquin/public_html/Tukey/ParaviewOnTukeyThroughVNC on the viz nodes.<br /> <br /> Note that because vncserver on the Tukey head node does not support OpenGL, this method does not allow the export of png pictures from the ParaView Gui. Indeed, the result will be completely fuzzy. The first method is therefore strongly recommended.<br /> <br /> [[Category:Paraview]]</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Convert&diff=1909 2022-09-21T18:02:34Z

<p>Prte0550: Created page with &quot;Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group....&quot;</p> <hr /> <div>Convert, often referred to simply as &quot;Convert&quot;, is a tool used to convert Geometric model files into a file type usable with the preprocessing tools utilized in this group. <br /> <br /> == Basic Overview ==<br /> The main goal of Convert is to take a Symmetrix model and mesh and convert it to &lt;code&gt;.cnn&lt;/code&gt; and &lt;/code&gt;.crd&lt;/code&gt; files for use with &lt;/code&gt;.crd&lt;/code&gt;.<br /> <br /> <br /> <br /> == Basic Usage ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt;, &lt;code&gt;.xmt_txt&lt;/code&gt;, and a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs &lt;code&gt;.cnn&lt;/code&gt;, &lt;/code&gt;.crd&lt;/code&gt; files, and a &lt;code./mdsMesh&lt;/code&gt; directory. A specific implementation will look like:<br /> <br /> &lt;code&gt; mpirun -np 1 /projects/tools/SCOREC-core/build16_Opt/test/convert --model-face-root=4321 --native_model=geom.xmt_txt geom.smd geom.sms mdsMesh/&lt;/code&gt;<br /> <br /> Where the root face is the face which holds the original meshing attributes in simmodeler (extrusion meshing from within simmodeler would originate from this face).<br /> <br /> == Model Convert ==<br /> Convert takes in a &lt;code&gt;.xmt_txt&lt;/code&gt; file and outputs a &lt;code&gt;.dmg&lt;/code&gt; file. This file type simply stores information about model faces, edges, and vertices, and their relationships to each other. This is needed to classify mesh points when generating a mesh with.<br /> <br /> &lt;!--Model Convert is a part of Chef, and by default a simple version will be built in the process of building Chef. There are also standalone builds of the tool that are required to be built for unique geometries, for instance, for the Gust Wing project, a version of the tool for closed test section slices is available at &lt;code&gt;/projects/tools/SCOREC-core/build-14-190604dev_omp110/test/mdlConvert&lt;/code&gt;.<br /> <br /> See MgenExtru_MGENClassificationAirfoilPt2 video--&gt;</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1908 2022-09-21T17:45:39Z

<p>Prte0550: </p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;(location)&lt;/code&gt; as of (date)<br /> <br /> <br /> == Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Convert]] tool for unstructured grids.<br /> <br /> Once the mesh files are in place, MGEN can be run with &lt;code&gt;./tm3Extrude&lt;/code&gt; as usual. The code will ask for inputs for zmin, zmax, numelz, and npart. These should be entered in a single string with spaces in between the values before hitting in order to continue code execution.<br /> <br /> == Outputs ==<br /> MGEN will write its outputs to the same working directory that the executable and source mesh files are in. There are multiple file types written, most with a suffix of a number to denote the part number of that file. The different parted files and their purposes are as follows:<br /> <br /> ;geom3D.class :Classification file describing what type of geometric entity each point lies on (vertex, edge, face, volume)<br /> ;geom3D.cnndt :Connectivity of the elements <br /> ;geom3D.coord :Node coordinates<br /> ;geom3D.fathr :Parent vertex from the 2D source mesh<br /> ;geom3D.match :Contains periodic partners<br /> <br /> <br /> There is also one more file:<br /> <br /> ; geom3DHead.cnn<br /> <br /> Which lists the headers containing information on the size of the file each of the above connectivity files.<br /> <br /> == Using the outputted files ==<br /> The outputted files from MGEN now need to be prepared for Chef, this is done via &lt;code&gt;matchedNodeElmReader&lt;/code&gt;. The provided example will be for a build on Cooley.<br /> <br /> First, the enviroment needs to be prepared via setting &lt;code&gt;SIM_LICENSE_FILE&lt;/code&gt; and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt;. Examples of this are:<br /> <br /> <br /> &lt;code&gt;<br /> export SIM_LICENSE_FILE=/eagle/PHASTA_aesp/SCOREC-CORE/deps/Simmetrix/UCBoulder<br /> <br /> export LD_LIBRARY_PATH=/eagle/PHASTA_aesp/SCOREC-CORE/deps/16.0-220326/lib/x64_rhel_gcc48/psKrnl/:$LD_LIBRARY_PATH<br /> &lt;/code&gt;<br /> <br /> <br /> From here, &lt;code&gt;matchedNodeElmReader&lt;/code&gt; can be run with:<br /> <br /> &lt;code&gt;<br /> mpirun -f /var/tmp/cobalt.2137783 -np &lt;np&gt; -genvall /eagle/PHASTA_aesp/SCOREC-CORE/build_gtvertCorruption/test/matchedNodeElmReader ../geom3D.cnndt ../geom3D.coord ../geom3D.match ../geom3D.class ../geom3D.fathr NULL ../geom3DHead.cnn outModel.dmg outModel/<br /> &lt;/code&gt;<br /> <br /> Where &lt;np&gt; should be replaced by the same number as used for npart when running MGEN.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1907 2022-09-21T17:23:46Z

<p>Prte0550: </p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;(location)&lt;/code&gt; as of (date)<br /> <br /> <br /> == Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Model Convert]] tool for unstructured grids.<br /> <br /> Once the mesh files are in place, MGEN can be run with &lt;code&gt;./tm3Extrude&lt;/code&gt; as usual. The code will ask for inputs for zmin, zmax, numelz, and npart. These should be entered in a single string with spaces in between the values before hitting in order to continue code execution.<br /> <br /> == Outputs ==<br /> MGEN will write its outputs to the same working directory that the executable and source mesh files are in. There are multiple file types written, most with a suffix of a number to denote the part number of that file. The different parted files and their purposes are as follows:<br /> <br /> ;geom3D.class :Classification file describing what type of geometric entity each point lies on (vertex, edge, face, volume)<br /> ;geom3D.cnndt :Connectivity of the elements <br /> ;geom3D.coord :Node coordinates<br /> ;geom3D.fathr :Parent vertex from the 2D source mesh<br /> ;geom3D.match :Contains periodic partners<br /> <br /> <br /> There is also one more file:<br /> <br /> ; geom3DHead.cnn<br /> <br /> Which lists the headers containing information on the size of the file each of the above connectivity files.<br /> <br /> == Using the outputted files ==<br /> The outputted files from MGEN now need to be prepared for Chef, this is done via &lt;code&gt;matchedNodeElmReader&lt;/code&gt;. The provided example will be for a build on Cooley.<br /> <br /> First, the enviroment needs to be prepared via setting &lt;code&gt;SIM_LICENSE_FILE&lt;/code&gt; and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt;. Examples of this are:<br /> <br /> <br /> &lt;code&gt;<br /> export SIM_LICENSE_FILE=/eagle/PHASTA_aesp/SCOREC-CORE/deps/Simmetrix/UCBoulder<br /> <br /> export LD_LIBRARY_PATH=/eagle/PHASTA_aesp/SCOREC-CORE/deps/16.0-220326/lib/x64_rhel_gcc48/psKrnl/:$LD_LIBRARY_PATH<br /> &lt;/code&gt;<br /> <br /> <br /> From here, &lt;code&gt;matchedNodeElmReader&lt;/code&gt; can be run with:<br /> <br /> &lt;code&gt;<br /> mpirun -f /var/tmp/cobalt.2137783 -np &lt;np&gt; -genvall /eagle/PHASTA_aesp/SCOREC-CORE/build_gtvertCorruption/test/matchedNodeElmReader ../geom3D.cnndt ../geom3D.coord ../geom3D.match ../geom3D.class ../geom3D.fathr NULL ../geom3DHead.cnn outModel.dmg outModel/<br /> &lt;/code&gt;<br /> <br /> Where &lt;np&gt; should be replaced by the same number as used for npart when running MGEN.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1818 2022-08-23T17:46:25Z

<p>Prte0550: Initial HPSS setup info</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Basics ==<br /> <br /> &lt;code&gt;hsi&lt;/code&gt; is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.<br /> <br /> If you need to retrieve data from tape and put it back on the &quot;local&quot; system, the &lt;code&gt;get&lt;/code&gt; and &lt;code&gt;cget&lt;/code&gt; commands act in the same way as &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; but in reverse.<br /> <br /> == Troubleshooting ==<br /> If you are a first time user of HPSS, you will likely get an error regarding a key file. This is something that must be taken care of by ALCF support (support@alcf.anl.gov). Simply email them with your ALCF username and state that you need access set up for HPSS.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1817 2022-08-22T21:04:34Z

<p>Prte0550: Added get information</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Basics ==<br /> <br /> &lt;code&gt;hsi&lt;/code&gt; is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.<br /> <br /> If you need to retrieve data from tape and put it back on the &quot;local&quot; system, the &lt;code&gt;get&lt;/code&gt; and &lt;code&gt;cget&lt;/code&gt; commands act in the same way as &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; but in reverse.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1816 2022-08-22T21:03:02Z

<p>Prte0550: </p> <hr /> <div>ALCF's High Performance Storage System (HPSS) is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Basics ==<br /> <br /> &lt;code&gt;hsi&lt;/code&gt; is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1815 2022-08-22T21:02:42Z

<p>Prte0550: </p> <hr /> <div>ALCF's High Performance Storage System (HPSS) is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Basics ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1814 2022-08-22T21:02:17Z

<p>Prte0550: </p> <hr /> <div>ALCF's High Performance Storage System (HPSS) is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1813 2022-08-22T19:47:54Z

<p>Prte0550: </p> <hr /> <div>ALCF's High Performance Storage System (HPSS) system is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> * &lt;code&gt;put&lt;/code&gt;<br /> * &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1812 2022-08-22T17:41:13Z

<p>Prte0550: List fix</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) system is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> # &lt;code&gt;put&lt;/code&gt;<br /> # &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1811 2022-08-22T17:40:54Z

<p>Prte0550: Finalization of initial version</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) system is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will place you into your home HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> While standard use cases of &lt;code&gt;hsi&lt;/code&gt; will be covered below, more documentation (that is more thorough and helpful than the ALCF documentation is available here: [https://docs.nersc.gov/filesystems/archive/]. <br /> <br /> == Archiving of Data ==<br /> <br /> Once the destination directory for the data has been created and/or navigated to, there are a few options and considerations to actually archive the data. These are most notably:<br /> <br /> # &lt;code&gt;put&lt;/code&gt;<br /> <br /> # &lt;code&gt;cput&lt;/code&gt;<br /> <br /> &lt;code&gt;put&lt;/code&gt; is the most basic archiving tool, and will overwrite any versions of the files being archived already on HPSS. &lt;code&gt;cput&lt;/code&gt; is a conditional version of &lt;code&gt;put&lt;/code&gt; that will only overwrite files if there is a newer version &quot;locally&quot; compared to the file already on HPSS. This makes &lt;code&gt;cput&lt;/code&gt; the tool of choice for updating partially archived datasets, but due to its otherwise similar functionality to &lt;code&gt;put&lt;/code&gt;, it is also the recommended default command to use.<br /> <br /> Both &lt;code&gt;put&lt;/code&gt; and &lt;code&gt;cput&lt;/code&gt; have similar syntax, and the following will cover both, but &lt;code&gt;cput&lt;/code&gt; will be used as an example. It is assumed that the &lt;code&gt;hsi&lt;/code&gt; command has already been run to enter the &lt;code&gt;hsi&lt;/code&gt; utility before attempting the following.<br /> <br /> Simple usage to store a single file is:<br /> <br /> &lt;code&gt;cput &lt;filename&gt;&lt;/code&gt;<br /> <br /> To change the name of a file as it is archived:<br /> <br /> &lt;code&gt;cput &lt;filename&gt; : &lt;newFilename&gt;&lt;/code&gt;<br /> <br /> Whole directories can be stored using:<br /> <br /> &lt;code&gt;cput -R &lt;dirName&gt;&lt;/code&gt;<br /> <br /> If you with to keep the parent directory intact, or with:<br /> <br /> &lt;code&gt;cput -R &quot;*&quot;&lt;/code&gt;<br /> <br /> If you simply want to move the contents of a directory and everything beneath it. Simply be mindful of your &quot;local&quot; directory location when choosing between these options.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1810 2022-08-22T16:00:36Z

<p>Prte0550: External link fix</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) system is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in ALCF documentation ([https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]) that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will dump you into your &quot;home&quot; HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> == Archiving of Data ==<br /> <br /> == Example Usage ==</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=ALCF/Archiving_Data_at_ALCF&diff=1809 2022-08-22T15:59:07Z

<p>Prte0550: Initial page creation and outlining</p> <hr /> <div>ALCF's High Performance Storage System (HPSS) system is a robotic tape drive system used for large amount of archival data storage that will not be often accessed. The system has two interfaces listed in [[documentation|https://www.alcf.anl.gov/support-center/theta/using-hpss-theta]] that can be used, &lt;code&gt;hsi&lt;/code&gt; and &lt;code&gt;htar&lt;/code&gt;. In this wiki, the &lt;code&gt;hsi&lt;/code&gt; interface will be focused on. <br /> <br /> == HSI Overview ==<br /> <br /> HSI is a utility to interface with the HPSS system. It looks and operates much like the typical bash command lines that we are used to, but with some added complexities. When you enter &lt;code&gt;hsi&lt;/code&gt;, the system will dump you into your &quot;home&quot; HPSS space at &lt;code&gt;/home/username&lt;/code&gt;. &lt;code&gt;hsi&lt;/code&gt; keeps track of both your location in this HPSS space and also your location in the &quot;local&quot; system that you are running &lt;code&gt;hsi&lt;/code&gt; from. The &lt;code&gt;hsi&lt;/code&gt; system will automatically set the &quot;local&quot; directory location to be the location that you entered the utility from. <br /> <br /> Navigation through HPSS operates the same as a normal command line, with &lt;code&gt;ls&lt;/code&gt; &lt;code&gt;cd&lt;/code&gt;, and &lt;code&gt;mkdir&lt;/code&gt; among others being valid commands for navigation through HPSS. If you need to navigate though the &quot;local&quot; directories though, this can still be done by appending an &quot;l&quot; to the front of these standard commands (i.e. &lt;code&gt;lls&lt;/code&gt; &lt;code&gt;lcd&lt;/code&gt;, and &lt;code&gt;lmkdir&lt;/code&gt;). This can be useful if you entered &lt;code&gt;hsi&lt;/code&gt; at the incorrect point or wish to archive data in multiple locations.<br /> <br /> It should be noted that &lt;code&gt;hsi&lt;/code&gt; does support tab-complete or up-arrowing for past commands. It is recommended that you enter the utility with a defined plan in order to reduce the amount of annoyance that the lack of these luxuries can cause.<br /> <br /> == Archiving of Data ==<br /> <br /> == Example Usage ==</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1808 2022-08-18T18:40:14Z

<p>Prte0550: More initial info - output and afterwards oriented</p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;(location)&lt;/code&gt; as of (date)<br /> <br /> <br /> == Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Convert]] tool for unstructured grids.<br /> <br /> Once the mesh files are in place, MGEN can be run with &lt;code&gt;./tm3Extrude&lt;/code&gt; as usual. The code will ask for inputs for zmin, zmax, numelz, and npart. These should be entered in a single string with spaces in between the values before hitting in order to continue code execution.<br /> <br /> == Outputs ==<br /> MGEN will write its outputs to the same working directory that the executable and source mesh files are in. There are multiple file types written, most with a suffix of a number to denote the part number of that file. The different parted files and their purposes are as follows:<br /> <br /> ;geom3D.class :Classification file describing what type of geometric entity each point lies on (vertex, edge, face, volume)<br /> ;geom3D.cnndt :Connectivity of the elements <br /> ;geom3D.coord :Node coordinates<br /> ;geom3D.fathr :Parent vertex from the 2D source mesh<br /> ;geom3D.match :Contains periodic partners<br /> <br /> <br /> There is also one more file:<br /> <br /> ; geom3DHead.cnn<br /> <br /> Which lists the headers containing information on the size of the file each of the above connectivity files.<br /> <br /> == Using the outputted files ==<br /> The outputted files from MGEN now need to be prepared for Chef, this is done via &lt;code&gt;matchedNodeElmReader&lt;/code&gt;. The provided example will be for a build on Cooley.<br /> <br /> First, the enviroment needs to be prepared via setting &lt;code&gt;SIM_LICENSE_FILE&lt;/code&gt; and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt;. Examples of this are:<br /> <br /> <br /> &lt;code&gt;<br /> export SIM_LICENSE_FILE=/eagle/PHASTA_aesp/SCOREC-CORE/deps/Simmetrix/UCBoulder<br /> <br /> export LD_LIBRARY_PATH=/eagle/PHASTA_aesp/SCOREC-CORE/deps/16.0-220326/lib/x64_rhel_gcc48/psKrnl/:$LD_LIBRARY_PATH<br /> &lt;/code&gt;<br /> <br /> <br /> From here, &lt;code&gt;matchedNodeElmReader&lt;/code&gt; can be run with:<br /> <br /> &lt;code&gt;<br /> mpirun -f /var/tmp/cobalt.2137783 -np &lt;np&gt; -genvall /eagle/PHASTA_aesp/SCOREC-CORE/build_gtvertCorruption/test/matchedNodeElmReader ../geom3D.cnndt ../geom3D.coord ../geom3D.match ../geom3D.class ../geom3D.fathr NULL ../geom3DHead.cnn outModel.dmg outModel/<br /> &lt;/code&gt;<br /> <br /> Where &lt;np&gt; should be replaced by the same number as used for npart when running MGEN.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=MGEN_Extrude&diff=1807 2022-08-18T16:02:21Z

<p>Prte0550: Initial edits</p> <hr /> <div>MGEN is a tool in the meshing workflow that takes a 2D source mesh and extrudes it in the third dimension based off of user input. The tool was originally created for use on structured grids on the Boeing bump, but has since been generalized for use in unstructured setups.<br /> <br /> == Basic Overview ==<br /> <br /> MGEN code is stored in &lt;code&gt;tm3Extrude.f&lt;/code&gt; and written in FORTRAN. The code takes in a source 2D mesh, z-coordinates to extrude between, the number of elements to populate the extrusion with, and the number of partitions to write the mesh to. <br /> <br /> Partitioning in MGEN is simply a method to reduce the cost of initial runs of Chef, but is not a replacement for the initial configuring that Chef does (via 1-1-Chef). Parting in MGEN simply allows the first run of Chef to be in parallel (i.e. 8-8-Chef). Starting Chef from parallel is most important on large grids that would take prohibitively long to run though Chef in serial.<br /> <br /> The most current copy of the code is available at &lt;code&gt;(location)&lt;/code&gt; as of (date)<br /> <br /> <br /> == Usage ==<br /> <br /> Once a suitable version of &lt;code&gt;tm3Extrude.f&lt;/code&gt; has been located and moved to a working directory, it first needs to be complied if this has not already been done. The FORTRAN compiler to compile &lt;code&gt;tm3Extrude.f&lt;/code&gt; should be the same version that was/will be used to compile the version of Chef to be used later in the meshing pipeline in order to reduce the risk of complications.<br /> <br /> Once a compiler version is selected and added using &lt;code&gt;soft add&lt;/code&gt; or &lt;code&gt;module load&lt;/code&gt; (depending on the system), it can be compiled. As an example, if using &lt;code&gt;gcc-6.3.0&lt;/code&gt; on Cooley compiling would look like:<br /> <br /> &lt;code&gt;<br /> soft add +gcc-6.3.0<br /> <br /> gfortran -03 tm3Extrude.f -o tm3Extrude<br /> &lt;/code&gt;<br /> <br /> Once the code is compiled, the working directory needs to be prepared to run MGEN. MGEN needs the source 2D mesh in the form of &lt;code&gt;geom.crd&lt;/code&gt; and &lt;code&gt;geom.cnn&lt;/code&gt; files in the same directory as the compiled code. These source files can be produced from scratch with MATLAB for structured grids, or through the use of [[Getting Started with Simmodeler|Simmetrix]] and the [[Convert]] tool for unstructured grids.</div>

Prte0550 https://fluid.colorado.edu/wiki/index.php?title=Chef&diff=1600 2021-06-17T21:58:31Z

<p>Prte0550: Adding a link to better introductory information for new users</p> <hr /> <div>'''Chef''' is the mesh partitioning and file preparation program used to create the files read by [[PHASTA]]. It is a part of the [[SCOREC-core]] tools.<br /> <br /> Basic usage information is located in the [[Level 1 Partition]] page, while other related pages can be found in the [[:Category:Chef|Chef Category]].<br /> <br /> [[Category: Chef]]</div>

Prte0550