View source for PhParAdapt/Simmetrix

The following page provides details for performing mesh adaptation using the PhParAdapt tool with Simmetrix routines. 

== Initial Notes to User ==
* The solution migration feature is currently broken and thus requires other means for transferring the solution to new meshes that are created from phParAdapt (e.g. solution interpolation in Paraview)
* To date, the steps described herein have only been tried with the following restrictions:
** Tetrahedral elements
*** Extruded element types apparently can be used but require removal of any extrusion constraints in the mesh. Below are some incomplete details regarding that process for reference. See  [[#Removal_of_Extrusion_Constraint |Removal of Extrusion Constraint]].
** Serial case
** Geometries created using SimModeler7.0-190626
** The version of phParAdapt noted here uses only the 6th entry of the error field for identifying regions for adaptation

== Adaptation Process ==

=== Creating Initial Restart and Error Files ===
Beginning inside of the 1-1-Chef directory,
 <nowiki>mkdir 1-1-phParAdapt
cd 1-1-phParAdapt </nowiki>

Create soft links to the geom files above the 1-1-Chef directory,
 <nowiki>ln -s ../../geom.smd
ln -s ../../geom.sms
ln -s ../../geom_nat.x_t </nowiki>

Create an <code>adapt.inp</code> file using the template [[#Adapt.inp_(non-adaptation_step) | Adapt.inp (non-adaptation step)]]. Keep <code>adaptFlag</code> set to zero as the purpose of running phParAdapt here is to create restart.<timeStepNumber>.1 file associated with a new mesh format that can be used for adaptation. 

Run phParAdapt in this directory, which for this specific example was:

<code>mpirun -np 1 /projects/tools/Simmetrix.develop_19/phParAdapt-Sim/phParAdapt-Sim19/bin/x86_64_linux/phParAdapt-parasolid-openmpi-O 2>&1 | tee phParAdapt.log</code>


After completion, the 1-procs_case and mesh_parts.sms directories should now be present. If a solution has already been generated for a different mesh on the same geometry, it can now be interpolated onto the new restart.<#>.1 file that was generated. If no solution exists, run Phasta using this restart file until the desired solution state has been achieved. Be sure to set <code>Print Error Indicators: True</code> in the solver.inp file so the error fields are saved in the final restart file. Note, error fields are not printed in intermediate restart files. Before proceeding, check that the error fields exist using <code>grep -a " : <" restart.<#>.1</code>

=== Performing the Adaptation Step ===
Within the 1-1-phParAdapt directory, 
 <nowiki>mkdir A1-phParAdapt
cd A1-phParAdapt
ln -s ../geom.smd
ln -s ../geom_nat.x_t
ln -s ../mesh_parts.sms geom.sms
ln -s geom.sms parts.sms
ln -s ../1-procs_case/restart.1.1
ln -s restart.1.1 errors.1.1 </nowiki>

In the above step, it is assumed that the solution was interpolated onto restart.1.1. If that was not the case and the solution was advanced until a desired step was reached, replace restart.1.1 & errors.1.1 with restart.<#>.1 & errors.<#>.1 corresponding to the desired solution step number with error fields.

Create an adapt.inp file using [[#Adapt.inp_(adaptation step) |Adapt.inp_(adaptation step)]] as a template. Change the <code>timeStepNumber</code> parameter to correspond to the restart file that was linked into the current directory. Make sure the <code>adaptFlag</code> parameter is set to one and change additional threshold flags to control the result of the refinement.

Run phParAdapt using the mpirun command from the previous section. A directory corresponding to the <code>timeStepNumber</code> parameter should have been created with a mesh_parts.sms folder inside upon successful completion.

=== Generate Restart File for Refined Mesh ===
From within the A1-phParAdapt directory, execute the following (noting to replace <#> with the appropriate time step number):
 <nowiki>cd <#>
mkdir 1-1-phParAdapt
cd 1-1-phParAdapt
ln -s ../../../geom.smd
ln -s ../../../geom_nat.x_t
ln -s ../mesh_parts.sms geom.sms
cp ../../../adapt.inp .</nowiki>

In the above list of commands, the adapt.inp file from the non-adaptation step was copied into the current directory. Run phParAdapt using the same mpirun command as above to generate the 1-procs_case folder and the new restart file with the refined mesh. Again, since the solution migration feature is not working, the restart file will contain the initial solution and no error fields. A previously obtained solution can now be interpolated to the newly created one or PHASTA can be run in this directory to obtain a solution on the new mesh. 

In order to perform a second (or more) adaptation step, create a restart file in the 1-procs_case folder in the current directory (either by interpolation or running PHASTA) and make sure it contains the error fields. Create a new directory <code>A2-phParAdapt</code> at this level and repeat the above steps. Be sure to use the time step number moving forward that is associated with the restart file just mentioned.

== File Examples ==

The file examples below are what were used for this specific example and may contain parameter flags that may need to be changed or are not used at all.

=== Adapt.inp (non-adaptation step) ===
 <nowiki>numberSolutionVars 6
numberErrorVars 10 
refWeights 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
refThreshold 0.01
globalP 1
timeStepNumber 1 
numelX 0
NSFTAG -1
ensa_dof 6
attributeFileName geom.smd
meshFileName geom.sms
modelFileName geom_nat.x_t
Idirection 0
BYPASS 0
zScale 0
adaptFlag 0 
errorName 0
SONFATH 0
lStart 0
rRead 6
rStart 0 
AdaptStrategy 5
AdaptFactor 0.00001
AdaptOption 11
hmax 0.04
hmin 0.000052083
multipleRestarts 0
Periodic 0
prCD 0
timing 0
wGraph 0
phastaVersion 1.9.5
old_format 0
FortFormFlag 0
outputFormat binary
CUBES 0
internalBCNodes 0
version UNKNOWN
WRITEASC 0
phastaIO 1
numTotParts 1
SolutionMigration 0
DisplacementMigration 0
isReorder 0 
isMCOPI 0
isBLAdapt 0
isThickAdapt 0
isSizeLimit 1
MaxLimitFact 2
MinLimitFact 2
rho 1.225
mu 1.7825e-5
dwalMigration 0
buildMapping 1 </nowiki>



=== Adapt.inp (adaptation step) ===
 <nowiki>eviscScal 0
pgrad 0
ub 1.45
lb 0.70
numSplit 5
sizeRatio 0.35 # confirmed that 0.25 does a 4x ref
numSmooth  0  #confirmed that this gets used  
ratioThresh 0.75   #  this is h_new/h_orig from isotropic days
localAdapt 1    # makes setLocal force adaptivity only around nodes set size
AnisoSimmetrix 4 # > 0 use Simmetrix, else ours 1 does largest length reduction by sizeRatio, 2 does both does medium and largest reduction by sizeRatio
coarsenMode 0 # disables coarsening see coarsenMode docs for 1, and 2
numberSolutionVars 6
numberErrorVars 10 
refWeights 1 1 1 1 1 1 1 1 1 1
refThreshold 0.01
globalP 1
timeStepNumber 1 
numelX 0
NSFTAG -1
ensa_dof 6
attributeFileName geom.smd
meshFileName  geom.sms
modelFileName geom_nat.x_t
Idirection 0
BYPASS 0
zScale 0
adaptFlag 1 
errorName 0
SONFATH 0
lStart 0
rRead 6
rStart 0 
AdaptStrategy 5
AdaptFactor 0.5e-4
AdaptOption 11
hmax 1e6
hmin 1e-5
multipleRestarts 0
Periodic 0
prCD 0
timing 0
wGraph 0
phastaVersion 1.9.5
old_format 0
FortFormFlag 0
outputFormat binary
CUBES 0
internalBCNodes 0
version UNKNOWN
WRITEASC 0
phastaIO 0
numTotParts 1
SolutionMigration 1
DisplacementMigration 0
isReorder 0 
isMCOPI 0
isBLAdapt 0
isThickAdapt 0
isSizeLimit 0
MaxLimitFact 2
MinLimitFact 2
rho 1.225
mu 1.7825e-5
dwalMigration 0
buildMapping 1 </nowiki>


== Notes on Debugging ==
If debugging of phParAdapt is desired, replace the normal <code>mpirun</code> command with the following:

<code>mpirun -np 1 -tv /projects/tools/Simmetrix.develop_19/phParAdapt-Sim/phParAdapt-Sim19/bin/x86_64_linux/phParAdapt-parasolid-openmpi</code>

where the <code>-O</code> has been removed from the path to the executable to call the build version that was not optimized. Note that this requires phParAdapt to have been built at some point prior with the <code>Debug</code> flag.

== Notes on Folder Structure ==
Below is an overview of the folder structure to reference during the adaptation process:
# [ geom.smd, geom.sms, geom_nat.x_t ] 
# 1-1-Chef
## 1-1-phParAdapt
### adapt.inp
### run_phParAdapt.sh
### 1-procs_case
#### geombc.dat.1
#### restart.<#>.1
### mesh_parts.sms
### A1-phParAdapt
#### restart.<#>.1
#### errors.<#>.1
#### [geom.smd, geom.sms, geom_nat.x_t, parts.sms]
#### adapt.inp
#### <#>     ( step number directory )
##### mesh_parts.sms
##### 1-1-phParAdapt
###### [geom.smd, geom.sms, geom_nat.x_t]
###### adapt.inp
###### 1-procs_case ( restart files with adapted mesh )

== Removal of Extrusion Constraint ==
Riccardo has used a procedure in the past to remove the extrusion constraint from a mesh that enabled adaptation to be performed on extrusion-type elements. 

The following directory contains an example of this process:
<code> /projects/tools/Models/NASAWingBodyJunction/RajMeshFine/Mesh/1-1-phParAdapt/RemExtrusion/ </code>

The executable that is called for removal of the constraint is a part of SCOREC-core, found at <code> <path/to/SCOREC-core/build/dir>/test/rm_extrusion </code> and the source code, in case edits are required, is found at <code> <path/to/SCOREC-core/source/dir>/core/test/rm_extrusion.cc </code>