Partition

Chef

Chef an open source SCOREC tool that is this group's primary tool to partition a problem domain to many subdomains. This is done to allow any array of compute nodes to each focus on solving the problem in parallel. That is, to divide the problem domain into subdomains to give to different computational workers. When we perform this step, all that is needed for inputs are the, 1) SCOREC mesh constructed from the output of the conversion and meshing steps, and 2) the number of parts (subdomains) to divide the problem up into. A generic workflow for this step is described below and is a good place to start when desiring to complete the workflow for the first time. As the readers' studies continue, many aspects of the layout and modifiers will not be consistent with this onRanp documentation.

Partioning from serial via Viz nodes

Within a folder, name it Chef, softlink to the <case>.smd file, naming the link as geom.smd. Also, have the path to the directory containing the 0.smb file that was gathered from the previous steps. Create a 1-1-Chef subdirectory that contains an adapt.inp file and a runChef.sh bash script. Edit the runChef.sh script to look something like this:

#!/bin/bash
/usr/local/openmpi/1.10.6-gnu49-thread/bin/mpirun -np $1 /projects/tools/SCOREC-core/build-viz003/test/chef 2>&1 | tee chef.log

The adapt.inp file should contain specifications that looks as follows:

timeStepNumber 0 
ensa_dof 6
attributeFileName ../geom.smd
modelFileName ../geom.smd
meshFileName bz2:<path to .smb dir>
outMeshFileName bz2:mdsMesh_bz2/
adaptFlag 0
phastaIO 1
AdaptStrategy 7 
RecursiveUR 0
splitFactor 1
elementsPerMigration 10000
SolutionMigration 0
DisplacementMigration 0
Tetrahedronize 0
partitionMethod graph
dwalMigration 0
buildMapping 1

Pay particular attention to the splitFactor, attributeFileName, modelFileName, meshFileName bz2, and outMeshFileName bz2 specifications. These are often the first nobs that the new user learns to control. When it comes to first creating "checkpoint" files that are needed for the PHASTA executable. splitFactor will tell the Chef executable how many partitions each incoming part needs to be partitioned by. Immediately after the conversion step we want this to be set to 1 because we desire a single "checkpoint" file. attributeFileName will tell Chef where the file that contains the attributes of each geometry entity (region, face, line, and vertex info) is. Likewise, the modelFileName will tell Chef where the file that contains the geometry entity information is. For our purposes we will point Chef to the softlink that was created for the <case>.smd file. The meshFileName bz2 and outMeshFileName bz2 specifications determine the directory where Chef will look to gather the SCOREC mesh and where to put the partitioned SCOREC mesh.

When the bash script runCheh.sh is executed Chef should be able to load the input and output 1)the 1-procs_case directory containing numstart.dat, restart.dat.0.1, and geombc.1.dat, as well as 2) mdsMesh_bz2 directory containing 0.smb.