PhParAdapt/SCOREC

---

Download and compile (viz nodes)

This sections is inspired from <http://redmine.scorec.rpi.edu/projects/pumi/wiki/Build_Instructions>. Please read it first before continuing, as some information on this wiki page is not repeated. The SCOREC tools now build against CMake. Only Parma, the PHASTA utilities and phParAdapt still build against Autotools. From a terminal, run the following commands:

 soft add +cmake
 soft add +git
 cd ~
 mkdir SCOREC.develop
 cd SCOREC.develop
 git clone ssh://your_username@jumpgate-phasta.colorado.edu/users/mrasquin/SCOREC.develop/compilation 

The git command will download these instructions, along with a checkout and compile script for Parma and phParAdapt. The next commands to compile the whole stack with the GNU compiler on the viz nodes are:

 soft add +gcc-4.8.1
 soft add +openmpi-gnu
 mkdir install-gcc-O2 //all the libraries and executable will go there
 mkdir build-gcc-O2 // this is where you will run the cmake commands
 cd build-gcc-O2  // Go to the your build directory to execute the next cmake command
 cmake ../CMake.SCOREC -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=/users/mrasquin/SCOREC.develop/install-gcc-O2 -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx 

The last cmake command creates the Makefiles. Do not forget to update the path to PREFIX according to your install directory. There are additional parameters and options that you can set up but this should allow you to get started on the viz nodes. If you want to see all the parameters, you can replace cmake by ccmake (note the 2 consecutive "c") in the command above. By default, it generates the Makefile with meshmodel. See the redmine link above about how to compile with parasolid or other geometric model instead of meshmodel. The cmake command also downloads the parmetis package inside the build-gcc-O2 directory. If you want to use 64 bit integer in parmetis in order to avoid integer overflow while partitioning a mesh with more than 100M elements par part (typically for the initial partitioning from 1 to n), edit

 ./parmetis-4.0.2/metis/include/metis.h

and set

 #define IDXTYPEWIDTH 64

Now, compile the SCOREC tools (in serial only for now - do not try make -j 8 for instance):

 make 2>&1 | tee log_cmake_make.dat #The output is saved in log_cmake_make.dat
 make install 2>&1 | tee log_cmake_install.dat #The output is saved in log_cmake_install.dat
 cd .. #back to SCOREC.develop

Now you should have the pumi tools installed in SCOREC.develop/install-gcc-O2. It is time to download and compile the Parma library. From SCOREC.develop,

 mkdir Autotools.SCOREC
 cd Autotools.SCOREC
 cp ../compilation/parma_install.sh .

Edit and update the script parma_install.sh. In particular, edit PREFIX to the same path used in the cmake command above. Other parameters you need to update are the compilation options (DEBUG), the svn username if you have one, etc. Then:

 ./parma_install.sh checkout_all 2>&1 | tee log_autotools_parma_checkout.dat
 ./parma_install.sh setup_all 2>&1 | tee log_autotools_parma_setup.dat
 ./parma_install.sh intall_all 2>&1 | tee log_autotools_parma_install.dat

For phParAdapt (and PHASTA utilites), the procedure is similar:

 cp ../compilation/phParAdaptBuild.sh
 edit path to PREFIX in parma_install.sh + other parameters
 ./phParAdaptBuild.sh checkout_all 2>&1 | tee log_autotools_phparadapt_checkout.dat
 ./phParAdaptBuild.sh setup_all 2>&1 | tee log_autotools_phparadapt_setup.dat
 ./phParAdaptBuild.sh install_all 2>&1 | tee log_autotools_phparadapt_install.dat

Your phParAdapt executable should be in ~/SCOREC.develop/install-gcc-O2/bin/phParAdapt

Convert geom_ver63.sms to FMDB format

• This can be done in same directory in which you created geom_ver63.sms. You first need to convert geom_ver63.sms to FMDB format. The executable that will do this is at /users/jema6380/runSimMesh2FMDB.sh
• Make a link from the output file, geom_FMDB.sms, to geom.sms (ln -s geom_FMDB.sms geom.sms)

Create geom.spj

• If you have not done so already, create geom.spj. This is not done with Simulation Maker as was done previously, but can simply be done with a vim editor. See the file /users/jema6380/Models/Boeing/BetaScale2CaltechTunnel/CT_Open32_B0_D30_U20/9-A0-24jets/geom.spj for some guidance. As a quick example, if your boundary condition is a velocity of 20 i_hat m/s (comp3) on the inlet face (which let's say has a face number of 1651), then you would enter:

  comp3: 1651 2 20 1 0 0

where the "2" designates that we are talking about a face, which has 2 dimensions; the "20" is the speed; and the "1 0 0" is the direction.

• Michel created a script that will make this process faster, which is desirable if you have a large number of faces. If you want to use the script, then first use the executable at /users/jema6380/getFaceList.sh to create a list of faces. From the terminal type:

  ./getFaceList.sh "input_file"

where "input_file" is the file that contains the face numbers, which presumably does not already list them consecutively. This might be BLattr.inp, for exmple.

• Now create a new directory called "FacesModelAttributes" and put all of the boundary conditions in .dat files. See the files in /users/jema6380/Models/Boeing/BetaScale2CaltechTunnel/CT_Open32_B0_D30_U20/Simplified_SPJ_file/FacesModelAttributes/ for guidance. Make sure all the .dat files have a name that begins with "attr_" .
• copy /users/jema6380/buildSimplifiedAttrFile.sh to the Simplified_SPJ_file directory and do:

  ./buildSimplifiedAttrFile.sh

• This should create a file called result.spj, which contains all of the boundary conditions. cd back to your working mesh directory and make a link from result.spj to geom.spj. Also make a link from result.spj to geomNOIC.spj

Partition geom_ver63.sms

• Make sure you have geom.xmt_txt in your working directory.