Compiling Chef Summit - Revision history

Jrwrigh: Redirected page to Chef: Building#Summit (RMACC)

2020-07-13T20:53:28Z

Redirected page to Chef: Building#Summit (RMACC)

Mabl7046 at 18:21, 13 July 2020

2020-07-13T18:21:24Z

Mabl7046 at 18:11, 13 July 2020

2020-07-13T18:11:09Z

Mabl7046 at 18:09, 13 July 2020

2020-07-13T18:09:35Z

Mabl7046 at 17:59, 13 July 2020

2020-07-13T17:59:53Z

Mabl7046 at 17:58, 13 July 2020

2020-07-13T17:58:25Z

Mabl7046 at 17:40, 13 July 2020

2020-07-13T17:40:05Z

Mabl7046 at 17:25, 13 July 2020

2020-07-13T17:25:30Z

Mabl7046 at 16:52, 13 July 2020

2020-07-13T16:52:34Z

Mabl7046 at 16:49, 13 July 2020

2020-07-13T16:49:02Z

← Older revision		Revision as of 20:53, 13 July 2020
Line 1:		Line 1:
−	~~==Copy Simmetrix Libraries from Viz Nodes==~~	+	#REDIRECT [[Chef:_Building#Summit (RMACC)]]
−
−	~~Note that in order for~~ Chef to operate, it must be compiled with the proper Simmetrix libraries. That is to say, there needs to be compatibility between Simmetrix and SCOREC-core for compilation of Chef to run smoothly. Please see https://fluid.colorado.edu/wiki/index.php/Building_SCOREC_Core for instructions on cloning the source code. An example (what worked for me) is provided throughout the wiki. From the viz nodes (/projects/tools/SimmetrixTest), I ran
−
−	~~scp -r 14.0-190626beta/ <username>@login.rc.colorado.edu:<path>~~
−
−	~~to copy over the directory 14.0-190626beta/ to Summit.~~
−
−
−	~~==Set Proper Environment==~~
−
−	~~To start with a clean slate, run~~
−
−	~~module purge~~
−
−	~~This unloads all currently loaded modules.~~
−
−	~~Next, run~~
−
−	~~module load gcc/6.1.0~~
−	~~module load openmpi/2.0.1~~
−
−	~~Note that /6.1.0 and /2.0.1 can be swapped with the intended versions of gcc and openmpi the user intends on using.~~
−
−	~~==Generate Partition Wrapper==~~
−
−	~~After the directory is copied over to~~ Summit ~~and the environment is set, a Partition Wrapper needs to be generated. Go to:~~
−
−	~~<path>/14.0-190626beta/code/PartitionWrapper~~
−
−	~~and open the file~~
−
−	~~Makefile.custom~~
−
−	Obviously, 14.0-190626beta/ is a placeholder and any version of Simmetrix (i.e. the version you used to create the mesh) will work. In this example, I used open openmpi (A High Performance Message Passing Library). I also used mpicc and mpicxx as MPI-C and MPI-C++ compilers respectively. If this is what is intended, in the Makefile, set the following variables:
−
−	~~PQUAL:= -openmpi~~
−	~~CC:=/curc/sw/openmpi/2.0.1/gcc/6.1.0/bin/mpicc~~
−	~~CXX:= /curc/sw/openmpi/2.0.1/gcc/6.1.0/bin/mpicxx~~
−
−	~~Notice the compatibility between the loaded modules and the directory structure above. Once this is set, in the terminal run~~
−
−	~~make -f Makefile.custom PARALLEL=openmpi~~
−
−	~~This will generate a file named libSimPartitionWrapper-openmpi.a. Then, run~~
−
−	~~chmod +x libSimPartitionWrapper-openmpi.a.~~
−
−	Once this is completed, copy this newly generated partition wrapper to ../../lib/x64_rhel7_gcc48/. Unfortunately, the instructions from this point forward will be less explicit as the best way to learn the process is to view example scripts and adapt these to fit your needs. From this point forward, it is assumed that a directory named SCOREC-core was created and the source code from github was properly cloned in this directory.
−
−	~~==Generating the Scripts to Compile Chef==~~
−
−	In order to run Chef, two scripts must be generated. The naming conventions of the scripts are typically doConfigure.sh and env_simmtrix_version. Below, an example doConfigure.sh script is shown. This script primarily executes the necessary cmake commands for compilation. It also runs the env_simmtrix_version script. In the example below, it calls env_14.0-190626. Note that both of these scripts are located in /projects/mabl7046/SCOREC-core/build_14.0-190626, or the directory where Chef is to be compiled. Note that the line -DMESHES="/projects/mabl7046/pumi-meshes" was copied from a script that implemented successfully on another machine. I recommend looking at scripts on the Viz nodes that worked for the version of Chef you are compiling and copying over the directory analogous to pumi-meshes to summit to implement your script.
−
−
−
−	~~#!/bin/bash -l~~
−	~~. /projects/mabl7046/SCOREC-core/build_14.0-190626/env_14.0-190626~~
−	~~#unload default modules~~
−	~~module purge~~
−	~~#load what we need~~
−	~~module load gcc/6.1.0~~
−	~~module load openmpi/2.0.1~~
−	~~module load cmake/3.14.1~~
−	~~### For Chef~~
−	~~cmake -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx -DSCOREC_CXX_WARNINGS=ON -DSCOREC_CXX_OPTIMIZE=ON -DSCOREC_CXX_SYMBOLS=ON -DPCU_COMPRESS=ON -DENABLE_ZOLTAN=ON -~~
−	~~DENABLE_SIMMETRIX=ON -DSIM_MPI="openmpi" -DSIM_PARASOLID=ON -DMDS_SET_MAX=1024 -DMDS_ID_TYPE=long -DIS_TESTING=OFF -DMESHES="/projects/mabl7046/pumi-meshes" -~~
−	~~DCMAKE_INSTALL_PREFIX=$PWD/install ../core~~
−	~~#-DMDS_ID_TYPE=int or long~~
−	~~sleep 3~~
−	~~make -j32~~
−	~~make -j32~~
−
−
−	Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, the SIM_LICENSE_FILE (denoted UCBoulder below) needed to be obtained. The libraries for Zoltan and parmetis also needed to be collected. The SIM_LICENSE_FILE was copied over from Cooley (~~Machine at ALCF~~) ~~although, I am sure that this can be obtained from the Viz nodes as well. To obtain the libraries for Zoltan and parmetis on Summit, run:~~
−
−	~~cp -r /projects/ASEN5119/SCOREC-core/deps/zoltan .~~
−	~~cp -r /projects/ASEN5119/SCOREC-core/deps/parmetis .~~
−
−	~~from the directory where you wish these to be. All this does is move these from a directory Ken created for his Boundary Layers course to a directory that is more convenient for personal use.~~
−
−
−	~~### For CMake.SCOREC~~
−	~~export buildRoot=/projects/mabl7046/SCOREC-core/~~
−	~~export simModVer=$buildRoot/deps/Simmetrix/14.0-190626beta/~~
−	~~export SIM_LICENSE_FILE=$buildRoot/deps/Simmetrix/UCBoulder~~
−	~~export CMAKE_PREFIX_PATH=$simModVer~~
−	~~export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$simModVer/lib/x64_rhel7_gcc48~~
−	~~export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$simModVer/lib/x64_rhel7_gcc48/psKrnl~~
−	~~#MIGHT NEED THIS export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:/projects/tools/Simmetrix.develop/simModSuite/PartitionWrapper/lib~~
−	~~### For Zoltan and parmetis~~
−	~~export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$buildRoot/deps/zoltan/3.83-gnu49-ompi210/~~
−	~~export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$buildRoot/deps/parmetis/4.0.3-gnu492-ompi210~~
−	~~# the following is to give the path to PartitionWrappers that have not been installed in the Simmetrix library~~
−	~~export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$simModVer/code/PartitionWrapper/lib~~
−
−
−	~~==Final Notes==~~
−	Once the scripts are set up, simply run ./doConfigure.sh to compile the code. Please be patient with the process as it is highly unlikely that it will compile successfully the first time you try. Be sure to pay close attention to your directory structure when setting your scripts up and be cognizant of the dependencies between programs.

@@ Line 73: / Line 73: @@
-Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, I needed to obtain the SIM_LICENSE_FILE (denoted UCBoulder below) as well as the libraries for Zoltan and parmetis. The SIM_LICENSE_FILE was copied over from Cooley, although I am sure that this can be obtained from the Viz nodes as well. To obtain the libraries for Zoltan and parmetis on Summit, I ran:
+Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, the SIM_LICENSE_FILE (denoted UCBoulder below) needed to be obtained. The libraries for Zoltan and parmetis also needed to be collected. The SIM_LICENSE_FILE was copied over from Cooley (Machine at ALCF) although, I am sure that this can be obtained from the Viz nodes as well. To obtain the libraries for Zoltan and parmetis on Summit, run:
   cp -r /projects/ASEN5119/SCOREC-core/deps/zoltan .
   cp -r /projects/ASEN5119/SCOREC-core/deps/parmetis .
-from the directory where I wished these to be. All this did was move these from a directory Ken created for his Boundary Layers course to a directory that was more convenient for my use.
+from the directory where you wish these to be. All this does is move these from a directory Ken created for his Boundary Layers course to a directory that is more convenient for personal use.
   ### For CMake.SCOREC
@@ Line 96: / Line 97: @@
 ==Final Notes==
-Once the scripts are set up, simply run ./doConfigure.sh to compile the code. Please be patient with the process as it is highly unlikely that it will compile successfully the first time you try. Be sure to play close attention to your directory structure when setting your scripts up and be cognizant of the dependencies between programs.
+Once the scripts are set up, simply run ./doConfigure.sh to compile the code. Please be patient with the process as it is highly unlikely that it will compile successfully the first time you try. Be sure to pay close attention to your directory structure when setting your scripts up and be cognizant of the dependencies between programs.

← Older revision		Revision as of 18:11, 13 July 2020
Line 96:		Line 96:

	==Final Notes==		==Final Notes==
−	Once the scripts are set up, simply run ./doConfigure.sh to compile the code. Please be patient with the process as it is highly unlikely that it will compile successfully the first time you try. Be sure to play ~~cla\ose~~ attention to your directory structure when setting your scripts up.	+	Once the scripts are set up, simply run ./doConfigure.sh to compile the code. Please be patient with the process as it is highly unlikely that it will compile successfully the first time you try. Be sure to play close attention to your directory structure when setting your scripts up and be cognizant of the dependencies between programs.

@@ Line 47: / Line 47: @@
   chmod +x libSimPartitionWrapper-openmpi.a.
-Once this is completed, copy this newly generated partition wrapper to ../../lib/x64_rhel7_gcc48/. Unfortunately, the instructions from this point forward will be less explicit as the best way to learn the process is to view example scripts and adapt these to fit your needs. From this point forward it is assumed that a directory named SCOREC-core was created and the source code from github was properly cloned in this directory.
+Once this is completed, copy this newly generated partition wrapper to ../../lib/x64_rhel7_gcc48/. Unfortunately, the instructions from this point forward will be less explicit as the best way to learn the process is to view example scripts and adapt these to fit your needs. From this point forward, it is assumed that a directory named SCOREC-core was created and the source code from github was properly cloned in this directory.
 ==Generating the Scripts to Compile Chef==
@@ Line 73: / Line 73: @@
-Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, I needed to obtain the SIM_LICENSE_FILE (denoted UCBoulder below) as well as the libraries for Zoltan and parmetis. The SIM_LICENSE_FILE was copied over from Cooley, although I am sure that this can be obtained from the Viz nodes as well. To obtain the ibraries for Zoltan and parmetis on Summit, I ran:
+Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, I needed to obtain the SIM_LICENSE_FILE (denoted UCBoulder below) as well as the libraries for Zoltan and parmetis. The SIM_LICENSE_FILE was copied over from Cooley, although I am sure that this can be obtained from the Viz nodes as well. To obtain the libraries for Zoltan and parmetis on Summit, I ran:
   cp -r /projects/ASEN5119/SCOREC-core/deps/zoltan .
@@ Line 93: / Line 93: @@
   # the following is to give the path to PartitionWrappers that have not been installed in the Simmetrix library
   export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:$simModVer/code/PartitionWrapper/lib

@@ Line 1: / Line 1: @@
 ==Copy Simmetrix Libraries from Viz Nodes==
-Note that in order for Chef to operate, it must be compiled with the proper Simmetrix libraries. That is to say, there needs to be compatibility between Simmetrix and SCOREC-core for compilation of Chef to run smoothly. Please see https://fluid.colorado.edu/wiki/index.php/Building_SCOREC_Core for instructions on building SCOREC-core. An example (what worked for me) is provided through out the wiki. From the viz nodes (/projects/tools/SimmetrixTest), I ran
+Note that in order for Chef to operate, it must be compiled with the proper Simmetrix libraries. That is to say, there needs to be compatibility between Simmetrix and SCOREC-core for compilation of Chef to run smoothly. Please see https://fluid.colorado.edu/wiki/index.php/Building_SCOREC_Core for instructions on cloning the source code. An example (what worked for me) is provided throughout the wiki. From the viz nodes (/projects/tools/SimmetrixTest), I ran
   scp -r 14.0-190626beta/ <username>@login.rc.colorado.edu:<path>

← Older revision		Revision as of 17:58, 13 July 2020
Line 73:		Line 73:


−	Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, I needed to obtain the SIM_LICENSE_FILE as well as the libraries for Zoltan and parmetis.	+	Below, the script env_14.0-190626 is given. As stated in the comments, this is to set the environment for CMake.SCOREC. In order to make this script work, I needed to obtain the SIM_LICENSE_FILE (denoted UCBoulder below) as well as the libraries for Zoltan and parmetis. The SIM_LICENSE_FILE was copied over from Cooley, although I am sure that this can be obtained from the Viz nodes as well. To obtain the ibraries for Zoltan and parmetis on Summit, I ran:
		+
		+	cp -r /projects/ASEN5119/SCOREC-core/deps/zoltan .
		+	cp -r /projects/ASEN5119/SCOREC-core/deps/parmetis .
		+
		+	from the directory where I wished these to be. All this did was move these from a directory Ken created for his Boundary Layers course to a directory that was more convenient for my use.

	### For CMake.SCOREC		### For CMake.SCOREC

@@ Line 51: / Line 51: @@
 ==Generating the Scripts to Compile Chef==
-In order to run Chef, two scripts must be generated. The naming conventions of the scripts are typically doConfigure.sh and env_simmtrix_version. Below, an example doConfigure.sh script is shown. This script primarily executes the necessary cmake commands for compilation. It also runs the env_simmtrix_version script. In the example below, it calls env_14.0-190626. Note that both of these scripts are located in /projects/mabl7046/SCOREC-core/build_14.0-190626, or the directory where Chef is to be compiled.
+In order to run Chef, two scripts must be generated. The naming conventions of the scripts are typically doConfigure.sh and env_simmtrix_version. Below, an example doConfigure.sh script is shown. This script primarily executes the necessary cmake commands for compilation. It also runs the env_simmtrix_version script. In the example below, it calls env_14.0-190626. Note that both of these scripts are located in /projects/mabl7046/SCOREC-core/build_14.0-190626, or the directory where Chef is to be compiled. Note that the line -DMESHES="/projects/mabl7046/pumi-meshes" was copied from a script that implemented successfully on another machine. I recommend looking at scripts on the Viz nodes that worked for the version of Chef you are compiling and copying over the directory analogous to pumi-meshes to summit to implement your script.
@@ Line 71: / Line 71: @@
   make -j32
   make -j32

← Older revision		Revision as of 16:52, 13 July 2020
Line 50:		Line 50:


−	[[File:DoConfigure.~~jpg~~]]	+	[[File:DoConfigure.sh.JPG]]

← Older revision		Revision as of 16:49, 13 July 2020
Line 50:		Line 50:


−	[[File:~~Example~~.jpg]]	+	[[File:DoConfigure.jpg]]