FDSI Summer Program/HONEE - Revision history

Jrwrigh: /* Degree-based settings */

2024-07-02T16:07:37Z

‎Degree-based settings

Jrwrigh: /* Degree-based settings */

2024-07-02T16:03:34Z

‎Degree-based settings

Jrwrigh: /* Recommendations for HONEE performance */

2024-06-21T15:52:01Z

‎Recommendations for HONEE performance

Jrwrigh: /* Recommendations for HONEE performance */

2024-06-21T15:51:16Z

‎Recommendations for HONEE performance

Jrwrigh: /* Recommendations for HONEE performance */

2024-06-21T15:49:38Z

‎Recommendations for HONEE performance

Jrwrigh at 15:35, 21 June 2024

2024-06-21T15:35:48Z

Jrwrigh at 23:05, 5 June 2024

2024-06-05T23:05:14Z

Jrwrigh: /* Restarting Simulations */

2024-06-05T22:58:44Z

‎Restarting Simulations

Jrwrigh at 20:47, 4 June 2024

2024-06-04T20:47:24Z

Jrwrigh at 16:31, 3 June 2024

2024-06-03T16:31:20Z

@@ Line 173: / Line 173: @@
 While for cubic elements (<code>-degree 3</code>) I've seen better performance with:
-  -amat_type shell -snes_lag_jacobian 15 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu
+  -amat_type shell -snes_lag_jacobian 15 -snes_lag_jacobian_persists true -pc_type asm -sub_pc_type lu
 Note that these flags were when running the vortex shedding example, so the optimal options may change for other problems.

@@ Line 169: / Line 169: @@
 For linear elements (<code>-degree 1</code>), I've had success with:
-  -amat_type -snes_lag_jacobian 5 -snes_lag_jacobian_persists false -snes_lag_preconditioner 20 -snes_lag_preconditioner_persists true -py_type asm -sub_pc_type lu
+  -amat_type -snes_lag_jacobian 5 -snes_lag_jacobian_persists false -snes_lag_preconditioner 20 -snes_lag_preconditioner_persists true -pc_type asm -sub_pc_type lu
 While for cubic elements (<code>-degree 3</code>) I've seen better performance with:

← Older revision		Revision as of 15:52, 21 June 2024
Line 174:		Line 174:

	-amat_type shell -snes_lag_jacobian 15 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu		-amat_type shell -snes_lag_jacobian 15 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu
		+
		+
		+	Note that these flags were when running the vortex shedding example, so the optimal options may change for other problems.

← Older revision		Revision as of 15:51, 21 June 2024
Line 173:		Line 173:
	While for cubic elements (<code>-degree 3</code>) I've seen better performance with:		While for cubic elements (<code>-degree 3</code>) I've seen better performance with:

−	-amat_type shell -snes_lag_jacobian 20 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu	+	-amat_type shell -snes_lag_jacobian 15 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu

← Older revision		Revision as of 15:49, 21 June 2024
Line 152:		Line 152:

	== Recommendations for HONEE performance ==		== Recommendations for HONEE performance ==
		+
		+	=== Compilation Flags ===
		+	When compiling a program, you can pass certain flags in order to try and make HONEE run faster.
		+	For the Cisco nodes, this can be done by running the following in your libCEED directory:
		+
		+	make configure OPT='-O3 -march=native -g -ffp-contract=fast -fopenmp-simd'
		+	make build/fluids-navierstokes -Bj
		+
		+	=== General Running Settings ===
		+	Some general settings that will help improve performance are to lower the solver tolerances.
		+	Specifically <code>-snes_rtol 1e-4 -ksp_rtol 1e-4</code> have been found to be pretty good choices.
		+	These may require tweaking if you run into divergence issues.
		+
		+	=== Degree-based settings ===
		+	Optimal solver settings can change based on what degree elements you work with (determined by the <code>-degree</code> flag).
		+	For linear elements (<code>-degree 1</code>), I've had success with:
		+
		+	-amat_type -snes_lag_jacobian 5 -snes_lag_jacobian_persists false -snes_lag_preconditioner 20 -snes_lag_preconditioner_persists true -py_type asm -sub_pc_type lu
		+
		+	While for cubic elements (<code>-degree 3</code>) I've seen better performance with:
		+
		+	-amat_type shell -snes_lag_jacobian 20 -snes_lag_jacobian_persists true -py_type asm -sub_pc_type lu

← Older revision		Revision as of 15:35, 21 June 2024
Line 149:		Line 149:

	Note the extra <code>3010</code> in the "inlet" definition. I'm not sure if this means that correct ID number for it should be 3010 instead of 1, but that's a possibility.		Note the extra <code>3010</code> in the "inlet" definition. I'm not sure if this means that correct ID number for it should be 3010 instead of 1, but that's a possibility.
		+
		+
		+	== Recommendations for HONEE performance ==

← Older revision		Revision as of 23:05, 5 June 2024
Line 106:		Line 106:

	Note that restarting from a binary file can only be done if the binary file was written by a simulation running at the same part count.		Note that restarting from a binary file can only be done if the binary file was written by a simulation running at the same part count.
		+
		+	== HONEE and Gmsh ==
		+
		+	=== GMSH/Grid requirements: ===
		+	# The grid must be hexahedron based (so deformed cubes). HONEE can run with tetrahedron grids, but that will require some minor code modifications which we've done before (IIRC, it's a single line of code to change). If anyone is interested in this, let me know and we can work on it.
		+	#* Note that HONEE cannot handle mixed meshes, so meshes with both hexahedron and tetrahedron grids. This is not a minor code change and is probably out of scope for this Summer Program
		+	# In the geo file, you need to specify <code>Physical Surface</code>s to identify the domain boundaries to apply boundary conditions to.
		+	# In the geo file, you need to specify a single <code>Physical Volume</code> that has the entire domain. So if you split the domain into multiple pieces for meshing purposes (as is done with the vortex shedding grids), then you need the <code>Physical Volume</code> to identify all the pieces.
		+
		+	=== YAML Setup: ===
		+	In the YAML file, you need to associate each <code>Physical Surface</code>s with a boundary condition. I describe this briefly in the video, but each <code>Physical Surface</code> as an ID number associated with it. In the YAML file, you need to associate that ID number with a BC choice.
		+	For example, in the <code>vortexshedding.yaml</code> file there is:
		+	# Boundary Settings
		+	bc_slip_z: 6
		+	bc_wall: 5
		+	bc_freestream: 1
		+	bc_outflow: 2
		+	bc_slip_y: 3,4
		+
		+	and the <code>cylinder.geo</code> has:
		+
		+	Physical Surface("inlet") = {102};
		+	Physical Surface("outlet") = {116};
		+	Physical Surface("top") = {80, 120};
		+	Physical Surface("bottom") = {36, 112};
		+	Physical Surface("cylinderwalls") = {94, 28, 50, 72};
		+	Physical Surface("frontandback") = {37, 1, 4, 103, 3, 81, 2, 59, 5, 125};
		+
		+	The "inlet" face is the first identified boundary, so it gets the ID number 1.
		+	In the YAML file, you see <code>bc_freestream: 1</code>, which sets this inlet boundary to be controlled by a freestream BC.
		+	Similar with the "outlet" face; it's the second specified boundary in the geo file, so we set <code>bc_outflow: 2</code>.
		+	And so on with the other faces.
		+
		+
		+	'''NOTE:''' When using the GMSH GUI to identify the Physical Surfaces, it will actually put something like this in the geo file:
		+	Physical Surface("inlet", 3010) = {3005};
		+	Physical Surface("outlet", 3011) = {3003};
		+	Physical Surface("top", 3012) = {3004};
		+	Physical Surface("bottom", 3013) = {3002};
		+	Physical Surface("nacawalls", 3014) = {3008, 3006, 3007};
		+	Physical Surface("frontandback", 3015) = {3009, 3001};
		+
		+	Note the extra <code>3010</code> in the "inlet" definition. I'm not sure if this means that correct ID number for it should be 3010 instead of 1, but that's a possibility.

@@ Line 103: / Line 103: @@
 === Restarting Simulations ===
 To restart a simulation from a previous simulation's result, they need to be saved to a <code>*.bin</code> file.
-This is done using the `-continue` flag, which will load the file set by `-continue_filename`.
+This is done using the <code>-continue</code> flag, which will load the file set by <code>-continue_filename</code>.
 Note that restarting from a binary file can only be done if the binary file was written by a simulation running at the same part count.

← Older revision		Revision as of 20:47, 4 June 2024
Line 100:		Line 100:
	=== Notable Input Flags ===		=== Notable Input Flags ===
	* <code>-ts_monitor_solution</code>: This will save the results of a simulation to a file. Example: <code>-ts_monitor_solution cgns:flow_visualization.cgns</code> will save the results to a file called <code>flow_visualization.cgns</code>		* <code>-ts_monitor_solution</code>: This will save the results of a simulation to a file. Example: <code>-ts_monitor_solution cgns:flow_visualization.cgns</code> will save the results to a file called <code>flow_visualization.cgns</code>
		+
		+	=== Restarting Simulations ===
		+	To restart a simulation from a previous simulation's result, they need to be saved to a <code>*.bin</code> file.
		+	This is done using the `-continue` flag, which will load the file set by `-continue_filename`.
		+
		+	Note that restarting from a binary file can only be done if the binary file was written by a simulation running at the same part count.

← Older revision		Revision as of 16:31, 3 June 2024
Line 97:		Line 97:
	* [https://petsc.org/main/manualpages/SNES/SNESSetFromOptions/ Non-linear solver (SNES) flags]		* [https://petsc.org/main/manualpages/SNES/SNESSetFromOptions/ Non-linear solver (SNES) flags]
	* [https://petsc.org/main/manualpages/KSP/KSPSetFromOptions/ Linear solver (KSP) flags]		* [https://petsc.org/main/manualpages/KSP/KSPSetFromOptions/ Linear solver (KSP) flags]
		+
		+	=== Notable Input Flags ===
		+	* <code>-ts_monitor_solution</code>: This will save the results of a simulation to a file. Example: <code>-ts_monitor_solution cgns:flow_visualization.cgns</code> will save the results to a file called <code>flow_visualization.cgns</code>