Merge branch 'main' into jed/fluids-jacobian

* main: (24 commits)
op - more data in view
examples/fluids: Test for strong STG
examples/fluids: Add strong STG to documentation
examples/fluids

Merge branch 'main' into jed/fluids-jacobian

* main: (24 commits)
op - more data in view
examples/fluids: Test for strong STG
examples/fluids: Add strong STG to documentation
examples/fluids: Add option for BC from ICs
examples/fluids: Correct for non-origin domains
examples/fluids: Implement Strong STG BC
examples/fluids: Minor stg improvements/style
examples/fluids: Add Strong STG functions
examples/fluids: Fix memory leak in STG
GPU assembly: fix potential basis loading offset error
tests - more complex source include testing
pc - fix qf fallback cloning
restr - relax comp_stride check for num_comp == 1
make style
pc - avoid direct access of basis data in multigrid creation
ceed - refactor fallback Ceed creation into separate fn
minor - homogonize int vs CeedInt
basis - add input checks
restr - add input checks on element restrictions
examples/fluids: Update STG test for new mesh feats
...

show more ...

examples/fluids: Correct for non-origin domains

- Add correction for if domain is not set at the origin

examples/fluids: Minor stg improvements/style

examples/fluids: Add Strong STG functions

examples/fluids: fix zero-size Jacobian data and STG

Use -snes_fd_color for now because STG inflow Jacobian isn't implemented.

Merge branch 'main' into jed/fluids-jacobian

* main:
Fluids - Add STG inflow (#868)
ci - fix Nek5000 testing

Fluids - Add STG inflow (#868)

* doc(fluids): Add STG equations

* doc(fluids): Add basic data flow for STG

* doc(fluids): Add Shur et al. 2014 STG paper to bib

* doc(fluids): Specify STG in

Fluids - Add STG inflow (#868)

* doc(fluids): Add STG equations

* doc(fluids): Add basic data flow for STG

* doc(fluids): Add Shur et al. 2014 STG paper to bib

* doc(fluids): Specify STG inputs files, misc additions

* doc(fluids): Add intro for STG section

* fix(fluids): Add #include ceed.h for qfunctions

- In the spirit of "include what you use"

* feat(fluids): Start work on stg_shur14.h

* doc-fix: Correct kappa_min definition

* Move STG setup functions to problems/stg_shur14

* feat: Add cholesky decomposition function

* fix: Correct stg_ctx malloc, reorganize creation

Co-authored-by: Jed Brown <jed@jedbrown.org>

* fix(fluids): Correct return values of functions

* style: Fix up style

* feat(fluids): Get file paths from PetscOptions

- Also convert SetupSTGContext to return PetscErrorCode

* fix(fluids): Correct stg_ctx dereferencing

- Also move to size_t for type of the offsets

* feat(fluids): Add funcs for processing STG*.dat files

* feat(fluids): Move to PetscOptions* for STG flags

* feat: Use `PetscMax` instead of macro

* fix(fluids): Correct/Refactor file reading functions

- Move to `PetscSynchronizedFGets`
- Remove `inline`
- Pass `comm` between functions
- Add `OpenPHASTADatFile` to DRY

* docs(fluids): Fix equation typo

* fix(fluids): Correct calculation of kappa

* feat(fluids): Complete STGShur14_Calc

* feat(fluids): Add InterpolateProfile helper func

* feat(fluids): Add CalcSpectrum helper func

* feat(fluids): Add to STGShur14_Calc qfunction

* fix: Add M_PI, Update SETERRQ functions

- Also update style

* fix: Correct interpolation outside of datarange

* fix: Add missing definition for ke in CalcSpectrum

* feat: Migrate context and func signatures, Misc

- Create SetupSTGContext to be run in another Setup_____Context
function
- Migrate STGShur14Context, CreateSTGContext, and SetupSTGContext
signatures to navierstokes.h
- Add STG contexts to Physics and CeedData
- Add missing CHKERRQ to PetscFClose
- Move to SPDX license headers

* examples/fluids: Pass solution time via context label

* feat: Implement STG boundary integral

- Add theta0 and implicit members to STGShur14Context
- Tested via implementation to the blasius BL problem (though this will
probably go against the code history)

* feat: Fix STG Stuff

* feat: Implement STG inflow for blasius BL

- Note that fluctuations are turned off in this case

* examples/fluids: Add stg_mean_only flag

* examples/fluids: Check cholesky decomp for nans

- Also correct locaiton of cholesky decomposition in ReadSTGInflow()

* examples/fluids: Correct STG documentation

- Missing a 2 sqrt(3/2) factor and didn't take square root of q

* examples/fluids: Fix STGShur14_Calc

- Given the calculated spectrum, calculation of v' and u' verified
against python implementation (which was validated previously against
PHASTA)

* examples/fluids: Calc dXdx for boundary QFunctions

- Also calculate h from the dXdx in STGShur14_Inflow
- Replace h[0] result with constant dx spacing

* examples/fluids: Fix STG Spectra calcualtion

* examples/fluids: Fix build errors

- Ran into an include cycle collision that resulted in over-defining
SetupContext in advection.c
- newtonian_types.h (which has SetupContext defined) ->
stg_shur14_type.h -> navierstokes.h -> advection.c

* examples/fluids: Update and fix documentation

* examples/fluids: Correct dXdx comment, leave TODO

Co-authored-by: Jed Brown <jed@jedbrown.org>

* examples/fluids: Minor bib citation edits

Co-authored-by: Jed Brown <jed@jedbrown.org>

* examples/fluids: Add STGInflow.dat, fix blasius.yaml

* examples/fluids: int -> PetscInt | CeedInt

* examples/fluids: Style

* examples/fluids: Make Boolean names verb_noun format

- Also changes the stg flag to `-stg_use`

* examples/fluids: Add STG test

* examples/fluids: Style fix up

* examples/fluids: Update docs

* examples/fluids: Implement weakT option for STG

* examples/fluids: Fix casting for ROCm

* examples/fluids: avoid PETSc dependency in qfunctions

* examples/fluids: header cleanup

* backends/hip: avoid redundant inline

* examples/fluids: avoid VLA in qfunctions

GPUs don't like VLA and some compilers reject it when targeting GPUs.

* examples/fluids: Create STG_NMODES_MAX

* examples/fluids: Refactor stg setup out of blasius.c

* examples/fluids: Fix misc GPU bugs

Co-authored-by: Jed Brown <jed@jedbrown.org>

show more ...

examples/fluids: update Blasius inflow/outflow viscous for arbitrary normal

examples/fluids: add IJacobian for SUPG

I think this is exact for Galerkin on channel.yaml and blasius.yaml.
Specifically, SNES converges almost perfectly in this test than when
-snes_mf_operator is

examples/fluids: add IJacobian for SUPG

I think this is exact for Galerkin on channel.yaml and blasius.yaml.
Specifically, SNES converges almost perfectly in this test than when
-snes_mf_operator is added.

-options_file examples/fluids/channel.yaml -snes_monitor -ksp_converged_reason -stab none -ts_max_steps 3 -newtonian_unit_tests -ksp_rtol 1e-10 -order 2 -ts_dt 3e-6 -pc_type lu

I say "think" because something is fishy with the viscous energy term
velocity*stress. If I disable that term, then convergence is clearly
better with the analytic Jacobian (makes sense because it's accuracy is
machine epsilon) than with -snes_mf_operator. This is unexpected, but
the difference is near the limit of numerical stability so I'm not
confident it's wrong. Also, the term itself is so simple.

The Jacobian is inexact for SUPG in that Tau_d is frozen.

WIP: examples/fluids: support MatShell and Blasius_Inflow_Jacobian

show more ...

examples/fluids: record state and tau in preparation for Jacobian

Change notation from dq to Grad_q so that dq can be used consistently as
variations.

examples/fluids: small consolidation

examples/fluids: factor Newtonian with better helpers

examples/fluids: avoid pow(double, int) due to CUDA_ERROR_INVALID_PTX

NVRTC with various CUDA versions 11.2 to 11.6 return
CUDA_ERROR_INVALID_PTX when calling pow(double, int) from kernels.
Typicial

examples/fluids: avoid pow(double, int) due to CUDA_ERROR_INVALID_PTX

NVRTC with various CUDA versions 11.2 to 11.6 return
CUDA_ERROR_INVALID_PTX when calling pow(double, int) from kernels.
Typicially pow(x, 2.) is optimized to x*x at -O1 and above with gcc, but
pow(x, 4.) is not optimized.

show more ...

examples/fluids: make setup contexts independent

These are now private to each example and can be developed independently.

examples/fluids: consolidate two-step setup

Contexts are created in one place now and we have removed duplicate
struct definitions for physics.

Shock Tube + YZB Discontinuity Capturing (#851)

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the

Shock Tube + YZB Discontinuity Capturing (#851)

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Changed variable name midpt to mid_point, removed redundant slip BC declarations.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Rebased with main

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Implemented fixes to the flux jacobians

* Updated documentation, tau definition in SU, and recommended fixes to the flux Jacobian.

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Missed a style fix

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixes to style and documentation.

* Fixing unnecessary problem->dim declaration and missing field in comment describing initial condition density.

* Added warnings for incompatible user input combinations.

* Added comments on beta to documentation, fixed j_gradn style issues.

* Compilation errors on my end. Committing to see if it's local or not.

* Style fixes

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Changed variable name midpt to mid_point, removed redundant slip BC declarations.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Rebased with main

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Implemented fixes to the flux jacobians

* Updated documentation, tau definition in SU, and recommended fixes to the flux Jacobian.

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Missed a style fix

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixes to style and documentation.

* Fixing unnecessary problem->dim declaration and missing field in comment describing initial condition density.

* Added warnings for incompatible user input combinations.

* Added comments on beta to documentation, fixed j_gradn style issues.

* Compilation errors on my end. Committing to see if it's local or not.

* Fluids: SETERRQ1 -> SETERRQ to be compatible with PETSc 3.17

* Fluids - Update shock-tube with #881

* Small fix to mid_point definition and removed repeat entries for Shock Tube problem in README.md.

* Including test output.

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Changed variable name midpt to mid_point, removed redundant slip BC declarations.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Rebased with main

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixed style error, made variable names clearer, and removed dead code.

* Missed a style fix

* Implemented fixes to the flux jacobians

* Updated documentation, tau definition in SU, and recommended fixes to the flux Jacobian.

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Missed a style fix

* Implemented the Sod Shock Tube problem with YZB discontinuity capturing. for the explicit formulation. SU stabilization is also implemented for the explicit formulation.

* Fixes to style and documentation.

* Fixing unnecessary problem->dim declaration and missing field in comment describing initial condition density.

* Added warnings for incompatible user input combinations.

* Added comments on beta to documentation, fixed j_gradn style issues.

* Compilation errors on my end. Committing to see if it's local or not.

* Fluids: SETERRQ1 -> SETERRQ to be compatible with PETSc 3.17

* Fluids - Update shock-tube with #881

* Fluids - added test output file for shocktube

* Small fix to mid_point definition and removed repeat entries for Shock Tube problem in README.md.

* Including test output.

* Update doc/sphinx/source/references.bib

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Update examples/fluids/README.md

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Update examples/fluids/index.md

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Update examples/fluids/navierstokes.h

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Update examples/fluids/problems/shocktube.c

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Update examples/fluids/qfunctions/shocktube.h

Co-authored-by: Jed Brown <jed@jedbrown.org>

* Removed unused IFunction

* Added shocktube variables to SetupContext in advection.h, advection2d.h, and densitycurrent.h

* Missed some merge conflicts that have now been cleared up.

* Added helper function for covariant element length along a given vector.

* Added SetupContext to shocktube.h

* Fix to regression test path

* remove variable transpose of the Jacobian

* Fix index swap in implicit SUPG

* examples/fluids shocktube: fix stab summation and transpose

* delete IFunction

* example/fluids- updates reference solution

Co-authored-by: Timothy Aiken <timothyaiken@Timothys-MacBook-Pro.local>
Co-authored-by: Timothy Aiken <timothyaiken@cu-biot-10-10.203.167.69.int.colorado.edu>
Co-authored-by: Leila Ghaffari <Leila.Ghaffari@colorado.edu>
Co-authored-by: Jed Brown <jed@jedbrown.org>
Co-authored-by: Timothy Aiken <timothyaiken@cu-genvpn-tcom-10.180.164.177.int.colorado.edu>
Co-authored-by: AdelekeBankole <adeleke.bankole@colorado.edu>

show more ...

Cleaner density inflow (#953)

* density weakly set on the inflow....also seems to have a commit intended for ken/primitiveSUPGterm OOOP

Merged onto main this density ONLY inflow step...fixed E_k

Cleaner density inflow (#953)

* density weakly set on the inflow....also seems to have a commit intended for ken/primitiveSUPGterm OOOP

Merged onto main this density ONLY inflow step...fixed E_kinitec bug along the way which used interior velocity.

* first cut at choose your weak inflow rho or T

* cleanup and a missing gamma

* dropped time step to 2e-6, added time limit of 0.001, added suggested parameters for tau constants for linear vs. quadratic. Note this is far from certain, expecially quadratic. Also left commented the required settings to forces SNES to do a fixed number of NL iterations. The current settings do 2 NL iterations for a while then drop to 1 which is ok for steady state but might not be good for SRS.

* potentially better Ctau_{C,M,E} that have much lower oscillations in the free stream even at large time steps like 3.2e-5

* examples/fluids: fix output_freq

* examples/fluids: use bool for weakT

Suggested-by: James Wright <james@jameswright.xyz>

Co-authored-by: Jed Brown <jed@jedbrown.org>

show more ...

feat: Add blasius boundary layer and channel flow examples to examples/fluids (#942)

* examples/fluids: Upload current case for collaboration

* examples/fluids: Replace developing BL with duct

feat: Add blasius boundary layer and channel flow examples to examples/fluids (#942)

* examples/fluids: Upload current case for collaboration

* examples/fluids: Replace developing BL with duct

* examples/fluids: Allow density to "float" at inflow

* fix: Add u_normal

* examples/fluids: bug fix in density flux

* examples/fluids: implement "floating" pressure; prescribed u, T

* examples/fluids: fixing initialization

We need has_neumann=true to get the weak boundary integrals called. It
is initialized in problem->setup_ctx and used in SetupLibceed(). This is
a hack to always apply.

* examples/fluids: include kinetic energy in Blasius IC

* examples/fluids: Blasius quasi-2D (slip in z planes)

* feat: Add Exact_Channel function

* feat: Use Exact_Channel for IC and Inflow

* examples/fluids: Add channel example

* examples/fluids: Define mu in blasius.c, cleanup

* examples/fluid: Add Blasius solution calculator

* examples/fluids: Add blasius IC and BCs

* Convert to implicit TS

* examples/fluids: Update blasius.yaml

* Move to ChannelContext, refactor DC and newtonian

- Added a dedicated ChannelContext, which allows for user setting of
flow parameters and consistent sharing of parameters between
QFunctions (instead of hardcoding)
- Moved density current (DC) specific settings from newtonian.c
densitycurrent.c

* Make gravity into vector quantity

* Add in body force for channel

* examples/fluids: remove explicitly setting coordinate field

This is created automatically now by DMPlex and DMProjectCoordinates
breaks (localized) periodicity, resulting in a tangled mesh.

* examples/fluids: transpose flux Jacobian in SUPG stabilization

* examples/fluids: update docs commensurate with flux Jacobian transpose fix

* examples/fluids: work on docs for tau

* examples/fluids: Increase blasius Re, implement Xi to SUPG

- Increased the Uinf and mu by 4x for the blasius problem
- Implemented the calculation of Xi for the SUPG term in newtonian.h

* examples/fluids: Remove flux jacobian transpose

- Removed for both eulervortex.h and newtonian.h

* examples/fluids: Reset Newtonian unit scaling to neutral

- Note that this resets the scaling for the densitycurrent problem,
thus the examples (and possibly tests) will probably need to be
changed

* examples/fluids: fix PetscOptionsBegin for new PETSc

* feat: Add slanted domain top surface

* feat: Add graded mesh option for blasius

* fix: Correct inflow boundary condition handling

* feat: Move to blasius_context, add CLI options

* feat: Move Blasius to physical air properties

* fix: Blasius profile and mesh generation

- Add the correct float->int rounding
- Fix eta to a value after exceeding the table's bounds

fix: Add fix eta after exceeding table

* examples/fluids: Move to physical mu default, fix tests

* doc: Add minimum documentation for channel and blasius

* examples/fluids: Add primitive jacobian and tau

- Also add misc comments

* examples/fluids: propose alternative flux Jacobian via prim-to-conservative

* examples/fluids: Add PHASTA diagonal tau

* fix: Add back in timestep obtaining

* examples/fluids: Add analytic tractions to inflow and outflow

- This is a stop-gap until viscous flux can be computed properly on the
boundary from current solution.
- Tau constants in a state of flux still but this produced a decent v at
inflow and outflow, removing the steep dive caused by forcing
dv/dx=-du/dy from a zero traction of omission.

* examples/fluids: CLI options for diagonal c_tau's

* examples/fluids: Update dt via context labels

* feat: Add more blasius table points

* examples/fluids: Cleanup stab and adjust Ctau_E

* examples/fluids: Replace spatial tau with diagonal

* examples/fluids: Fix stab in Euler

- Initialize `stab` to zero, add sum over loop

* examples/fluids: Document diagonal tau formulation

* examples/fluids: Fix compiler warnings

* examples/fluids: check error codes and elide unused header

* examples/fluids: Correct blasius example CLI

- Also add mention of the `blasius.yaml` file

* examples/fluids: Add supg to default blasius.yaml

* examples/fluids: Use yaml for channel and blasius

- Instead of long CLI options
- Also cleaned up the yaml formatting
- Added Cartesian directions to the faceMarker names

* examples/fluids: Show example yamls in README

Co-authored-by: Jed Brown <jed@jedbrown.org>
Co-authored-by: Kenneth E. Jansen <Kenneth.Jansen@colorado.edu>

show more ...

minor - update copyright headers

Extract generic Navier-Stokes code into 'newtonian.h' (#903)

* feat(fluids): Extract newtownian.h from DC problem

- Also trade out `int` for `StabilizationType` enum

* style: Fix parameter sp

Extract generic Navier-Stokes code into 'newtonian.h' (#903)

* feat(fluids): Extract newtownian.h from DC problem

- Also trade out `int` for `StabilizationType` enum

* style: Fix parameter spacing

* feat: Move generic setup from DC -> newtonian

- Includes moving from `DCContext` to `NewtonianIdealGasContext` at
appropriate locations

* feat: Add "still" IC for generic Newtonian IG problems

* feat: Move SetupContext from DC -> newtonian.c

* fix: Remove unused variables, general cleanup

* examples/fluids: remove unnecessary includes

* examples/fluids: copy declarations to make newtonian.h self-contained

* examples/fluids: full name for context

Co-authored-by: Jed Brown <jed@jedbrown.org>

show more ...

Move euler_outflow to eulervortex.h to make HIP happy.

Set BCs in CL options

Factor out euler_outflow QFunction to be used in different problem cases

Use DMCreate() to create mesh instead of DMPlexCreateBoxMesh()

eulervortex: add test_5 - non-smooth cylinder

examples/fluids: tau as spatial criterion 2 of hughesetal2010

Testing traveling vortex with

$ build/fluids-navierstokes -problem euler_vortex -ts_adapt_monitor -lx 1000 -ly 1000 -lz 50 -dm_plex_box

examples/fluids: tau as spatial criterion 2 of hughesetal2010

Testing traveling vortex with

$ build/fluids-navierstokes -problem euler_vortex -ts_adapt_monitor -lx 1000 -ly 1000 -lz 50 -dm_plex_box_faces 20,20,1 -ts_adapt_safety 0.8 -implicit -snes_atol 1e-3 -ksp_atol 1e-4 -ksp_rtol 1e-3 -ts_type alpha -ts_dt 0.1 -ksp_type bcgs -pc_type lu -log_view -snes_monitor -ksp_monitor -snes_converged_reason -output_freq 1 -degree 1 -ts_max_time 5. -stab supg -snes_lag_jacobian 30 -snes_lag_jacobian_persists -snes_mf_operator

show more ...