style - consistent nameing and style for gpu backends

gpu - drop unused argument in init

gpu - drop unused [HIP, CUDA]_MAX_PATH constant

gpu - split common cuda/hip data into separate folder

cuda - separate compile functionality into new header

vec/qf - initial valid/borrowed/owned split for data (#853)

* vec/qf - initial valid/borrowed/owned split for data

* vec/qf - tidy logic for checking active/stale data

* minor - add missing NU

vec/qf - initial valid/borrowed/owned split for data (#853)

* vec/qf - initial valid/borrowed/owned split for data

* vec/qf - tidy logic for checking active/stale data

* minor - add missing NULL

* doc - explain VectorTakeArray update

* minor - update error messages

* test - update error message in junit/tap

* gpu - fix stray CeedScalar vs void for QFunctionContext

* vec/qf - clarify/simplify access logic

* vec - calloc host arrays when no value set to make empty

* style - minor

* style - minor

* minor - fix error messages

* vec/qf - move data validity checking to backend interface

* gpu - add missing sync error checking for qfcontext

* gpu - homogonize use of impl for backend data to reduce confusion

* vec - clarify access conditions

* python - update test for stricter vector access

* vec - minor fixes

* minor - fix ipython change

* vec - add missing declarations in ceed/backend.h

* ctx - mirror vector borrowed data check in ctx interface

* vec - add CeedVectorGetArrayWrite

* vec - consistent use of CeedVectorGetArray vs CeedVectorGetArrayWrite

* python - small vec fixes

* doc - describe vector data semantics

* magma - update restriction

* gpu - fix restr bug I added, need to sum into target

* magma - fix restriction bug

* cpu - fix restriction bug here too

* op - fix evec allocations

* julia - fix ElemRestriction for new vector access rules

* op - double check GetArray vs Read vs Write usage

* doc - small fix

* restr - clean up read/write logic for restr

* python - add vec.array_write

* magma - typo fix

show more ...

gpu - remove edata from backend operator structs

op - turn magic number 16 into constant CEED_FIELD_MAX

debug - create CeedDebugEnv macro, refactor CeedDebug macro

tidy - suppress spurious warning

gpu - refactor common jit code into common functions

qf - add function to retreive kernel name, stored path now excludes kernel name

gpu - cache two more vectors used in operator diagonal assembly

op - add CeedOperatorLinearAssembleQFunctionBuildOrUpdate to reduce kernel calls in diagonal/full/fdm assembly

cuda - drop unused function declarations

interface - refactor *GetFields to include number of fields

backends/cuda: record cudaDeviceProp struct instead of just max block size

backends/cuda: choose block size based on number of registers used by kernel

Complicate QFunctions, such as those in solid mechanics, use too many
registers to launch blocks of 1024 threads (hardwar

backends/cuda: choose block size based on number of registers used by kernel

Complicate QFunctions, such as those in solid mechanics, use too many
registers to launch blocks of 1024 threads (hardware max on
Volta/Ampere). We ask the kernel how large a block it can use and select
that block size. As a refinement, we could consider making the block
sizes smaller if there are fewer blocks than SMs (strong scaling limit).

show more ...

Allow CeedScalar to be single precision (#788)

One can modify `ceed.h` to include `ceed-f32.h` and then use single precision. This is tested for C in CI and has been tested by developers with Rust,

Allow CeedScalar to be single precision (#788)

One can modify `ceed.h` to include `ceed-f32.h` and then use single precision. This is tested for C in CI and has been tested by developers with Rust, Julia, and Python. This interface is evolving and should be considered experimental at this time (thus lack of automated build support).

* Introduce CeedScalarType enum

* WIP changes to allow different definitions of CeedScalar

* Introduce new header files for float and double

* Only use avx tensor contract and MAGMA non-tensor basis if CeedScalar is double

* WIP changes to allow CeedScalar to be float

* WIP start trying to adjust test tolerances for float or double

* fix typos in comments

* install ceed-f32/64 headers

* Fix missing casts for hipMAGMA element restrictions

* make CeedQFunctionContextGetContextSize available for Python bindings

* Changes to Python bindings to allow CeedScalar to be float

* WIP adjust Python tests for float or double

* make style

* remove QFunctionContextGetContextSize from backend header

* Use quotes instead of <> in include statement

* Remove unncessary includes

* Update tolerances for tests

* [Julia] allow CeedScalar to be Float32

* [Julia] Use Preferences instead of custom build configuration

# Conflicts:
# julia/LibCEED.jl/src/C.jl

* [Makefile] Change definition of CC_VENDOR so it works with cross-compilation

* [Julia] Use Preferences in CI

# Conflicts:
# .github/workflows/julia-test-with-style.yml

* [Julia] Update docs about preferences

* [Julia] Add test/Project.toml workaround for Preferences

* Add CeedGetScalarType to get the type of CeedScalar at runtime

* [Julia] Move functions from Ceed.jl to LibCEED.jl

* [Julia] Add support for getting library path and scalar type at runtime

* [Julia] Minor change to checking if CUDA is loaded

* [Julia] Check correct CeedScalar types in basis functions

* [Julia] Fix tests comparing with output file

* [Julia] Change devtests to use CeedScalar instead of Float64

* Update test 402 so context will be same size in double or float

* Update tolerances for ceed examples

* [Julia] CUDA fixes

* remove unused variable in t208

* SchurDecomposition: do not compute tau on final iteration

* Update tolerances for some basis tests (for single precision)

* Make style

* Python style fixes for basis test

* Add single precision output for t300 and t320 and adjust checks; skip t541 in single

* Add LCOV exclusions after moving to new line

* fix spacing

* Python: make CEED_EPSILON available as libceed.EPSILON

* Python: optional parameter to specify different output file for test comparison

* Python: update tests' use of EPSILON and change test_300 output file for single precision

* Python: add convenience function for getting dtype corresponding to CeedScalar

* rust - add single precision support

* [Julia] Fall back on Float64 if CeedGetScalarType is not available

* [Julia] style

* Adjust tolerance for t301

* xsmm - add single precision support

* avx - add single precision support

* Add initial single precision support for MAGMA non-tensor basis

* Skip t300 and t320 in single precision; revert Python t300 changes

* Revert output changes for t300 and t320 in junit

* [Julia] Changes to autogenerated bindings for mixed precision

* [Julia] style

* [Julia] Check scalar type when changing libceed library path

The check is also performed when the package is loaded. This prevents having to
restart the Julia session twice

* [Julia] Require JLLWrappers version 1.3

This is needed to use Preferences to change the library path

* Add documentation page for precision development

Co-authored-by: Will Pazner <will.e.p@gmail.com>

* Cleanup from merge: remove old README

* Return CEED_ALIGN to backend.h

* Make Fortran compiler (FC) optional; empty skips Fortran tests

Use in Python and Rust builds, which may not have a Fortran compiler
installed and thus would produce confusing output.

* Add single precision CI test for Noether

Co-authored-by: Jeremy L Thompson <jeremy@jeremylt.org>

Co-authored-by: Will Pazner <will.e.p@gmail.com>
Co-authored-by: Jeremy L Thompson <jeremy@jeremylt.org>
Co-authored-by: Jed Brown <jed@jedbrown.org>

show more ...

cuda/ref: modify weight kernels to avoid CUDA-11.2+ bug on RTX 2080 (issue #802)

Refactor Preconditioning Ingredients (#795)

* op - split preconditioning into separate file

* pc - move FDM to interface level

* pc - initial move of diagonal assembly to interface

* interf

Refactor Preconditioning Ingredients (#795)

* op - split preconditioning into separate file

* pc - move FDM to interface level

* pc - initial move of diagonal assembly to interface

* interface - only return error code or error

* doc - improve preconditioning helper function documentation

* pc - use qdata layout for interface diagonal assembly

* pc - update for consistency in naming, use of CeedChk, and static

* doc - spelling

* style - minor

* doc - add doc for CeedBuildMassLaplace

* doc - update release notes

Co-authored-by: Jed Brown <jed@jedbrown.org>

Co-authored-by: Jed Brown <jed@jedbrown.org>

show more ...

Fluids - Refactor and add GPU support (#754)

* Initial commit for using PetscFunctionList in fluids example

* WIP: debugging

* Debug: initialize PetscFunctionList with NULL

* allocate memor

Fluids - Refactor and add GPU support (#754)

* Initial commit for using PetscFunctionList in fluids example

* WIP: debugging

* Debug: initialize PetscFunctionList with NULL

* allocate memory for context problem

* freed allocated memory

* WIP: fluids: split density current into new compilation unit
I have dropped last changes before this commit so there might be some
bugs

* Fixed bugs after dropping the commit containing unnecessary changes

* NS: dropped unused variable from User struct

* WIP: factored out DMAddBoundary() for density_current

* split advection and advection2d into separate compilation units as well

* fluids: added *.o to make clean

* removed bc from problemData

* style

* commented the error handlers and moved petsc options regarding the boundary conditions to each compilation unit

* call bc functions inside SetUpDM()

* Moved all bc related statements from SetUpDM() to each compilation unit

* examples/fluids/advection.h: style

* WIP: modified CeedQFunctionContexts

* FFR: By creating a struct for DC context, one of the last test crushed so I had to reverse it. This should be investigated later.

* style

* examples/navierstokes: Use heap memory for ctxSetup

* examples/fluids: WIP - reading arrays through cl options for DC is enabled by using double pointers but there is still a problem with unit conversions and also need to fix other structs

* style

* WIP: Trying to assign values to context data in each problem function

* bug: there is a problem in pointer to struct

* WIP: Everything seems to be working but it breaks and I haven't figured out the reason ...

* fixed typo

* comment out stuff that have been moved to densitycurrent.c

* WIP: The first two regression test pass. There is a problem with stabilization.

* style

* Fluids: QFContext -> Physics

* WIP: passing StabilizationType to Physics (but PETSc is not happy and it diverges for SU and SUPG)

* Initial commit for refactoring advection(2d)

* WIP: Advection is set up except for wind_type

* WIP: refactored advection2d; need to fix the BCs for translation and also passing implicit to advection correctly

* WIP: pass wind_type to all functions correctly but there is still a problem with BCs in the case of translation (changes are not applied to advection yet)

* Apply the same fixes for BCs in advection

* Fluids - Refactor: Moved warnings to advection(2d)

* Fluids - Refactor: remove dead code

* Fluids - Refactor: Move *.h to /qfunctions and *.c to /src

* style

* Fluids - Refactor: drop the supg regression test for now

We need to debug SU/SUPG for density_current. The test does not work.

* Fluids - Refactor: Fixed the double-pointer issue

* Fluids - Refactor: Assign ctxSetup->time = 0 inside each problem's *.c

* Fluids - Refactor: make implicit a member of Physics

* Fluids - Refactor: Add warning for the usage of explicit and supg

* Some comments and naming

* Merge main after v0.8 (#723)

* Fluids - Five regression tests added for euler_vortex

* Fluids - Drop some of the regression tests for now (Needs to get fixed)

* Fluids - Moved all functions to their corresponding source files

* Fluids - move problem source files to /problems

* Fluids - drop dead regression test reference files

* Fluids - more consistent file names

* Fluids - drop dead code and comments

* Fluids - removed CreateVectorFromPetscVec() since it was only used once

* Fluids - created AppCtx for cl option variables and moved cl options to a cloption.c

* Fluids: change style of AppCtx

* style

* Fluids: /src -> /utils

* Created CreateDistributedDM() and some style

* Fluids - create RegisterProblem()

* style - minor

* style - minor

* Fluids - some more C style

* Fluids - /utils -> /src

* Fluids - create VizRefineDM() for viz_refine

* Fluids: some style and name change

* Fluids - style

* Fluids - remove the second ceed initialization

* Fluids - Created CeedData and SetupLibCEED()

* Fluids - create SetupContextForProblems()

* Fluids - Assign structs after they are created

* Fluids - style, minor

* Fluids - created RegressionTests_NS

* Fluids - created GetError_NS()

* Fluids - created PrintOutput_NS()

This processes bash output except for the problem summary which will be
included after mem_type is managed properly.

* Fluids - remove unused variables

* Fluids - created TSSolve_NS() which creates, sets up, and solves ts

* Fluids: Fixed a bug caused by merging#714

* Fluids - style

* Fluids - created SetupICsFromBinary() for case of -continue

* Fluids - create SetBCsFromICs_NS()

* Fluids - default degree had is 1 not 2

* Fluids: clean-up

* Fluids: style - minor

* Fluids - drop redundant operation

* Fluids: clean-up SetupLibCEED()

* Fluids: style - minor

* Fluids: WIP - C style

* Fluids: style - minor

* Fluids: move ceed vector creation to SetupLibCEED()

* Fluids: WIP - clean-up CreateOperatorForDomain()

* Fluids: major clean-up in CreateOperatorForDomain()

* Fluids: style - minor

* Fluids: drop VectorPlacePetscVec()

* Fluids: move ICs_FixMultiplicity() to misc.c

* Fluids: WIP - C style

* Fluids: fix merge conflict after d63eecaab22a26d3072677a1b66c0cf77aa9009d

* Fluids: drop unused function call DMLocalizeCoordinates()

* Fluids: replace user requested memType with MemTypeP2C()

* Fluids: C style

* Fluids: Q->num_Q (Q (# qpts) and Q (global state vector) were confusing my fresh petsc config)

* Fluids: WIP - add GPU vectors

* Fluids: Fixed the memory leak caused by Physics struct

* Fluids: style -minor

* Fluids: no need for casting

* Fluids: Fix mem leak (q0_ceed had to be destroyed at the end of navierstokes.c)

* Fluids: Fixed all mem leaks

* Fluids: Set VecType for dm_hierarchy

* Fluids/doc: fix style

* Fluids: fix missing VecRestoreArray()

* Fluids: clean-up ComputeLumpedMassMatrix()

* Fluids: Cleanup ICs_FixMultiplicity()

* Fluids: WIP - cleanup setupts.c

* Fluids: style - minor

* Fluids: Create element coord ceed vector locally and drop it from CeedData

* Fluids: fix some todos

* Fluids: drop unused code for EULER BCs

* Fluids: PetscMalloc1 -> PetscCalloc1

* Fluids - Define BCs for ADVECTION(2D) for all cases without having the user specify them and change the regressions accordingly

ToDo: For some reason I can't create new regression tests for explicit scheme.

* Fluids - Define problem->print_info() for each problem

* Fluids: Shorten some problem specific command line variables & style

* Fluids - Cleanup BCs in the translation mode

* Fluids - cleanup

* Fluids - more cleanup in BCs

* Fluids - Updated doc, README, and cleanup in the code and cl options

* Fluids - add more regression tests

* Fluids - Define tolerance for the convergence test

* Fluids: include x_coord in CeedData

* wip - gpu

Revert "Fluids: include x_coord in CeedData"

I had to revert this commit since it crashes DC

* style

* Fluids - remove unused variables

* Fluids - fix unused function warnings

* Fluids - fix the remaining warnings

* Fluids - style

* Fluids - add missing licence

* Fluids - bring back contexts to the QFunction header files

* Fluids - transfer QFunctions for boundary integral to their corresponding setupgeo(2d).h header files and add description for Setup2d

* fluids - fix context data sizes

* GPU - add helper function macro

* ex - use consistent MPI info output

* GPU - improve CEED_QFUNCTION_HELPER macro, more flexible

* Fluids - turn on some gpu tests

* Fluids - bring back element coordinate to CeedData

* Fluids - cleanup

* qfunction - clean up helper function macro

* Fixed the remaining conflicts from #769 (helped-by: @rezgarshakeri)

Co-authored-by: Jed Brown <jed@jedbrown.org>
Co-authored-by: jeremylt <thompson.jeremy.luke@gmail.com>

show more ...

context - add QFunctionContextTakeData to silence docs warning about missing ref

vec - add CeedVectorScale

device - drop forced device->host sync in certain cases, only sync when requested