1# libCEED: the CEED API Library 2 3## Code for Efficient Extensible Discretization 4 5This repository contains an initial low-level API library for the efficient 6high-order discretization methods developed by the ECP co-design [Center for 7Efficient Exascale Discretizations (CEED)](http://ceed.exascaleproject.org). 8While our focus is on high-order finite elements, the approach is mostly 9algebraic and thus applicable to other discretizations in factored form, see the 10[API documentation](doc/libCEED.md). 11 12One of the challenges with high-order methods is that a global sparse matrix is 13no longer a good representation of a high-order linear operator, both with 14respect to the FLOPs needed for its evaluation, as well as the memory transfer 15needed for a matvec. Thus, high-order methods require a new "format" that still 16represents a linear (or more generally non-linear) operator, but not through a 17sparse matrix. 18 19The goal of libCEED is to propose such a format, as well as supporting 20implementations and data structures, that enable efficient operator evaluation 21on a variety of computational device types (CPUs, GPUs, etc.). This new operator 22description is based on algebraically [factored form](doc/libCEED.md), which is 23easy to incorporate in a wide variety of applications, without significant 24refactoring of their own discretization infrastructure. 25 26The repository is part of the [CEED software suite][ceed-soft], a collection of 27software benchmarks, miniapps, libraries and APIs for efficient exascale 28discretizations based on high-order finite element and spectral element methods. 29See http://github.com/ceed for more information and source code availability. 30 31The CEED research is supported by the [Exascale Computing Project][ecp] 32(17-SC-20-SC), a collaborative effort of two U.S. Department of Energy 33organizations (Office of Science and the National Nuclear Security 34Administration) responsible for the planning and preparation of a [capable 35exascale ecosystem](https://exascaleproject.org/what-is-exascale), including 36software, applications, hardware, advanced system engineering and early testbed 37platforms, in support of the nation’s exascale computing imperative. 38 39For more details on the CEED API see http://ceed.exascaleproject.org/ceed-code/. 40 41## Building 42 43The CEED library, `libceed`, is a C99 library with no external dependencies. 44It can be built using 45 46 make 47 48## Testing 49 50The test suite produces [TAP](https://testanything.org) output and is run by: 51 52 make test 53 54or, using the `prove` tool distributed with Perl (recommended) 55 56 make prove 57 58## Examples 59 60libCEED comes with several examples of its usage, ranging from standalone C 61codes in the `/examples/ceed` directory to examples based on external packages, 62such as MFEM, PETSc and Nek5000. 63 64To build the examples, set the `MFEM_DIR`, `PETSC_DIR` and `NEK5K_DIR` variables 65and run: 66 67``` 68# libCEED examples on CPU and GPU 69cd examples/ceed 70make 71./ex1 -ceed /cpu/self 72./ex1 -ceed /gpu/occa 73cd ../.. 74 75# MFEM+libCEED examples on CPU and GPU 76cd examples/mfem 77make 78./bp1 -ceed /cpu/self -no-vis 79./bp1 -ceed /gpu/occa -no-vis 80cd ../.. 81 82# PETSc+libCEED examples on CPU and GPU 83cd examples/petsc 84make 85./bp1 -ceed /cpu/self 86./bp1 -ceed /gpu/occa 87cd ../.. 88 89# Nek+libCEED examples on CPU and GPU 90cd examples/nek5000 91./generate-boxes.sh 2 4 92./make-nek-examples.sh 93./run-nek-example.sh -ceed /cpu/self -b b3 94./run-nek-example.sh -ceed /gpu/occa -b b3 95cd ../.. 96``` 97 98The above code assumes a GPU-capable machine enabled in the OCCA 99backend. Depending on the availabl backends, other Ceed resource specifiers can 100be provided with the `-ceed` option, for example: 101 102CEED resource (`-ceed`) | Backend 103----------------------- | --------------------------------- 104`/cpu/self` | Serial reference implementation 105`/cpu/occa` | Serial OCCA kernels 106`/gpu/occa` | CUDA OCCA kernels 107`/omp/occa` | OpenMP OCCA kernels 108`/ocl/occa` | OpenCL OCCA kernels 109 110## Install 111 112To install libCEED, run 113 114 make install prefix=/usr/local 115 116or (e.g., if creating packages), 117 118 make install prefix=/usr DESTDIR=/packaging/path 119 120Note that along with the library, libCEED installs kernel sources, e.g. OCCA 121kernels are installed in `$prefix/lib/okl`. This allows the OCCA backend to 122build specialized kernels at run-time. In a normal setting, the kernel sources 123will be found automatically (relative to the library file `libceed.so`). 124However, if that fails (e.g. if `libceed.so` is moved), one can copy (cache) the 125kernel sources inside the user OCCA directory, `~/.occa` using 126 127 $(OCCA_DIR)/bin/occa cache ceed $(CEED_DIR)/lib/okl/*.okl 128 129This will allow OCCA to find the sources regardless of the location of the CEED 130library. One may occasionally need to clear the OCCA cache, which can be accomplished 131by removing the `~/.occa` directory or by calling `$(OCCA_DIR)/bin/occa clear -a`. 132 133### pkg-config 134 135In addition to library and header, libCEED provides a [pkg-config][pkg-config1] 136file that can be used to easily compile and link. [For example][pkg-config2], if 137`$prefix` is a standard location or you set the environment variable 138`PKG_CONFIG_PATH`, 139 140 cc `pkg-config --cflags --libs ceed` -o myapp myapp.c 141 142will build `myapp` with libCEED. This can be used with the source or 143installed directories. Most build systems have support for pkg-config. 144 145## Contact 146 147You can reach the libCEED team by emailing [ceed-users@llnl.gov](mailto:ceed-users@llnl.gov) 148or by leaving a comment in the [issue tracker](https://github.com/CEED/libCEED/issues). 149 150## Copyright 151 152The following copyright applies to each file in the CEED software suite, unless 153otherwise stated in the file: 154 155> Copyright (c) 2017, Lawrence Livermore National Security, LLC. Produced at the 156> Lawrence Livermore National Laboratory. LLNL-CODE-734707. All Rights reserved. 157 158See files LICENSE and NOTICE for details. 159 160[ceed-soft]: http://ceed.exascaleproject.org/software/ 161[ecp]: https://exascaleproject.org/exascale-computing-project 162[pkg-config1]: https://en.wikipedia.org/wiki/Pkg-config 163[pkg-config2]: https://people.freedesktop.org/~dbn/pkg-config-guide.html#faq 164