1(sec_runningtests)= 2 3# Running PETSc Tests 4 5## Quick start with the tests 6 7Users should set 8`$PETSC_DIR` and `$PETSC_ARCH` before running the tests, or can 9provide them on the command line as below. 10 11To check if the libraries are working do: 12 13```console 14$ make PETSC_DIR=<PETSC_DIR> PETSC_ARCH=<PETSC_ARCH> check 15``` 16 17For comprehensive testing of builds, the general invocation from the `$PETSC_DIR` is: 18 19```console 20$ make PETSC_DIR=<PETSC_DIR> $PETSC_ARCH=<PETSC_ARCH> alltests 21``` 22 23or 24 25```console 26$ make [-j <n>] test PETSC_ARCH=<PETSC_ARCH> 27``` 28 29For testing `configure` that used the `--prefix` option, the 30general invocation from the installation (prefix) directory is: 31 32```console 33$ make [-j <n>] -f share/petsc/examples/gmakefile.test test 34``` 35 36which will create/use the directories `tests/*` in the current directory for generated test files. 37You may pass an additional argument `TESTDIR=mytests` to place these generated files elsewhere. 38 39For a full list of options, use 40 41```console 42$ make help-test 43``` 44 45## Understanding test output and more information 46 47Depending on your machine’s configuration running the full test suite 48(above) can take from a few minutes to a couple hours. Note that 49currently we do not have a mechanism for automatically running the test 50suite on batch computer systems except to obtain an interactive compute 51node (via the batch system) and run the tests on that node (this assumes 52that the compilers are available on the interactive compute nodes. 53 54The test reporting system classifies them according to the Test Anywhere 55Protocal (TAP) [^id2]. In brief, the categories are 56 57- `ok` The test passed. 58- `not ok` The test failed. 59- `not ok #SKIP` The test was skipped, usually because build 60 requirements were not met (for example, an external solver library 61 was required, but PETSc was not `configure` for that library.) 62 compiled against it). 63- `ok #TODO` The test is under development by the developers. 64 65The tests are a series of shell scripts, generated by information 66contained within the test source file, that are invoked by the makefile 67system. The tests are run in `$PETSC_DIR/$PETSC_ARCH/tests` with 68the same directory as the source tree underneath. For testing installs, 69the default location is `${PREFIX_DIR}/tests` but this can be changed 70with the `TESTDIR` location. (See {any}`sec_directory`). A 71label is used to denote where it can be found within the source tree. 72For example, test `vec_vec_tutorials-ex6`, which can be run e.g. with 73 74```console 75$ make test search='vec_vec_tutorials-ex6' 76``` 77 78(see the discussion of `search` below), denotes the shell script: 79 80```console 81$ $PETSC_DIR/$PETSC_ARCH/tests/vec/vec/tutorials/runex6.sh 82``` 83 84These shell scripts can be run independently in those directories, and 85take arguments to show the commands run, change arguments, etc. Use the 86`-h` option to the shell script to see these options. 87 88Often, you want to run only a subset of tests. Our makefiles use 89`gmake`’s wildcard syntax. In this syntax, `%` is a wild card 90character and is passed in using the `search` argument. Two wildcard 91characters cannot be used in a search, so the `searchin` argument is 92used to provide the equivalent of `%pattern%` search. The default 93examples have default arguments, and we often wish to test examples with 94various arguments; we use the `argsearch` argument for these searches. 95Like `searchin`, it does not use wildcards, but rather whether the 96string is within the arguments. 97 98Some examples are: 99 100```console 101$ make test search='ts*' # Run all TS examples 102$ make test searchin='tutorials' # Run all tutorials 103$ make test search='ts*' searchin='tutorials' # Run all TS tutorials 104$ make test argsearch='cuda' # Run examples with cuda in arguments 105$ make test test-fail='1' 106$ make test query='requires' queryval='*MPI_PROCESS_SHARED_MEMORY*' 107``` 108 109It is useful before invoking the tests to see what targets will be run. 110The `print-test` target helps with this: 111 112```console 113$ make print-test argsearch='cuda' 114``` 115 116To see all of the test targets which would be run, this command can be 117used: 118 119```console 120$ make print-test 121``` 122 123To learn more about the test system details, one can look at the 124{doc}`the PETSc developers documentation </developers/testing>`. 125 126## Using the test harness for your own code 127 128Select a package name, for example, `mypkg` and create a sub-directory with that name, say `/home/mine/mypackage/src/mypkg`. 129In any sub-directory of that directory named `tests` or `tutorials` put a PETSc makefile, for example, 130`src/ts/tutorials/makefile` and standalone test applications that the makefile can compile with, for example 131 132```console 133$ make mytest 134``` 135 136Include at the bottom of the test code a formatted comment indicating what tests should be run, see 137{any}`test_harness`. Also select a directory where you wish the tests to be compiled and run, say `/home/mine/mytests`. 138 139You can build and run the tests with 140 141```console 142$ make -f ${PETSC_DIR}/gmakefile.test TESTSRCDIR=/home/mine/mypackage/src TESTDIR=/home/mine/mytests pkgs=mypkg 143``` 144 145There is not yet a mechanism to have your test code also link against your library, contact us for ideas. 146 147[^id2]: See <https://testanything.org/tap-specification.html> 148