gtest: windows fixes. add filter to remove filename from compiler stdout. And fix filters to use petscdiff

test: runscripts should perhaps not depend on env variables - this way they can be run independently.
This workarrounds the issue with 'make -f gmakefile test' not setting PETSC_DIR env variable

Enable timed removal of the executable files

Remove executables at the pkg-lang level which is close
to the legacy method. It does so by having generated rules, so
this could possibly be improved,

Enable timed removal of the executable files

Remove executables at the pkg-lang level which is close
to the legacy method. It does so by having generated rules, so
this could possibly be improved, or perhaps the INTERMEDIATE
will work eventually, but this is progress.

Did some cleanup after Jed's excellent implicit rules which
went off of the counts.

Also, `make cleantest` now regenerates the files.
This seems to be the safest method due to issues mentioned in previous
commit. Basically, the idea of clean is to create a state similar to
after configure, and that includes generating the test targets

show more ...

Test verbosity improved, new implicit rules

Verbosity now handled for tests in the same way as the main library
compilation by using the testquiet command.

Executables now have implicit rules in gm

Test verbosity improved, new implicit rules

Verbosity now handled for tests in the same way as the main library
compilation by using the testquiet command.

Executables now have implicit rules in gmakefile instead of being
explicit in the testfiles file.

Some prototype code is in place for doing implicit rules for running
the scripts, but it is not working.

show more ...

gtest: filter TAP output instead of suppressing at the origin

To produce the output Barry likes -- without ok and SKIP lines, use

make allgtest

For TAP-compliant output, use either

make allgt

gtest: filter TAP output instead of suppressing at the origin

To produce the output Barry likes -- without ok and SKIP lines, use

make allgtest

For TAP-compliant output, use either

make allgtest-tap

or

make -f gmakefile test

show more ...

Need full path of testcompile.log; do not print ok or skip cases it just makes the logs unreadable

Time: .25 hours

Do not copy over test_harness.sh and report_tests

Invoke report_tests.py with the counts dir specified.
test_harness.sh sourced using PETSC_DIR

Fixed typos found by Jed.

datafilepath -> datafilespath
testsroot -> testroot.

Merge branch 'scott/testharness-update1' of https://bitbucket.org/petsc/petsc into scott/testharness-update1

gmaketest: diff ref_output cur_output (as with legacy tests)

fix bsd test wrt bash. Also remove circular dependency, and add 'cleantest' target

make[2]: Circular report_tests <- report_tests dependency dropped.

Since bash may not always be located in /bin

Reported-by: Satish Balay <balay@mcs.anl.gov>

petsc_harness.sh uses bashisms not supported by all /bin/sh (for example FreeBSD) therefore explicitly invoke bash

Is bash always available? If not we will need to have configure check for it and tu

petsc_harness.sh uses bashisms not supported by all /bin/sh (for example FreeBSD) therefore explicitly invoke bash

Is bash always available? If not we will need to have configure check for it and turn off testing
Time: .2 hours
Reported-by: nightly tests

show more ...

Use petscdiff for comparisons with new test harness

Time: .1 hours

Fixes from pull request comments

This commit addresses issues found during the code review:
- gmakefile had the wrong dependency of library on objects
- gmakefile: $(generatedtestargs) now depends o

Fixes from pull request comments

This commit addresses issues found during the code review:
- gmakefile had the wrong dependency of library on objects
- gmakefile: $(generatedtestargs) now depends on $(generatedtest)
instead of invoking gmakegentest.py as well
- TESTTARGETS should be calculated as needed ("=" instead of ":=")
- Determining requirements based on arguments is fraught with peril, but
is now more robust
- Added support for finding the yaml input file so that it is
automatically added to localrunfiles. This allows for more robust testing.
- Bug fix for localrunfiles: Shell script test harness now copies
runfiles to local directories.

show more ...

Bugs and features per discussion on bitbucket

1 convertExamples now handles multiple filters correctly
2 There was a bug in writing out the TODO block that was caught in
implementing filter
3 Ver

Bugs and features per discussion on bitbucket

1 convertExamples now handles multiple filters correctly
2 There was a bug in writing out the TODO block that was caught in
implementing filter
3 Verbosity level changed to be more silent by default, but to enable
V=1 to make more verbose
4 Skip changed from 'not ok' to 'ok'
5 localrunfiles feature implemented:
This copies over input file if the description language contains localrunfiles.
6 filter_output implemented:
This has a new diff line that allows filtering of the output before
diffing.
7 Preliminary logging enabled. This uses a simple tee similar to the
build. The logging is done in the tests directory however.

show more ...

Introduce new test harness and script to convert from the old makefile approach to the new test harness