1*4bcd95a3SBarry Smith(pipelines)= 2*4bcd95a3SBarry Smith 3*4bcd95a3SBarry Smith# GitLab CI Pipelines 4*4bcd95a3SBarry Smith 5*4bcd95a3SBarry SmithPETSc uses [GitLab Pipelines](https://docs.gitlab.com/ee/ci/pipelines/) for testing during continuous integration. 6*4bcd95a3SBarry Smith 7*4bcd95a3SBarry SmithDo not overdo requesting testing; it is a limited resource, so if you 8*4bcd95a3SBarry Smithrealize a currently running pipeline is no longer needed, cancel it. 9*4bcd95a3SBarry Smith 10*4bcd95a3SBarry SmithThe pipeline status for a merge request (MR) is displayed near the top of the MR page and in the pipelines tab. 11*4bcd95a3SBarry Smith 12*4bcd95a3SBarry Smith(The figures below are high resolution, so zoom in if needed) 13*4bcd95a3SBarry Smith 14*4bcd95a3SBarry Smith:::{figure} /images/developers/pipeline-from-MR.png 15*4bcd95a3SBarry Smith:align: center 16*4bcd95a3SBarry Smith:width: 90% 17*4bcd95a3SBarry Smith 18*4bcd95a3SBarry SmithPipeline status for a merge request (MR) 19*4bcd95a3SBarry Smith::: 20*4bcd95a3SBarry Smith 21*4bcd95a3SBarry SmithTo un-pause the pipeline, click the "play" button (or start a new one with "Run Pipeline" if necessary). 22*4bcd95a3SBarry Smith 23*4bcd95a3SBarry Smith:::{figure} /images/developers/run-paused-pipeline.png 24*4bcd95a3SBarry Smith:align: center 25*4bcd95a3SBarry Smith:width: 90% 26*4bcd95a3SBarry Smith 27*4bcd95a3SBarry SmithUn-pausing a pipeline. 28*4bcd95a3SBarry Smith::: 29*4bcd95a3SBarry Smith 30*4bcd95a3SBarry SmithA pipeline consists of "Stages" each with multiple "Jobs". Every job is one configuration on one machine. 31*4bcd95a3SBarry Smith 32*4bcd95a3SBarry Smith:::{figure} /images/developers/show-failure.png 33*4bcd95a3SBarry Smith:align: center 34*4bcd95a3SBarry Smith:width: 90% 35*4bcd95a3SBarry Smith 36*4bcd95a3SBarry SmithExamining a failed pipeline stage. 37*4bcd95a3SBarry Smith::: 38*4bcd95a3SBarry Smith 39*4bcd95a3SBarry SmithYou can see the failed jobs by clicking on the X. 40*4bcd95a3SBarry Smith 41*4bcd95a3SBarry Smith:::{figure} /images/developers/find-exact-bad-job.png 42*4bcd95a3SBarry Smith:align: center 43*4bcd95a3SBarry Smith:width: 90% 44*4bcd95a3SBarry Smith 45*4bcd95a3SBarry SmithLocating the exact failed job in a pipeline stage. 46*4bcd95a3SBarry Smith::: 47*4bcd95a3SBarry Smith 48*4bcd95a3SBarry SmithA job is a run of the {any}`PETSc test harness<test_harness>` and consists of many "examples". 49*4bcd95a3SBarry SmithEach test is a run of an example with a particular set of command line options 50*4bcd95a3SBarry Smith 51*4bcd95a3SBarry SmithA failure in running the job's tests will have `FAILED` and a list of the failed tests 52*4bcd95a3SBarry Smith 53*4bcd95a3SBarry Smith:::{figure} /images/developers/failed-examples.png 54*4bcd95a3SBarry Smith:align: center 55*4bcd95a3SBarry Smith:width: 90% 56*4bcd95a3SBarry Smith 57*4bcd95a3SBarry SmithFailed examples in a pipeline job. 58*4bcd95a3SBarry Smith::: 59*4bcd95a3SBarry Smith 60*4bcd95a3SBarry SmithSearch for `not ok` in the jobs output to find the exact failure 61*4bcd95a3SBarry Smith 62*4bcd95a3SBarry Smith:::{figure} /images/developers/unfreed-memory.png 63*4bcd95a3SBarry Smith:align: center 64*4bcd95a3SBarry Smith:width: 90% 65*4bcd95a3SBarry Smith 66*4bcd95a3SBarry SmithA test that failed because of unfreed memory. 67*4bcd95a3SBarry Smith::: 68*4bcd95a3SBarry Smith 69*4bcd95a3SBarry SmithAfter you have fixed a problem that appeared in a particular (set of) job(s) you may want to test your fix only on those jobs. To do this use 70*4bcd95a3SBarry Smith 71*4bcd95a3SBarry Smith```console 72*4bcd95a3SBarry Smith$ ./lib/petsc/bin/maint/runjobs.py [job_1 job_2 ... job_N] 73*4bcd95a3SBarry Smith``` 74*4bcd95a3SBarry Smith 75*4bcd95a3SBarry Smithwhere `job_1` is, for example, `linux-intel`. The script will then prompt you to push the branch of the MR, once pushed the listed jobs will automatically be run 76*4bcd95a3SBarry Smithwithout you needing to access the GitLab website to un-pause the pipeline. 77*4bcd95a3SBarry Smith 78*4bcd95a3SBarry Smith(more_test_failures)= 79*4bcd95a3SBarry Smith 80*4bcd95a3SBarry Smith## Examples of pipeline failures 81*4bcd95a3SBarry Smith 82*4bcd95a3SBarry SmithIf your source code is not properly formatted you will see an error from `make checkbadSource`. Always run `make checkbadSource` on your machine 83*4bcd95a3SBarry Smithbefore submitting a pipeline. 84*4bcd95a3SBarry Smith 85*4bcd95a3SBarry Smith:::{figure} /images/developers/badsource.png 86*4bcd95a3SBarry Smith:align: center 87*4bcd95a3SBarry Smith:width: 90% 88*4bcd95a3SBarry Smith 89*4bcd95a3SBarry Smith`checkbadSource` failure. 90*4bcd95a3SBarry Smith::: 91*4bcd95a3SBarry Smith 92*4bcd95a3SBarry Smith:::{figure} /images/developers/another-failure.png 93*4bcd95a3SBarry Smith:align: center 94*4bcd95a3SBarry Smith:width: 90% 95*4bcd95a3SBarry Smith 96*4bcd95a3SBarry SmithA test failing with a PETSc error. 97*4bcd95a3SBarry Smith::: 98*4bcd95a3SBarry Smith 99*4bcd95a3SBarry Smith:::{figure} /images/developers/error-compiling-source.png 100*4bcd95a3SBarry Smith:align: center 101*4bcd95a3SBarry Smith:width: 90% 102*4bcd95a3SBarry Smith 103*4bcd95a3SBarry SmithError in compiling the source code. 104*4bcd95a3SBarry Smith::: 105*4bcd95a3SBarry Smith 106*4bcd95a3SBarry SmithYou can download the `configure.log` file to find the problem using the "Browse" button and following the paths to the configure file. 107*4bcd95a3SBarry Smith 108*4bcd95a3SBarry Smith:::{figure} /images/developers/pipeline-configure.png 109*4bcd95a3SBarry Smith:align: center 110*4bcd95a3SBarry Smith:width: 90% 111*4bcd95a3SBarry Smith 112*4bcd95a3SBarry SmithError in running configure. 113*4bcd95a3SBarry Smith::: 114*4bcd95a3SBarry Smith 115*4bcd95a3SBarry Smith:::{figure} /images/developers/pipeline-configure-browse.png 116*4bcd95a3SBarry Smith:align: center 117*4bcd95a3SBarry Smith:width: 90% 118*4bcd95a3SBarry Smith 119*4bcd95a3SBarry SmithDownloading `configure.log` from a failed pipeline job. 120*4bcd95a3SBarry Smith::: 121*4bcd95a3SBarry Smith 122*4bcd95a3SBarry SmithThe "Retry" button at the top of a previous pipeline or job does **not** use any 123*4bcd95a3SBarry Smithnew changes to the branch you have pushed since that pipeline was started - it retries the 124*4bcd95a3SBarry Smithsame Git commit that was previously tried. The job "Retry" should only be used this way 125*4bcd95a3SBarry Smithwhen you suspect the testing system has some intermittent error unrelated to your branch. 126*4bcd95a3SBarry Smith 127*4bcd95a3SBarry SmithPlease report all "odd" errors in the testing that don’t seem related 128*4bcd95a3SBarry Smithto your branch in the PETSc Discord channel `testing-ci-forum`. 129*4bcd95a3SBarry Smith 130*4bcd95a3SBarry SmithCheck the forum's threads to see if the error is listed and add it there, with a link to your MR (e.g. `!1234`). Otherwise, create a new thread. 131
