14.4.5 Running the scripts

Testing & pushing new commits

lilypond-patchy-staging.py is run without any arguments. It then checks to see if remote/origin/staging is “further ahead” than remote/origin/master.

If there are no new differences between the two branches since the last run check, the script will report something like this:

(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
Success:		No new commits in staging

If there are any differences between the two branches since the last run check, (or if the script cannot for any reason, locate the last instance of a commit that it checked) it will report something like this:

(UTC) Begin LilyPond compile, previous commit at 4726764cb591f622e7893407db0e7d42bcde90d9
Merged staging, now at:	79e98a773b6570cfa28a15775a9dea3d3e54d6b5
	Success:		./autogen.sh --noconfigure
	Success:		/tmp/lilypond-autobuild/configure --disable-optimising
...

and proceed with running make, make test and a make doc. Unlike test-patches.py if all the tests pass, the script then pushes the changes to remote/origin/master.

...
Success:		nice make clean
Success:		nice make -j7 CPU_COUNT=7
Success:		nice make test -j7 CPU_COUNT=7
Success:		nice make doc -j7 CPU_COUNT=7
To ssh://joe@git.sv.gnu.org/srv/git/lilypond.git
   79e98a7..4726764  test-staging -> master
	Success:		pushed to master

Testing new patches

When run without any argument, test-patches.py will check http://code.google.com/p/lilypond/issues/list for all tracker issues that are marked with the label Patch-new. It then scrapes the issue, looking for the last Rietveld URL entered. It then downloads the patch file and applies it to test-master.

Here is an example where two tracker issues labeled as Patch-new were detected:

...
issues [4007, 4008]
Trying issue 4007
Found url: http://codereview.appspot.com/112210043
Found patch: 4007,/home/joe/lilypond-git/issue112210043_1.diff,
Trying issue 4008
Found url: http://codereview.appspot.com/115770043
Found patch: 4008,/home/joe/lilypond-git/issue115770043_1.diff,
Fetching, cloning, compiling master.
...

If run no tracker items with the Patch-New label are found it will report:

issues []

The script can also be run using the tracker issue number(s) as an argument regardless if the Patch-New label has been assigned;

test-patches.py 4006

or

test-patches.py 4006 3992 4020

The script then checks to see if any previously make test-baselines have been generated and if the commit ID of remote/origin/master is different from that previously completed test.

If no previous make test-baseline test is discovered or if the commit ID of remote/origin/master has changed, then a new make test-baseline will run first automatically before the patch is tested:

This shows when the commit ID has changed:

...
(UTC) Begin LilyPond compile, previous commit at 	3f92dcb2c81dcd2755542b57a0a5f2039f29a211
Merged master, now at:	4726764cb591f622e7893407db0e7d42bcde90d9
	Success:		./autogen.sh --noconfigure
	Success:		/tmp/lilypond-autobuild/configure --disable-optimising
	Success:		nice make clean
	Success:		nice make -j3 CPU_COUNT=3
	Success:		nice make test-baseline -j3 CPU_COUNT=3
	Success:		nice make doc -j3 CPU_COUNT=3
	Success:		nice make doc-clean
...

If a previous regression test is discovered and if the commit ID of remote/origin/master has not changed, then the patch will be tested against the previous make test-baseline without the need to re-generate a new one:

...
issues [4009]
Trying issue 4009
Found url: http://codereview.appspot.com/110540043
Found patch: 4009,/home/joe/lilypond-extra/patches/issue110540043_1.diff,
Fetching, cloning, compiling master.
(UTC) Begin LilyPond compile, previous commit at 	4726764cb591f622e7893407db0e7d42bcde90d9
	Success:		No new commits in master
Using test baseline from previous build.
...

The patch is then applied and a make and make check are run. A full make doc is also run if the ‘.lilypond-patchy-config’ file has been edited accordingly;

...
Issue 4009:
Issue 4009: Testing patch issue110540043_1.diff
	Success:		git apply --index /home/joe/lilypond-git/issue112210043_1.diff
	Success:		./autogen.sh --noconfigure
	Success:		/tmp/lilypond-autobuild/configure --disable-optimising
	Success:		nice make clean
	Success:		nice make -j3 CPU_COUNT=3
	Success:		nice make check -j3 CPU_COUNT=3
	Success:		nice make doc -j3 CPU_COUNT=3
...

Once all the tests have run (successfully or not), the script will clean up from the previous patch and, if required, start testing the next issue;

...
Issue 4007: Cleaning up
	Success:		nice make test-clean
	Success:		nice make doc-clean
	Success:		nice make clean
	Success:		git reset --hard
Issue 4007: Done.
Issue 4008:
Issue 4008: Testing patch issue115770043_1_diff
	Success:		git apply --index /home/joe/lilypond-git/issue115770043_1.diff
	Success:		./autogen.sh --noconfigure
...

and so on.

Checking the regression test results

Assuming the patch passed all the make tests, the regression differences will be located in the ‘/test-results/’ directory within the build location for the patch issue number;

/tmp/show-4007/test-results/

Open ‘index.html’ in a browser of your choice to view any differences.

Alternatively if the Firefox browser is installed, then the regression test results can be opened by calling the appropriate ‘show-regtests-’ file located in the auto-compile log location;

sh ~/lilypond-auto-compile-results/show-regtests-4007

See Regression tests.

Reporting test results

Once a patch has been tested and the regression tests have been manually checked, the tracker can be updated manually by editing the tracker issue directly in the web browser or by using two additional python scripts that are included as part of the Patchy suite.

For patches that have passed

Use the accept-patch.py script and run it with the Google issue tracker number (not the Rietveld issue number) as an argument;

accept-patch.py 4007

This will automatically update the tracker issue with the phrase “Patchy the autobot says: passes tests.”.

It is also possible to add additional information to the default message by adding a second argument within double-quote marks.

accept-patch.py 4007 "This also includes a full documentation build."

The tracker issue’s label is then changed automatically to “Patch-review”.

Patches that have failed

Use the reject-patch.py script and run it with the Google issue tracker number (not the Rietveld issue number) as an argument but you must also include a second argument, in double-quotes, stating the reason the patch has been rejected;

reject-patch.py 4007 "Fails the 'make check' test."

Once the reject-patch.py script has been run, the tracker issue’s label is changed automatically to “Patch-Needs_work”.

<< Back to Documentation Index

LilyPond — Contributor’s Guide v2.19.24 (development-branch).