[Development] Suggestion for new CI: "early warning" for qt5.git

[Kent Hansen]

Den 30. jan. 2012 06:03, skrev Rohan McGovern:
>
> As Joao mentioned, this relates to "a dry run of the CI" feature.
> I think that we need a method of requesting that some test procedure is
> run over your change in gerrit, without also testing some set of everyone
> else's changes at the same time, and without requesting that the change
> is actually merged into the repository at the end of the test.

Definitely, being able to test a change in isolation would be required, 
I think; otherwise you lose the confidence in the test results again 
("was it me, or did that other change sneak in and break everything?").

(That's not a problem specific to testing/"dry runs", though; right now 
anyone could be the unlucky person that gets his/her change staged 
together with an unrelated change that breaks stuff. I don't have a good 
suggestion for reducing that problem; Jedrzej had the idea of limiting 
the number of unique commit authors per CI round to some constant N (3 
or 4?).)


> You mentioned a button in gerrit, but what do you think of soliciting
> machine reviewers instead, as you currently do for humans?  e.g.
>
>    1. push change to gerrit
>    2. go to web UI, add reviewer named something like
>    "Test : qt5.git tester : compile/test all of qt5.git with your change"
>    3. qt5.git tester immediately adds comment "I'll start testing your
>    change in (~ some time estimate)"
>    4. qt5.git tester later adds pass or fail comment, with a link to a
>    build/test log.
>
> 3 could be omitted if that's perceived as too noisy.
>
> You could add multiple testers with different goals this way, and they'd
> operate independent from each other.

That's a great idea! I'd start using it today. :-)
Maybe the "reviewer bots" could be displayed as "suggested reviewers" or 
something to make their existence more known in the web UI.

Regards,
Kent