[Development] Use make before you push and stage

[marius.storm-olsen at nokia.com]

On 1/25/12 11:32 AM, "ext morten.sorvig at nokia.com<mailto:morten.sorvig at nokia.com>" <morten.sorvig at nokia.com<mailto:morten.sorvig at nokia.com>> wrote:
On Jan 25, 2012, at 10:47 AM, ext jiang.jiang at nokia.com<mailto:jiang.jiang at nokia.com> wrote:

For somewhat big changes and refactor changes, it usually took many iterations
of compile/testing until they finally went in, because it's not always possible
for the developers to try all the different configurations on their development
machine. Would it make sense to setup a semi-final staging area to test that
kind of changes before we finally cherry-pick them to master? It should make
smaller changes integration smoother.

I like the idea, but let's not make the gerrit interface more complicated. Instead the CI system could test all changes that are uploaded, in parallel (and perhaps at off-peak hours to better use available capasity).

While that it would be awesome to have a system which could compile test each uploaded patch set, and preferably have the results available before developers would review them, I think it something that will not scale; at least not without a system which can guarantee proper incremental builds (ccache would be a hack, still waste lots of resources, and only works with GCC.)

Lets take a look at the current numbers:

In the last week (now-7 days) we have across all projects merged 592 uploaded patch sets. Lets assume that we stage 3 patch sets per integration round, and that they are all successful on the first try. That would give us 197 rounds in the CI system. A round in the CI system currently takes ~3h, although that does include running auto tests of course.

In the same time period, we uploaded 1461 patch sets, since most review tasks go through several revisions. To properly evaluate each uploaded patch set, we would need 4383h of processing time, equal to 182.63 days if processed in order. To scale it up to manage that in parallel within 1 day, we'd need 182 more Mac machines, Linux boxes and Windows machines; and that's at the current contribution rate :)

Of course, there are things we can do to streamline it more, and make such a system more likely. So, first we drop running auto tests, just simply compile the code. We'd perhaps be down to ~1h on VMs; measured by the slowest platform in VMs. Let's also drop webkit in the compile, and perhaps reduce the compile time be down to 20min? Then we're down to 20.3 days of compilation.

Could we get there? Perhaps. But we would need quite a few compromising shortcuts to get there, I think :)

--
.marius
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20120125/2d3d53dc/attachment.html>