[Development] Build Hotspots in the Qt build process

Shane McIntosh mcintosh at cs.queensu.ca
Wed Feb 5 14:17:20 CET 2014


Hi Tobias,

On Feb 5, 2014, at 11:45 AM, Tobias Hunger <tobias.hunger at gmail.com> wrote:

> The list of yours seems to be ordered alphabetically. I guess that is
> not necessarily the order in which we should look at the files:-) This
> list is rather long and slide 36 of your presentation shows that the
> files listed trigger between 90s and more than 8000s of rebuild time!
> That is two orders of magnitude difference, so if we look at this we
> should start by looking at those in the 8000s range:-)
> 
> Could you generate a list of files sorted by "hotness" when you get
> back to your lab? Maybe the distance of the file's datapoint from the
> origin would be a good measure for that? That might need some
> normalizing though.

D’oh! You’re correct! Sorting by “hotness” is a great idea. I will try this out and share the results next week.

> I just went over your list and have not been overly surprised with
> which files are causing long rebuild times: All of them are header
> files that are widely used in Qt and by Qt applications. E.g. almost
> the complete set of non-private headers in qtbase/src/corelib is
> listed.

True. It really isn’t too surprising that the corelib header files trigger slow rebuilds. Maybe it will be more useful when we sort by hotness. Perhaps there are specific corelib headers that will pop out.

> There actually is one .cpp file in the list, which is rather unexpected.

Yes, that was an interesting one. My notes seem to suggest that that file is used in several test binaries. Perhaps that’s why its so slow to rebuild?

> Seeing the same data for Qt webkit would also be cool. That is the
> part of code that feels like it is taking the longest to build in all
> of Qt:-)

Thanks for pointing this out. I will kick off a new analysis with Qt webkit included when I return to the lab :-)

Kind regards,
-Shane


> On Mon, Feb 3, 2014 at 10:39 PM, Shane McIntosh <mcintosh at cs.queensu.ca> wrote:
>> Hi Qt developers!
>> 
>> My name is Shane. I’m a PhD student at Queen’s University in Canada.
>> 
>> I’ve been working on an approach for detecting build hotspots, i.e., files that not only take a long time to rebuild, but also change often. We think that these files are ideal candidates for refactoring that could shave time off of incremental builds that are really impacting software teams.
>> 
>> We came up with an approach that I presented last weekend at FOSDEM ( slides are available here: http://www.slideshare.net/shanemcintosh/identifying-hotspots-in-software-build-process ). One of the projects that we analyzed was Qt.
>> 
>> I bumped into Tobias at FOSDEM and he suggested that I post the list of Qt hotspots here. So, I’ve made the hotspot list available here: http://sailhome.cs.queensu.ca/~shane/content/qt_hotspots.txt
>> 
>> I’m happy to provide a more detailed Qt dataset when I return to my lab next week.
>> 
>> Kind regards,
>> -Shane
>> 
>> P.S.: We are conducting a survey on how build performance is impacting developers ( http://is.gd/DbMRTr ). If you could spare 5 minutes to fill out our survey, we’d really appreciate it!
>> _______________________________________________
>> Development mailing list
>> Development at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development




More information about the Development mailing list