[Interest] Problems with Qt5's build form git (release branch) due to qtjsondb and a nasty build setup

[Jan Kundrát]

Hi, I tried compiling Qt5 from git (release branch) again tonight on my 
Linux machine (64bit Gentoo). The last time I did it was with Qt 5.1, so I 
had plenty of old stuff around.

TL;DR version -- despite the fact that I restored git modules as documented 
on the wiki, I still got hard to debug errors and wasted hours of time 
(both human and CPU) on this. A couple of questions/suggestions:

- If a submodule is known [1] to be broken, why is is linked to from the 
release branch of the top-level repo?

- Why is `make first` the default target instead of `make all`?

- How comes that running `make` does not report an error at the very end of 
the process?



The longer version follows now. If it sounds like a rant, I sincerely 
apologize. I realize that some of this might be obvious to a developer who 
builds Qt six times a day, but half of these came as a surprise to me 
despite the fact that I've been using a build-from-source distribution for 
years.

So I followed the docs on the wiki which told me to run the following after 
I realized that I really do not want to fight with branches of submodules 
by hand, and that this is going to be much, much easier.

  $ git pull
  $ git submodule sync
  $ git submodule update --recursive

OK, I actually want the "release" branch -- fair enough, `git checkout 
release && git pull` fixed this. After that, I re-run the `git submodule 
sync` and `git submodule update` again, and just to be sure cleared any 
leftovers via this:

 $ git submodule foreach --recursive "git clean -dfx"

Please note that I did not run `init-repository` because I remember quite 
well that it ate my data before (I have some WIP branches in qttools, etc), 
and I do not want to repeat that experience. The docs talk about doing 
these steps via git commands alone anyway, so why shall I bother when I 
already run it months ago?

Anyway, after restoring the tree to a reasonable state ("release" in the 
top level, and synced submodules to whatever the top-level repo suggested, 
and no local modifications whatsoever) via the commands I showed above, I 
run configure:

  $ ./configure -opensource -confirm-license -no-gtkstyle -system-sqlite 
-prefix /opt/qt5.2

...followed by make:

  $ time make -j4

...which failed, sort of, because after many lines of output, this is what 
I got:

ln -s libQt1JsonDbPartition.so.1.0.0 libQt1JsonDbPartition.so
ln -s libQt1JsonDbPartition.so.1.0.0 libQt1JsonDbPartition.so.1
ln -s libQt1JsonDbPartition.so.1.0.0 libQt1JsonDbPartition.so.1.0
rm -f ../../lib/libQt1JsonDbPartition.so.1.0.0
rm -f ../../lib/libQt1JsonDbPartition.so
rm -f ../../lib/libQt1JsonDbPartition.so.1
rm -f ../../lib/libQt1JsonDbPartition.so.1.0
mv -f libQt1JsonDbPartition.so.1.0.0  ../../lib/ 
mv -f libQt1JsonDbPartition.so ../../lib/ 
mv -f libQt1JsonDbPartition.so.1 ../../lib/ 
mv -f libQt1JsonDbPartition.so.1.0 ../../lib/ 
make[3]: Leaving directory `/home/jkt/work/prog/qt5/qtjsondb/src/partition'
cd client/ && ( test -e Makefile || 
/home/jkt/work/prog/qt5/qtbase/bin/qmake 
/home/jkt/work/prog/qt5/qtjsondb/src/client/client.pro -o Makefile ) && 
make -f Makefile 
Project MESSAGE: perl -w /home/jkt/work/prog/qt5/qtbase/bin/syncqt.pl 
-module QtJsonDb -version 1.0.0 -outdir /home/jkt/work/prog/qt5/qtjsondb 
/home/jkt/work/prog/qt5/qtjsondb
<srcbase> = /home/jkt/work/prog/qt5/qtjsondb 
<outbase> = /home/jkt/work/prog/qt5/qtjsondb 
QtJsonDb: created fwd-include header(s) for <srcbase>/src/client/ { 
qjsondbconnection.h (2), qjsondbconnection_p.h (1), qjsondbflushrequest_p.h 
(1), qjsondbflushrequest_p_p.h (1
), qjsondbglobal.h (2), qjsondblogrequest_p.h (1), qjsondblogrequest_p_p.h 
(1), qjsondbmodelcache_p.h (1), qjsondbmodelutils_p.h (1), qjsondbobject.h 
(2), qjsondbprivatepartitio
n_p.h (1), qjsondbquerymodel_p.h (1), qjsondbquerymodel_p_p.h (1), 
qjsondbreadrequest.h (3), qjsondbreadrequest_p.h (1), qjsondbrequest.h (2), 
qjsondbrequest_p.h (1), qjsondbsta
ndardpaths_p.h (1), qjsondbstrings_p.h (1), qjsondbwatcher.h (3), 
qjsondbwatcher_p.h (1), qjsondbwriterequest.h (5), qjsondbwriterequest_p.h 
(1) }
QtJsonDb: created version header
QtJsonDb: created master header
QtJsonDb: created headers.pri file
Project ERROR: Unknown module(s) in QT: jsondbpartition
make[2]: *** [sub-client-make_first-ordered] Error 3
make[2]: Leaving directory `/home/jkt/work/prog/qt5/qtjsondb/src'
make[1]: *** [sub-src-make_first] Error 2
make[1]: Leaving directory `/home/jkt/work/prog/qt5/qtjsondb'
make: *** [module-qtjsondb-make_first] Error 2
make: *** Waiting for unfinished jobs...

...after which the build carried on for five more minutes. At the end, it 
left no *obvious* indication that it failed; this was displayed when the 
top-level make finished:

g++ -c -pipe -O2 -fno-exceptions -Wall -W -D_REENTRANT -fPIE 
-DQT_NO_EXCEPTIONS -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -DQT_NO_DEBUG 
-DQT_MULTIMEDIAWIDGETS_LIB -DQT_MULTIMEDI
A_LIB -DQT_WIDGETS_LIB -DQT_NETWORK_LIB -DQT_GUI_LIB -DQT_CORE_LIB 
-I/home/jkt/work/prog/qt5/qtbase/mkspecs/linux-g++ -I. 
-I../../../../include -I../../../../include/QtMultimedi
aWidgets -I../../../../include/QtMultimedia 
-I/home/jkt/work/prog/qt5/qtbase/include 
-I/home/jkt/work/prog/qt5/qtbase/include/QtWidgets 
-I/home/jkt/work/prog/qt5/qtbase/include/
QtNetwork -I/home/jkt/work/prog/qt5/qtbase/include/QtGui 
-I/home/jkt/work/prog/qt5/qtbase/include/QtCore -I.moc -o 
.obj/moc_videowidgetsurface.o .moc/moc_videowidgetsurface.cpp
g++ -Wl,-O1 -Wl,-rpath,/opt/qt5.2/lib 
-Wl,-rpath-link,/home/jkt/work/prog/qt5/qtbase/lib -o customvideowidget 
.obj/main.o .obj/videoplayer.o .obj/videowidget.o .obj/videowidgets
urface.o .obj/moc_videoplayer.o .obj/moc_videowidget.o 
.obj/moc_videowidgetsurface.o   -L/home/jkt/work/prog/qt5/qtmultimedia/lib 
-lQt5MultimediaWidgets -L/home/jkt/work/prog/qt
5/qtbase/lib -lQt5Multimedia -lQt5Widgets -lQt5Network -lQt5Gui -lQt5Core 
-lGL -lpthread 
make[5]: Leaving directory 
`/home/jkt/work/prog/qt5/qtmultimedia/examples/multimediawidgets/customvideosurface/customvideowidget'
make[4]: Leaving directory 
`/home/jkt/work/prog/qt5/qtmultimedia/examples/multimediawidgets/customvideosurface'
make[3]: Leaving directory 
`/home/jkt/work/prog/qt5/qtmultimedia/examples/multimediawidgets'
make[2]: Leaving directory `/home/jkt/work/prog/qt5/qtmultimedia/examples'
make[1]: Leaving directory `/home/jkt/work/prog/qt5/qtmultimedia'

real    39m34.330s
user    135m34.843s
sys     6m22.613s


But if one looks closely enough, the process surely failed:

  $ echo $?
  2

...except that any indication about the failure is now five minutes deep in 
the scrollback (this is a SandyBridge i5 with 16 GB of RAM). There is *no* 
user-friendly error message which might possibly tell me that there was an 
error. I have to go actively looking for it via the exit status. This is 
not sane.

But surely I can just re-run make to see what was broken, right? (In fact, 
I was purposely logging the output so I know what happened now that I'm 
trying to debug this, but re-running make is what I would do if I was lazy, 
because it just works on projects with sane build setup.) Nope, not with 
Qt5, because the default target is "first", not "all", and running `make 
-j4` with its default target of "first" just spews plenty of lines of debug 
output saying which directory it descends into, but it does not produce any 
files and finishes succesfully. How exactly am I supposed to know that 
make's default target is completely useless for me, and that I absolutely 
must run `make all`?

In the meanwhile, I've removed "qtjsondb", "qtpim" and "qtdocgallery" from 
the Makefile rules for "all" and "install" and let the build run -- hope it 
finishes over the night. It made it as far as to half of webkit already :).

Cheers,
Jan

[1] https://bugreports.qt-project.org/browse/QTBUG-34932

-- 
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/