[Qt-creator] JSON wizard: build systems and sub-projects

[Tobias Hunger]

Hi Christian,

On Mon, 2019-05-13 at 13:03 +1200, Christian Gagneraud wrote:
> On Mon, 13 May 2019 at 10:38, Christian Gagneraud <chgans at gmail.com> wrote:
> > One last comments, sorry to sound negative, but currently all project
> > templates are a massive pile of copy/paste, with complex and error
> > prone macro/javascript expressions.
> > That is clearly a sign that something is "far from ideal".
> > I'm not moaning or anything, i'm really just trying to help this
> > wizard thing going. I can't wait to be able to do with qbs/cmake what
> > i can currently do only with qmake.
> 
> <RandomIdea>
> I'm sure i'm not the first one to think that the wizard could use
> python instead of JSON.
> I guess the idea with JSON was to have something declarative (and
> decoupled from QtC internal?), but the introduction of macro and
> javascript shows that there is a need for logic too.
> Python could help here.
> </RandomIdea>

This is how we ended up with the JSON wizards (at least as I remember how
things happened, please correct me if I misremember something!):

Creator used to only know C++-based wizards when it started.

At some point somebody had the great idea of writing a C++ wizard that read
a bit of XML data to have one custom page in an otherwise hard-coded series
of pages. That enabled some wizards to actually be implemented using a bit of
configuration instead of having to do custom C++ code. That was a huge step
forward for creator.

At some point I wanted to write another wizard. I could have used an
XML wizard for what I needed, but I needed to disable one of the default
pages it had and I also wanted two custom pages. So I decided to extend the
XML wizard.

I wanted to be able to have a selection of different pages
(written in C++) available for my new wizard. Plugins should be able to
add pages to the pool of possible pages. A user should be able to combine
these pages using a configuration file (using each page several times if
needed!) and pass extra information to each instance of these pages. One of
these pages was supposed to be a custom page where users can arrange line
edits, etc. to collect arbitrary data from the user.

When I realized that I was breaking the existing XML wizards while redoing
the XML wizard, I decided to switch over to a different format. This would
prevent me from breaking XML wizards users might have written and used for
themselves. I went for JSON instead: It looks similar to QML and readily
supported by Qt. This is a very simple configuration file without any logic,
so there is no need for something as complex as QML!



At the same time Creator got a new feature: The macro expander. It allowed
to use certain variables all over Creator. This included things like the
current time, etc. Some of the existing XML wizards used exactly that kind
of information e.g. when generation comments in the generated files. It was
a pretty obvious idea to use the macro expander in the JSON wizards instead
of redoing the same feature with custom code.

Users liked the macro expander, but wanted to be more flexible with the
output that generated. They wanted to e.g. lowercase a string and do similar
things. So that functionality was added to the macro expander. The wizards
also made us of this functionality in the macro expanders. But the
wizards soon needed quite a few different string manipulations and it became
tedious to implement all of them in the macro expander and especially to
come up with good names for them. So I got the idea to have a macroexpander
that accepts some simple JS strings instead -- replacing all the custom
string manipulation macro expanders. That was flexible enough to handle
basically all the string manipulations any user would ever need and JS is
easy to add to a Qt application. So we ended up with JSON wizards that
supported some JS...



People got surprisingly creative using JS in the JSON wizards! They just used
JS and the custom pages to implement all their logic instead of having their
plugins provide special pages written in C++ and just referencing those pages
in their wizard configuration files.

And this worked surprisingly well, too: We got rid of almost all the C++ based
wizards that were shipped with Creator, leaving only the JSON wizard and the XML
wizard (for compatibility).

Yes, in hindsight I should have opted for QML to define the wizards -- or
something equally powerful, but I completely misjudged where that train
was headed.

Best Regards,
Tobias

-- 
Tobias Hunger, Senior Software Engineer | The Qt Company
The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der
Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B