[Qt-creator] [cmake-developers] CMake daemon for user tools

[Milian Wolff]

On Dienstag, 12. Januar 2016 10:05:08 CET Brad King wrote:
> On 01/12/2016 05:15 AM, Tobias Hunger wrote:
> > I read Brad's reply as meaning that your cleanup work is a great
> > benefit to cmake, but will probably not be available in the
> > foreseeable future (if at all).
> 
> Most of Stephen's work completed so far is already in CMake 3.4, and
> a bit more is in 'master' and will be in 3.5.  It has no public-facing
> changes though so yes more work will be needed to get a solution IDEs
> can actually use.
> 
> > Alexander goes back to the generator approach we discussed a year ago
> > and explicitly says he won't work on that, so nothing will happen
> > there.
> 
> The generate-json-description approach remains a valid alternative.
> Aleix's work on it got pretty far before Stephen proposed the daemon
> alternative.  See more below.  IIRC Alexander Neundorf raised concerns
> only about how to activate the generation of the json description, and
> has now proposed an approach to resolve those concerns.
> 
> > So could we please get a *documented* way that an IDE can *rely on to
> > be available* that provides basic information on a project/configuration:
> > 
> > * Source directory, build directory
> > * Files that belong to the project (sources, headers, custom cmake
> > 
> >   modules, CMakeLists.txts, resources, ...)
> > 
> > * Files that belong to the build system (anything that needs watching
> > 
> >   to re-generate the IDE-integration information at the right times)
> > 
> > * Targets that can be built
> > * Files that belong to each target
> > * Compiler flags
> > * Defines in effect
> > * Include paths used
> > * The compilers used to build those files
> 
> Thank you for summarizing the goal so succinctly.  This is different than
> I had in mind when writing my previous response.  Before I was thinking
> this was about IDEs helping users edit the CMake build system description
> itself (adding sources, etc.).  The above is about IDEs helping users
> edit their C code with knowledge of how it will be built, thus providing
> for completion and such.
> 
> These problems are orthogonal, and your requirements certainly need
> generation-time information regardless of how the build specification
> itself is represented.  Therefore the discussion here should proceed
> independent of where the alternative language thread goes.
> 
> With that cleared up design discussion can proceed.
>
> ----------------------------------------------------------------------------
> -
> 
> IIUC two approaches have been proposed:
> 
> 1.  CMake generates a JSON description of the above-listed information
>     about the entire build tree for IDEs to use as needed.  It can be
>     either one giant file or an index and a bunch of per-target files.
>     The contract between CMake and IDEs is a JSON schema and file(s) on
>     disk.  Updating requires the IDE to do a full re-run of CMake.
> 
>     Aleix Pol has implemented a prototype of this approach.
> 
> 2.  CMake offers a daemon that IDEs can contact (e.g. via named pipes)
>     to ask for the information.  The contract between CMake and IDEs is
>     a (to-be-designed) protocol (e.g. JSON snippets).  Updating would
>     likely be handled internally by the daemon in response to changes
>     to the CMake input files between requests from the IDE.
> 
>     Stephen Kelly has done refactoring work on CMake internals that
>     will be needed to implement such a daemon, but much more work is
>     still needed.
> 
> Stephen no longer has time to continue his work except to guide others.
> Aleix is still interested in working on a solution but that makes sense
> only after we all agree on an achievable one.  Is anyone else interested
> in working on this?

I think Stephen has /some/ time, but needs help to get it to a useable state 
without burning out in the process.

Also, while above you say there are two orthogonal features (which is 
correct), there is an important fact that has not yet been stressed enough:

The daemon approach of Stephen allows for the implementation of both. And both 
of these features are crucial and important to have.

> Stephen Kelly started this discussion thread with the following summary
> of the advantages of the daemon approach over the generated JSON files:
> 
> On 01/10/2016 06:10 AM, Stephen Kelly wrote:
> > This design is independent of any generator, and compared to the
> > solution of writing json to a file, this solution doesn't require
> > generating anything that is not asked for.
> > This also side-steps the problem of 'stale' files remaining
> > in the build directory, and the problem of needing to trigger
> > re-generation
> > of the file, or watching it for changes.  This solution also allows
> > parameters (eg the config) to be specified in requests, side-stepping a
> > difficulty we encountered before to represent things like that.  This
> > solution also does not require actually generating the buildsystem
> > files. Only the configure and compute steps of cmake are executed.
> 
> Given the above summaries my reaction is that the daemon approach is
> more elegant and desirable in the long run.  However, it will require
> much more time and resources to implement.  It is not clear how much
> more, or if anyone has such time and resources to offer.
> 
> An intermediate solution could be to offer the daemon and protocol
> to IDEs but to implement it internally on top of pre-generated JSON
> files.  The schema of the JSON files would remain a private contract
> between "cmake" and "cmake-daemon".  Then over time the implementation
> of the daemon could be improved by computing things internally on
> demand instead of updating and reading the JSON files, but the public-
> facing protocol would not change.  We just need to make sure that the
> protocol is designed to allow such flexibility.  OTOH this approach
> requires designing both the daemon protocol and the JSON format.
> 
> ----------------------------------------------------------------------------
> -
> 
> Note that the generated JSON approach is essentially what we've been
> doing with the VS IDE since CMake started, just with a different format.
> Many developers use the VS IDE to develop their CMake-built projects,
> and they get IntelliSense for completion.  It has worked well for years.
> Perhaps the argument for the daemon approach puts too much focus on the
> updating case while most developer time is simply spent editing existing
> sources in their IDE.

We in KDevelop have heard and experiences this situation very different. It is 
actually not that hard to get defines, include paths and even targets from 
CMake. But anything that goes beyond that, most notably the mundane "add new 
target" features we've had in KDevelop 4 with our custom CMake parser are 
impossible currently.

And developers do that, very often. It's a nuisance that gets people angry 
because it kills their productivity. Especially in a TDD where you let your 
code evolve as you type, adding files and modifying targets in CMakeLists.txt 
is wasting your time. I, and many others, would really welcome if CMake 
finally could make simple tasks simple.

Cheers

-- 
Milian Wolff
mail at milianw.de
http://milianw.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.qt-project.org/pipermail/qt-creator/attachments/20160112/bbfdd457/attachment.sig>