[Qt-creator] CMake daemon for user tools

[Tobias Hunger]

Hello Stephen,

thanks again for your work and interest in IDE integration!

I read Bard's reply as meaning that your cleanup work is a great
benefit to cmake, but will probably not be available in the
foreseeable future (if at all).

Alexander goes back to the generator approach we discussed a year ago
and explicitly says he won't work on that, so nothing will happen
there.

The current state -- as I see it -- is this: While I like cmake for
command line use, it is the worst build system with regards to IDE
integration that I have to support in Qt Creator.  Even autotools
support is less than half the code needed to support cmake -- and
believe me, I never expected to give autotools as a positive
example;-)

So could we please get a *documented* way that an IDE can *rely on to
be available* that provides basic information on a
project/configuration:

* Source directory, build directory
* Files that belong to the project (sources, headers, custom cmake
modules, CMakeLists.txts, resources, ...)
* Files that belong to the build system (anything that needs watching
to re-generate the IDE-integration information at the right times)
* Targets that can be built
* Files that belong to each target
* Compiler flags
* Defines in effect
* Include paths used
* The compilers used to build those files

Ideally I would not need to write files to disk to get that
information. I do not know enough about cmake internals to suggest the
best approach to implement this, but something that can be extended
would be great, as this is really just the very basic information
needed and more information would be welcome at a later point.

Interactive features like syntax highlighting of CMakeFiles or adding
files to the build system in are of little interest to me at this
point: I see little reason to implement anything along those lines
while I can not reliably read the information needed for the code
model.

Best Regards,
Tobias

On Sun, Jan 10, 2016 at 12:10 PM, Stephen Kelly <steveire at gmail.com> wrote:
>
> Hello,
>
> I've been working on adding a daemon mode for cmake to provide
> information to user tools - such as IDEs - about the buildsystem.
>
> Following the discussion about providing metadata for IDEs to consume
> I proposed creating a long-running process which would provide a protocol
> to access information about the buildsystem, and about the content of the
> cmake files themselves:
>
>  http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004
>
> The original post in that thread lists design goals.
>
> This design is independent of any generator, and compared to the solution of
> writing json to a file, this solution doesn't require generating
> anything that is
> not asked for.  This also side-steps the problem of 'stale' files remaining
> in the build directory, and the problem of needing to trigger re-generation
> of the file, or watching it for changes.  This solution also allows
> parameters (eg the config) to be specified in requests, side-stepping a
> difficulty we encountered before to represent things like that.  This
> solution also does not require actually generating the buildsystem
> files. Only the configure and compute steps of cmake are executed.
>
> I am scheduled to give a talk at FOSDEM about the feature and how user
> tools can interact with it:
>
>  https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code
>
> with a preview in Berlin:
>
>  http://www.meetup.com/berlincplusplus/events/227896427/
>
> I have now also pushed a branch to my github clone with the start of the
> server mode:
>
>  https://github.com/steveire/CMake/commits/cmake-daemon
>
> Currently the branch only makes targets and backtraces etc available via
> the protocol.  I have also created a plan for extending the protocol to
> make code completion and variable debugging possible, and analysed the
> depedencies of the tasks:
>
>  http://www.steveire.com/cmake-daemon-tasks.png
>
> However, I can't complete those tasks myself: I don't have
> relevant experience building IDEs to know how best to design the
> protocol, what IDE tools really need, how to design a fail-safe parser
> etc.  Additionally, I think a design which incorporates design ideas
> and implementation from more than one person will be better in the end.
>
> So, this needs to be a collaborative effort if it is to go anywhere, with
> more people writing commits in the cmake repo.
>
> The above (copied below) task list would be enough to get a read-only
> browser of a cmake project quite quickly.
>
> Following that, an effort would be needed to handle the difference
> between the dirty state of an editor and the on-disk file content. That's
> a solvable problem and together with filesystem notifications provided
> by libuv, it will lead to features like code completion.
>
> Also necessary eventually would be to make the cmake parser more
> fail-safe, so that information is still available if the user
> had not yet closed a command argument list for example, and make the
> processing more fail-safe in the case where they have written
> add_subdirectory(foo), but have not yet created foo/CMakeLists.txt.
>
> I would be looking to people who work on IDEs for guidance on how to do
> things like that, as some of them are likely already for handling C++
> code in the IDEs.
>
> So, this needs to be a collaborative effort. I can provide guidance to
> anyone who wishes to participate in changing the source code of CMake
> to make it possible.
>
> Is anyone interested enough in this potential cmake feature to
> join the development effort?
>
> Thanks,
>
> Steve.
>
>
> * Provide basic daemon mode with event loop and communication protocol
> -- Written, but not reviewed
> * Provide protocol to query the buildsystem structure
> -- Written, but not complete or reviewed
> * Extend cmListFileParser to record location of start and end of commands
> * Rewrite cmIfCommand to record which branches are executed
> * Store the location of start and end of user-defined functions
> * Provide protocol to query for unexecuted lines
> * Define a new snapshot type to occur at an arbitrary point
> * Create snapshots after branching to other files (after an include() or
> find_package() or add_subdirectory etc)
> * Create snapshots after points with external effects (file(WRITE),
> configure_file(), execute_process etc)
> * Find most recent snapshot occuring before a specified line
> * Implement feature to start at a specific snapshot and execute commands
> up to a particular line
> * Make server find correct snapshot to start at and execute required
> commands up to specified line
> * Provide protocol to dump variable defintions at a particular line
> * Extract implementation of IssueMessage from cmMakefile and
> cmLocalGenerator into cmake
> * Extract computed target properties from cmTarget::GetProperty into
> cmComputedTargetProperties
> * Implement cmGeneratorTarget::GetProperties in terms of
> cmComputedTargetProperties
> * Move implementation of cmTarget::{Get,Set,Append}Property to cmState
> * Extract all IssueMessage parts of cmake into a cmMessenger
> * Extract the cmSystemTools::Message part of cmMessenger into a virtual
> method
> * Publish messages through server protocol
> * Implement snapshot-versioning of target properties
> * Dump target property content
> * Implement snapshot versioning of policies
> * Implement snapshot-versioning of directory properties
> * Implement snapshot-versioning of global properties
> * Dump directory, global properties and policies
> * Extend protocol to provide dirty buffers from editors, instead of only
> executing on-disk commands
> * Provide code completion protocol
>
> _______________________________________________
> Qt-creator mailing list
> Qt-creator at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/qt-creator