[Development] [Qt3D] RFC: API for back-end callbacks

[Jan Marker]

Hi,

This mail is somewhat lengthy in order to provide background - if you know Qt 3D 
already you can skip to "Problem Statement".

# Background:
Qt 3D has a front-end/back-end architecture:
- front-end: developer sets up the scene with objects, input handlers, ...
     (runs on the main thread)
- Qt 3D copies this information over to back-end data structures
- back-end: executes algorithms and rendering on a thread pool on the back-end's 
data structures (not the front-end's)
    (e.g. animations that change positions of objects)
- Qt 3D notifies the front-end of changes
    (e.g. of position changes due to animations)

The developer is not aware of the back-end, the back-end is private API.

Given two cubes CubeA and CubeB. CubeA is animated to move vertically on the 
screen and be mirrored by the y axis. CubeB should follow CubeA's movement 
without being animated itself.
(see https://imagebin.ca/v/3UgjoVpAmXME, right is animated, left follows)
Currently, this is solved by QFrameAction: A QFrameAction is executed every 
frame (e.g. 60 times per second) and can access *front-end* data structures both 
to read and write, for example (dt is the time between this frame and the 
previous frame):
Qt3DLogic::QFrameAction *frameAction = new Qt3DLogic::QFrameAction(rootEntity);
QObject::connect(frameAction, &Qt3DLogic::QFrameAction::triggered,
                   frameAction, [cubeA, cubeB](float dt) {
                       Q_UNUSED(dt);
                       const auto t = cubeA->translation();
                       cubeB->setTranslation({-t.x(), t.y(), t.z()});
});

In general, QFrameAction is a function with a context (dt) that takes parameters 
and outputs a result.

# Problem statement:
QFrameAction operates on the *front-end*, therefore it does not run on the 
thread-pool and is not parallelized. If the main thread is heavily loaded or 
blocked this will delay or block the back-end, too, which might slow or block 
rendering. It also makes it impossible to have the result of the QFrameAction 
available in the current frame because of front-end/back-end synchronization.

The idea now is to provide a replacement for QFrameAction that can run on the 
back-end. Because the back-end API is private, there needs to be some kind of 
API contract on how the replacement takes parameters and provides a result.
We call this "QNodeBinding".

We have created multiple drafts of API contracts and are seeking your feedback 
with regard to:
- usability
- maintainability
- general concerns

The drafts are (table of contents):
structs                   |QVariantMap             |positional arguments
--------------------------|------------------------|--------------------------
(with QNodeBindingContext)|with QNodeBindingContext|(with QNodeBindingContext)

QNodeBindingContext can be combined with any other solution, the examples I 
provide do not contain the solutions "structs with QNodeBindingContext" and 
"positional arguments with QNodeBindingContext".

## structs
struct Input { QVector3D translation; }
struct Output { QVector3D translation; }
auto *action= new Qt3DLogic::QNodeBinding<Input, Output>(rootEntity);
action->addInput(cubeA, "translation", &Input::translation);
action->addOutput(&Output::translation, cubeB, "translation");
action->setAction([](float dt, Input *input, Output *output) {
      Q_UNUSED(dt)
      const auto &t = input->translation;
      output->translation = QVector3D{-t.x(), t.y(), t.z()};
});
proof-of-concept: https://codereview.qt-project.org/#/c/200263/
+ no unmarshalling of parameters
+ named input parameters
+ less chance for typos
- Input and Output structs need to be declared
- implementation relies heavily on templates

## QVariantMap
auto *action = new Qt3DLogic::QNodeBinding(rootEntity);
action->addInput(cubeA, "translation", "translationIn");
action->addOutput("translationOut", cubeB, "translation");
action->setAction([](float dt, const QVariantMap &inputs, QVariantMap *outputs) {
      Q_UNUSED(dt)
      const auto &t = inputs[QLatin1String("translationIn")].value<QVector3D>();
      outputs->insert(QLatin1String("translationOut"),
                      QVector3D{-t.x(), t.y(), t.z()});
});
proof-of-concept: https://codereview.qt-project.org/#/c/200834/
- unmarshalling of parameters
- easy to make typos
+ no need to define input and output structs

## positional arguments
auto *action = new Qt3DLogic::QNodeBinding(rootEntity);
action->addInput(cubeA, "translation");
action.->addInput(cubeA, "scale"); // for demonstration
action->addOutput(cubeB, "translation");
action->setAction(
      [](float dt, const QVector3D& translation, double s) -> QVector<QVariant> {
          Q_UNUSED(dt)
          const auto &t = translation;
          return {QVector3D{-t.x(), t.y(), t.z()}};
      }
);
proof-of-concept: https://codereview.qt-project.org/#/c/200833/
- possibly bugs due to wrong positioning
    (if parameters translation and s would be swapped)
- implementation relies on templates
+ no unmarshalling of parameters
+ named parameters

## QVariantMap with QNodeBindingContext
auto *action = new Qt3DLogic::QNodeBinding(rootEntity);
action->addInput(cubeA, "translation", "translationIn");
action->addOutput("translationOut", cubeB, "translation");
action->setAction([](Qt3DLogic::QNodeBindingContext &context,
                       const QVariantMap &inputs) {
      const auto dt = context.dt();
      const auto &t = inputs[QLatin1String("translationIn")].value<QVector3D>();
      context.emitChange(QLatin1String("translationOut"),
                         QVector3D(-t.x(), t.y(), t.z()));
});
proof-of-concept: https://codereview.qt-project.org/#/c/200835/
+ context can have new properties without hurting ABI compatibility
+ only output values that have changed need to be emitted
    (would require branching in the lambda)
- higher chance of typos

Any kind of feedback is highly appreciated.
If something is unclear or information is missing I'll be happy to elaborate.

Thanks for reading :-)

Jan

-- 
Jan Marker | jan.marker at kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3988 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.qt-project.org/pipermail/development/attachments/20170726/558c183d/attachment.bin>