[Interest] Routing all QML file requests through a custom resource provider

Fri Jun 17 22:32:10 CEST 2016

I share your concern about using private APIs but there is no other
solution at the moment.

And if it can reassure you, we are using those APIs since two year on an
application in production without any issue.

I think that those API wasn't pushed public since Qt5 because it have a
better support of all platforms out of the box than Qt4, but for some heavy
applications likes Home Design 3D it still import to be able to customize
our development pipeline.

2016-06-17 21:28 GMT+02:00 Max Savenkov <max.savenkov at gmail.com>:

> Thank you, I knew about this method, but I, as I wrote originally, I want
> to avoid the use of the private APIs if possible. I might vote for that
> bug, though - reasoning behind "privatizing" FileEngine seems a bit shaky
> to me.
>
> 17.06.2016 21:32, Xavier Bigand пишет:
>
> I think that we do something similar that you try to achieve.
>
> I am working on an application that almost works like a game. During
> development our resources on the regular file system of Windows, Mac,...
> but when building the final release all ressources are filtered and
> compressed in a custom package file format to optimize resources for each
> platforms,....
>
> So by using regular file system during the development, there is no issue
> with Qt tools including QtCreator and QML editor.
>
> The only little issue we have actually is that we need to extract all qml
> files from the package before loading our main.qml to the file system of
> the current device when we execute the final release.
> Else qml can access every resources from the package.
>
> To make Qt and Qml able to load resources from the package you need to
> implement a QAbstractFileEngine and QAbstractFileEngineHandler.
>
> The Engine handler contains only "create(const QString& fileName)" method
> that have to instanciate our derived class of QAbstractFileEngine if the
> fileName is in our package, else return nullptr to fallback to the regular
> file system
>
> We simply prefix all our resources path by "/PACKAGE_NAME/" when setting
> source properties in QML. To be precise in QML we call a c++ method that
> transform the relative path given in parameter to the full form, this is
> this method that prefix the path to force QML to read files from the
> package or not.
>
> I also didn't recommend you to use qrc files as it doesn't allow you to
> make your application able to reload QML,... when running. We do hot
> reloading because QML editor doesn't work if you bind C++ methods.
>
>
> PS:
> This method isn't officially supported by Qt as QAbstractFileEngine
> and QAbstractFileEngineHandler are privates.
> You will have to use includes like that :
>
> #if QT_VERSION == 0x050402
>
> #   include <QtCore/5.4.2/QtCore/private/qabstractfileengine_p.h>
>
> #elif QT_VERSION == 0x050500
>
> #   include <QtCore/5.5.0/QtCore/private/qabstractfileengine_p.h>
>
> #elif QT_VERSION == 0x050501
>
> #   include <QtCore/5.5.1/QtCore/private/qabstractfileengine_p.h>
>
> #elif QT_VERSION == 0x050600
>
> #   include <QtCore/5.6.0/QtCore/private/qabstractfileengine_p.h>
>
> #elif QT_VERSION == 0x050601
>
> #   include <QtCore/5.6.1/QtCore/private/qabstractfileengine_p.h>
>
> #elif QT_VERSION == 0x050700
>
> #   include <QtCore/5.7.0/QtCore/private/qabstractfileengine_p.h>
>
> #else
>
> #   error
>
> #endif
>
>
>
> Feel free to vote for the come back of those APIs :
>
> https://bugreports.qt.io/browse/QTBUG-41387?jql=text%20~%20%22QAbstractFileEngineHandler%22
> <https://bugreports.qt.io/browse/QTBUG-41387?jql=text%20%7E%20%22QAbstractFileEngineHandler%22>
>
>
>
>
> 2016-06-17 18:47 GMT+02:00 Jason H <jhihn at gmx.com>:
>
>> You can of course use all of Qt without QtCreator. I've done it, and it's
>> still possible.
>>
>> When you have a resource, it becomes availible in the binary's asset
>> system, and need not be distributed as a file on disk. This can prevent
>> tampering if the binary is signed. Also I would assume that it gets loaded
>> in the text segment of the executable and winds up on memory pages without
>> the EXEC bit set.
>>
>> http://doc.qt.io/qt-5/resources.html
>>
>>
>> *Sent:* Friday, June 17, 2016 at 12:37 PM
>> *From:* "Max Savenkov" < <max.savenkov at gmail.com>max.savenkov at gmail.com>
>> *To:* "Jason H" < <jhihn at gmx.com>jhihn at gmx.com>
>> *Cc:* "Dmitry Volosnykh" < <dmitry.volosnykh at gmail.com>
>> dmitry.volosnykh at gmail.com>, "<interest at qt-project.org>" <
>> interest at qt-project.org>
>>
>> *Subject:* Re: [Interest] Routing all QML file requests through a custom
>> resource provider
>> OK, but if I do not use QtCreator, is there no way to make it work? What
>> does adding a qrc file to RESOURCES actually do (besides triggering
>> resource compiler)? I'm looking for an understanding here.
>>
>> 2016-06-17 19:23 GMT+03:00 Jason H < <http://jhihn@gmx.com>jhihn at gmx.com>:
>>
>>>
>>>
>>> What Dimitry said plus:
>>>
>>> Add:
>>> CONFIG += resources_big
>>>
>>>
>>> to your .pro file, which should get around some long linking times.
>>>
>>>
>>> --------------------------------------
>>> Sent: Friday, June 17, 2016 at 5:45 AM
>>> From: "Dmitry Volosnykh" < <http://dmitry.volosnykh@gmail.com>
>>> dmitry.volosnykh at gmail.com>
>>> To: "Max Savenkov" < <http://max.savenkov@gmail.com>
>>> max.savenkov at gmail.com>, "<interest at qt-project.org>" <
>>> interest at qt-project.org>
>>> Subject: Re: [Interest] Routing all QML file requests through a custom
>>> resource provider
>>>
>>> Hi, Max!
>>>
>>> To be honest, I did not understand in all the details what you are
>>> trying to achieve. Anyway that looks way complex indeed.
>>>
>>> Since you are inclined to pack all you resources into auxiliary file
>>> like zip package, you are surely okay with static content. Having this
>>> said, you should be fine using conventional .qrc files (this is what you
>>> named Qt resource system). These files are easily accessed (i.e. viewed,
>>> edited, etc.) from Qt Creator. Qt Designer tools understand them perfectly
>>> fine, as well. Also, there should be no issues when importing QML
>>> components from your .qml files once you feed QML engine with root .qml
>>> file stored inside .qrc file.
>>>
>>> Regards,
>>> Dmitry.
>>>
>>> On Fri, Jun 17, 2016 at 12:09 PM Max Savenkov <
>>> <http://max.savenkov@gmail.com>max.savenkov at gmail.com> wrote:Hello,
>>>
>>> Once again, I'm trying to use Qt (in this case, QtQuick/QML) for game
>>> development. The game is going to store all its resources in a package
>>> (it might be zip, or a custom format) when deployed, but during
>>> development, resources are available in the usual filesystem. This
>>> difference is hidden by game's resource provider, which maps resource
>>> name to a file or an entry in package.
>>>
>>> This works for the game itself, but how can I make QML load images,
>>> imports and other resources from game's resource provider? (also, I'd
>>> like to view my QML files in Designer, which means I can't use custom
>>> image providers in QML code itself)
>>>
>>> I see two possibilities, but I'm not sure of either one.
>>>
>>> 1) Use dynamic resources produced by rcc. When loading a QML file from
>>> C++, also load relevant rcc data (from game's resources) and register it
>>> with Qt resource system to make it accessible. This sounds workable, but
>>> I'm not sure how exactly to make it happen. Suppose I got a binary
>>> buffer from game's resources. How do I register it in Qt?
>>>
>>> 2) Use a system of crutches.
>>> * Register Url Interceptor with my QMLEngine, such that:
>>>      - any URL that contains an image is changed into
>>> "image:/my_resource_provider/imagename.png"
>>>      - any URL that contains qml or qmldir is change into... What? There
>>> is no way to register a special loader for QML imports. But I guess I
>>> can change it into a network request, by substituting shcema to, for
>>> example, "my_provider"
>>> * Register image provider with name "my_resource_provider" that loads
>>> images from my resource system.
>>> * Register a custom network access manager factory/manager/network reply
>>> that load QML files from my system instead of network. This doesn't
>>> quite work, however, because QML at some point for whatever reason
>>> checks that URL used to load qml file is local...
>>>
>>> So, I'm stumped. This should be WAY easier... And it was, before
>>> AbstractFileEngine was made private. But I really don't want to use
>>> private APIs if I can avoid it.
>>> An argument against AbstractFileEngine was made, that virtual FS should
>>> be implemented "at OS level" - but I simply cannot see how a game can
>>> implement it at OS level, because it's a game 0_o.
>>> _______________________________________________
>>> Interest mailing list
>>> Interest at qt-project.org[ <http://Interest@qt-project.org>
>>> Interest at qt-project.org]
>>>
>>> http://lists.qt-project.org/mailman/listinfo/interest_______________________________________________
>>> Interest mailing list <http://Interest@qt-project.org>
>>> Interest at qt-project.org
>>> http://lists.qt-project.org/mailman/listinfo/interest[http://lists.qt-project.org/mailman/listinfo/interest]
>>
>>
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/interest
>>
>>
>
>
> --
> Xavier
>
>
>

-- 
Xavier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20160617/a3662b08/attachment.html>