[Interest] Routing all QML file requests through a custom resource provider

Fri Jun 17 20:32:48 CEST 2016

I think that we do something similar that you try to achieve.

I am working on an application that almost works like a game. During
development our resources on the regular file system of Windows, Mac,...
but when building the final release all ressources are filtered and
compressed in a custom package file format to optimize resources for each
platforms,....

So by using regular file system during the development, there is no issue
with Qt tools including QtCreator and QML editor.

The only little issue we have actually is that we need to extract all qml
files from the package before loading our main.qml to the file system of
the current device when we execute the final release.
Else qml can access every resources from the package.

To make Qt and Qml able to load resources from the package you need to
implement a QAbstractFileEngine and QAbstractFileEngineHandler.

The Engine handler contains only "create(const QString& fileName)" method
that have to instanciate our derived class of QAbstractFileEngine if the
fileName is in our package, else return nullptr to fallback to the regular
file system

We simply prefix all our resources path by "/PACKAGE_NAME/" when setting
source properties in QML. To be precise in QML we call a c++ method that
transform the relative path given in parameter to the full form, this is
this method that prefix the path to force QML to read files from the
package or not.

I also didn't recommend you to use qrc files as it doesn't allow you to
make your application able to reload QML,... when running. We do hot
reloading because QML editor doesn't work if you bind C++ methods.

PS:
This method isn't officially supported by Qt as QAbstractFileEngine
and QAbstractFileEngineHandler are privates.
You will have to use includes like that :

#if QT_VERSION == 0x050402

#   include <QtCore/5.4.2/QtCore/private/qabstractfileengine_p.h>

#elif QT_VERSION == 0x050500

#   include <QtCore/5.5.0/QtCore/private/qabstractfileengine_p.h>

#elif QT_VERSION == 0x050501

#   include <QtCore/5.5.1/QtCore/private/qabstractfileengine_p.h>

#elif QT_VERSION == 0x050600

#   include <QtCore/5.6.0/QtCore/private/qabstractfileengine_p.h>

#elif QT_VERSION == 0x050601

#   include <QtCore/5.6.1/QtCore/private/qabstractfileengine_p.h>

#elif QT_VERSION == 0x050700

#   include <QtCore/5.7.0/QtCore/private/qabstractfileengine_p.h>

#else

#   error

#endif

Feel free to vote for the come back of those APIs :
https://bugreports.qt.io/browse/QTBUG-41387?jql=text%20~%20%22QAbstractFileEngineHandler%22

2016-06-17 18:47 GMT+02:00 Jason H <jhihn at gmx.com>:

> You can of course use all of Qt without QtCreator. I've done it, and it's
> still possible.
>
> When you have a resource, it becomes availible in the binary's asset
> system, and need not be distributed as a file on disk. This can prevent
> tampering if the binary is signed. Also I would assume that it gets loaded
> in the text segment of the executable and winds up on memory pages without
> the EXEC bit set.
>
> http://doc.qt.io/qt-5/resources.html
>
>
> *Sent:* Friday, June 17, 2016 at 12:37 PM
> *From:* "Max Savenkov" <max.savenkov at gmail.com>
> *To:* "Jason H" <jhihn at gmx.com>
> *Cc:* "Dmitry Volosnykh" <dmitry.volosnykh at gmail.com>, "<
> interest at qt-project.org>" <interest at qt-project.org>
>
> *Subject:* Re: [Interest] Routing all QML file requests through a custom
> resource provider
> OK, but if I do not use QtCreator, is there no way to make it work? What
> does adding a qrc file to RESOURCES actually do (besides triggering
> resource compiler)? I'm looking for an understanding here.
>
> 2016-06-17 19:23 GMT+03:00 Jason H <jhihn at gmx.com>:
>>
>>
>> What Dimitry said plus:
>>
>> Add:
>> CONFIG += resources_big
>>
>>
>> to your .pro file, which should get around some long linking times.
>>
>>
>> --------------------------------------
>> Sent: Friday, June 17, 2016 at 5:45 AM
>> From: "Dmitry Volosnykh" <dmitry.volosnykh at gmail.com>
>> To: "Max Savenkov" <max.savenkov at gmail.com>, "<interest at qt-project.org>"
>> <interest at qt-project.org>
>> Subject: Re: [Interest] Routing all QML file requests through a custom
>> resource provider
>>
>> Hi, Max!
>>
>> To be honest, I did not understand in all the details what you are trying
>> to achieve. Anyway that looks way complex indeed.
>>
>> Since you are inclined to pack all you resources into auxiliary file like
>> zip package, you are surely okay with static content. Having this said, you
>> should be fine using conventional .qrc files (this is what you named Qt
>> resource system). These files are easily accessed (i.e. viewed, edited,
>> etc.) from Qt Creator. Qt Designer tools understand them perfectly fine, as
>> well. Also, there should be no issues when importing QML components from
>> your .qml files once you feed QML engine with root .qml file stored inside
>> .qrc file.
>>
>> Regards,
>> Dmitry.
>>
>> On Fri, Jun 17, 2016 at 12:09 PM Max Savenkov <max.savenkov at gmail.com>
>> wrote:Hello,
>>
>> Once again, I'm trying to use Qt (in this case, QtQuick/QML) for game
>> development. The game is going to store all its resources in a package
>> (it might be zip, or a custom format) when deployed, but during
>> development, resources are available in the usual filesystem. This
>> difference is hidden by game's resource provider, which maps resource
>> name to a file or an entry in package.
>>
>> This works for the game itself, but how can I make QML load images,
>> imports and other resources from game's resource provider? (also, I'd
>> like to view my QML files in Designer, which means I can't use custom
>> image providers in QML code itself)
>>
>> I see two possibilities, but I'm not sure of either one.
>>
>> 1) Use dynamic resources produced by rcc. When loading a QML file from
>> C++, also load relevant rcc data (from game's resources) and register it
>> with Qt resource system to make it accessible. This sounds workable, but
>> I'm not sure how exactly to make it happen. Suppose I got a binary
>> buffer from game's resources. How do I register it in Qt?
>>
>> 2) Use a system of crutches.
>> * Register Url Interceptor with my QMLEngine, such that:
>>      - any URL that contains an image is changed into
>> "image:/my_resource_provider/imagename.png"
>>      - any URL that contains qml or qmldir is change into... What? There
>> is no way to register a special loader for QML imports. But I guess I
>> can change it into a network request, by substituting shcema to, for
>> example, "my_provider"
>> * Register image provider with name "my_resource_provider" that loads
>> images from my resource system.
>> * Register a custom network access manager factory/manager/network reply
>> that load QML files from my system instead of network. This doesn't
>> quite work, however, because QML at some point for whatever reason
>> checks that URL used to load qml file is local...
>>
>> So, I'm stumped. This should be WAY easier... And it was, before
>> AbstractFileEngine was made private. But I really don't want to use
>> private APIs if I can avoid it.
>> An argument against AbstractFileEngine was made, that virtual FS should
>> be implemented "at OS level" - but I simply cannot see how a game can
>> implement it at OS level, because it's a game 0_o.
>> _______________________________________________
>> Interest mailing list
>> Interest at qt-project.org[Interest at qt-project.org]
>>
>> http://lists.qt-project.org/mailman/listinfo/interest_______________________________________________
>> Interest mailing list Interest at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/interest[http://lists.qt-project.org/mailman/listinfo/interest]
>
>
> _______________________________________________
> Interest mailing list
> Interest at qt-project.org
> http://lists.qt-project.org/mailman/listinfo/interest
>
>

-- 
Xavier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20160617/c3084f4e/attachment.html>