[Development] Getting QtWebEngineProcess.app to run in sandbox after being signed

Mon May 8 17:35:17 CEST 2017

I just meant are the changes to QtWebEngine you suggested stable enough for
"production". But if it's still using private API, then I guess there's no
point in trying to submit it to MAS.

Damn.

On Mon, May 8, 2017 at 11:32 AM, Alexandru Croitor <alexandru.croitor at qt.io>
wrote:

>
> On 08 May 2017, at 17:22, Adalid Claure <aclaure at gmail.com> wrote:
>
> I changed the string like you suggested and it worked!
>
> Of course, I know this is very hack'ish. How stable do you think this is?
>
>
> I wouldn't know because this is the first time I tried doing it, and
> pretty much nobody tests for this unless they need it.
>
> I've noticed that the Electron framework (based on Chromium) documents the
> whole submit to Mac App Store sandbox / codesign process, so I assume it
> worked for some people in some scenario. To what extent, I wouldn't know.
>
> My app generates all the HTML it displays using QtWebEngine and it's
> nothing complicated. Do you think these changes in QtWebEngine I made per
> your suggestions are worth submitting to the MAS?
>
>
> I'm not sure what you mean by "submitting the changes to MAS". You will
> obviously need the changes to your application, to at least try to submit
> to the MAS, but you will probably fail because of the private API functions
> that are still used in the code.
>
>
> So far though, the plan is to document for Qt 5.9.0 that Qt WebEngine in
> its current state will not be app store compliant (because of the sandbox
> issue, and because of the private API usage).
>
>
> On Mon, May 8, 2017 at 7:41 AM, Alexandru Croitor <alexandru.croitor at qt.io
> > wrote:
>
>>
>> On 08 May 2017, at 13:33, Adalid Claure <aclaure at gmail.com> wrote:
>>
>> > Well, to be fair, it seems you haven't heeded my advice, and are going
>> with the approach of sandboxing the
>> > whole application, instead of leaving that to be done by Chromium.
>>
>> Hmm.. I'm not sure what you mean. How exactly should I be signing my app
>> then? Per the MAS, I have to sign it my app with "
>> com.apple.security.app-sandbox".
>>
>>
>> I would have expected that signing without sandboxing should be possible.
>> But if it is absolutely required, then that's unfortunate, because of the
>> mentioned "unsupported by Chromium" territory.
>>
>> > Regarding the issues you are seeing now, it is because of the
>> "com.apple.security.application-groups"
>> > entitlement key. That value (team id + "." + random domain-like suffix
>> string) has to be used by Chromium
>> > code, so that access is granted to the cross-process resources. By
>> default Chromium uses
>> > "org.chromium.Chromium" prefix as the application group key, which
>> results in the permission denied
>> > issues.
>>
>> I see. I've been using the <teamid>.<app bundle id> entitlement for my
>> main app, but I guess Chromium's code ignores that and uses this hard coded
>> value?
>>
>>
>> Yes, currently the chromium part of WebEngine ignores the bundle id, and
>> just hardcodes it.
>>
>>
>> > You can modify the "BaseBundleID" method in "/qt_source/qtwebengine/src/
>> > 3rdparty/chromium/base/mac/foundation_util.mm:241" to hardcode the
>> bundle id to your "team_id + suffix"
>> > entitlement key. That made it work for me.
>>
>> Ok, I will give this a try.
>>
>> Thanks again for your help.
>>
>>
>> On Mon, May 8, 2017 at 4:43 AM, Alexandru Croitor <
>> alexandru.croitor at qt.io> wrote:
>>
>>> Hi,
>>>
>>> Well, to be fair, it seems you haven't heeded my advice, and are going
>>> with the approach of sandboxing the whole application, instead of leaving
>>> that to be done by Chromium.
>>>
>>> Regarding the issues you are seeing now, it is because of the
>>> "com.apple.security.application-groups" entitlement key. That value
>>> (team id + "." + random domain-like suffix string) has to be used by
>>> Chromium code, so that access is granted to the cross-process resources. By
>>> default Chromium uses "org.chromium.Chromium" prefix as the application
>>> group key, which results in the permission denied issues.
>>>
>>> You can modify the "BaseBundleID" method in
>>> "/qt_source/qtwebengine/src/3rdparty/chromium/base/mac/found
>>> ation_util.mm:241" to hardcode the bundle id to your "team_id + suffix"
>>> entitlement key. That made it work for me.
>>>
>>> Regards, Alex.
>>>
>>> On 06 May 2017, at 14:46, Adalid Claure <aclaure at gmail.com> wrote:
>>>
>>> FWIW, I tried these changes but like you said I ran into another error
>>> (though, I'm pretty sure it's one I've seen before).
>>>
>>> 5/6/17 08:35:40.000 kernel[0]: Sandbox: QtWebEngineProce(5958) deny(1)
>>> mach-lookup org.chromium.Chromium.rohitfork.5957
>>> 5/6/17 08:35:40.000 kernel[0]: Sandbox: QtWebEngineProce(5959) deny(1)
>>> mach-lookup org.chromium.Chromium.rohitfork.5957
>>> 5/6/17 08:35:40.070 QtWebEngineProcess[5958]: [0506/083540:ERROR:
>>> mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)
>>> 5/6/17 08:35:40.070 QtWebEngineProcess[5959]: [0506/083540:ERROR:
>>> mach_port_broker.mm(43)] bootstrap_look_up: Permission denied (1100)
>>>
>>> Does this look like the other issues you were referring to?
>>>
>>> There is progress in the sense that the QtWebEngine widget is now white,
>>> instead of invisible (since it wasn't loading, I presume) like before. Of
>>> course, nothing renders inside it though.
>>>
>>>
>>>
>>> On Thu, May 4, 2017 at 9:32 AM, Alexandru Croitor <
>>> alexandru.croitor at qt.io> wrote:
>>>
>>>> It's in the function SandboxCompiler::CompileAndApplyProfile in
>>>> qt59_source/qtwebengine/src/3rdparty/chromium/content/common/
>>>> sandbox_mac.mm:150
>>>> the whole if branch.
>>>>
>>>> But please do note my warning, even if it appears to work, it does not
>>>> mean you will not have issues. Chromium has its own special sandbox profile
>>>> (set of rules) that it applies to the subprocess, and it never applies it
>>>> to the main process. So by simply enabling default sandboxing for both, you
>>>> invite untested territories.
>>>>
>>>>
>>>> On 04 May 2017, at 15:26, Adalid Claure <aclaure at gmail.com> wrote:
>>>>
>>>> I am the one who logged the bug.
>>>>
>>>> Can you point me to the line of code that you commented out where you
>>>> were able to get it to work?
>>>>
>>>> On Thu, May 4, 2017 at 9:15 AM, Alexandru Croitor <
>>>> alexandru.croitor at qt.io> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm actually looking into this at the moment, as per the created issue
>>>>> here QTBUG-60443
>>>>>
>>>>> I haven't completely finished my investigation, which I will post to
>>>>> the bug report.
>>>>>
>>>>> But the gist of it is:
>>>>> - the WebEngine / Chromium sub process calls manually sandbox_init,
>>>>> which is the macOS private API for enabling the sandbox. That interferes
>>>>> with the sandbox which you enable via codesign entitlements, which
>>>>> gives the "sandbox reinit denied' issue, and crashed process.
>>>>>
>>>>> - if the above call is commented out, then you need to make sure that
>>>>> the Qt Application is signed with sandbox setup entitlements, and the
>>>>> WebEngineProcess is signed with sandbox "inherit" entitlements, otherwise I
>>>>> couldn't get it to  work.
>>>>>
>>>>> - Both of those entitlements have to specify a valid Team ID + prefix
>>>>> in the entitlment com.apple.security.application-groups
>>>>> key.
>>>>>
>>>>> - Google Chrome itself does not enable sandboxing via codesign
>>>>> entitlements, but only enables it for subprocess via sandbox_init, leaving
>>>>> the main process without a sandbox.
>>>>>
>>>>> Thus my preliminary suggestion is not to enable codesign sandboxing
>>>>> for QtWebengine applications.
>>>>>
>>>>> I'm open to discussion on this though.
>>>>>
>>>>> Note that the private API use inside WebEngine, which is removed via
>>>>> appstore_compliant configure switch, is a separate issue from sandboxing.
>>>>>
>>>>> Regards, Alex.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 04 May 2017, at 14:45, Morten Sørvig <Morten.Sorvig at qt.io> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Not sure if I can be of much help, but:
>>>>>
>>>>> - This thread discusses and solves a similar problem:
>>>>> https://forum.qt.io/topic/49250/solved-qtwebengineprocess-no
>>>>> t-working-in-sandboxed-application
>>>>>
>>>>> - If this could be reduced to a simple sandboxed-app-with-helper-process
>>>>> test case (no QtWebEngine usage), that that’s something I could look at,
>>>>> and something we could eventually add an autotest for.
>>>>>
>>>>>
>>>>> Morten
>>>>>
>>>>>
>>>>> On 28 Apr 2017, at 18:49, Adalid Claure <aclaure at gmail.com> wrote:
>>>>>
>>>>> I have a desktop app that I have been trying to get onto the Mac App
>>>>> store but I have been having problems getting it to run in sandbox mode.
>>>>> For context I am (preferably) using Qt 5.8 running on macOS 10.11.6.
>>>>>
>>>>> The crux seems to be QtWebEngineProcess.app refuses to run after I
>>>>> codesign the bundle. As a result, my QtWebEngine component doesn't load. I
>>>>> am using this QtWebEngine component as part of my app's UI.
>>>>>
>>>>> When the app starts I see the following errors in Console:
>>>>>
>>>>> kernel[0]: Sandbox: QtWebEngineProce(20764) deny(1) mach-lookup
>>>>> org.chromium.Chromium.rohitfork.20763
>>>>> kernel[0]: Sandbox: QtWebEngineProce(20765) deny(1) mach-lookup
>>>>> org.chromium.Chromium.rohitfork.20763
>>>>> QtWebEngineProcess[20764]: [0427/071053:ERROR:mach_broker_mac.mm(52)]
>>>>> bootstrap_look_up: Permission denied (1100)
>>>>> QtWebEngineProcess[20765]: [0427/071053:ERROR:mach_broker_mac.mm(52)]
>>>>> bootstrap_look_up: Permission denied (1100)
>>>>> kernel[0]: Sandbox: QtWebEngineProce(20764) deny(1)
>>>>> forbidden-sandbox-reinit
>>>>>
>>>>> My build process is pretty straight forward:
>>>>>
>>>>> 1. Run macdeployqt on the app, using the -appstore-compliant.
>>>>> 2. Sign all of the Qt Frameworks and PlugIns individually with my
>>>>> app's entitlement file.
>>>>> 3. Sign QtWebEngineProcess.app with the following entitlements file:
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "
>>>>> http://www.apple.com/DTDs/PropertyList-1.0.dtd">
>>>>> <plist version="1.0">
>>>>> <dict>
>>>>>    <key>com.apple.security.app-sandbox</key>
>>>>>    <true/>
>>>>>    <key>com.apple.security.inherit</key>
>>>>>    <true/>
>>>>> </dict>
>>>>> </plist>
>>>>>
>>>>> 4. Call codesign on the overall MyProgram.app bundle with the
>>>>> entitlements file from Step 2.
>>>>>
>>>>> I have tried numerous things all in combination with one another,
>>>>> including:
>>>>>
>>>>> a. built QtWebEngine using WEBENGINE_CONFIG+=use_appstore_compliant_code
>>>>> (per the notes here: https://doc.qt.io/qt-5/qtweben
>>>>> gine-platform-notes.html#mac-app-store-compatibility)
>>>>> b. use macdeployqt's -codesign, even though the binarys have to be
>>>>> signed a second time after this in order to apply the entitlements
>>>>> c. sign QtWebEngineProcess.app with CFBundleIdentifier equal to
>>>>> 'com.qt-project.Qt.QtWebEngineProcess' and with my own app's bundle
>>>>> ID.
>>>>> d. tried linking with Qt 5.7
>>>>> e. tried linking with Qt 5.6.2 which *did* run but then gets rejected
>>>>> by Apple because:
>>>>>
>>>>> -------------------------------
>>>>> Your app uses or references the following non-public API(s):
>>>>>
>>>>> framework: '/System/Library/Frameworks/Ap
>>>>> pKit.framework/Versions/C/AppKit'
>>>>> : NSAccessibilityUnregisterUniqueIdForUIElement
>>>>> : _NSAppendToKillRing
>>>>> : _NSDrawCarbonThemeBezel
>>>>> : _NSDrawCarbonThemeListBox
>>>>> : _NSInitializeKillRing
>>>>> : _NSNewKillRingSequence
>>>>> : _NSPrependToKillRing
>>>>> : _NSSetKillRingToYankedState
>>>>> : _NSYankFromKillRing
>>>>>
>>>>> framework: '/System/Library/Frameworks/Ap
>>>>> plicationServices.framework/Versions/A/ApplicationServices'
>>>>> : CGSSetDenyWindowServerConnections
>>>>> : CGSShutdownServerConnections
>>>>> : CTFontCopyDefaultCascadeList
>>>>>
>>>>> The use of non-public APIs is not permitted on the App Store as it can
>>>>> lead to a poor user experience should these APIs change.
>>>>> -------------------------------
>>>>>
>>>>> I have chronicled a lot of this in this thread here (
>>>>> https://forum.qt.io/topic/78518/sandbox-app-for-the-mac-app
>>>>> -store-with-qt-5-8-and-qtwebengineprocess) but the problem persists.
>>>>>
>>>>> Does anyone have any suggestions? Does anyone know of any apps on the
>>>>> Mac App Store that use QtWebEngine?
>>>>>
>>>>> Thanks.
>>>>> _______________________________________________
>>>>> Development mailing list
>>>>> Development at qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Development mailing list
>>>>> Development at qt-project.org
>>>>> http://lists.qt-project.org/mailman/listinfo/development
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20170508/2272d4e5/attachment.html>