[Interest] Windows 10 + native QFileDialog + QPrinter + some magic = application hangs

[Harri Porten]

On Wed, 20 Dec 2017, Rainer Wiesenfarth wrote:

> I have a strange problem that is bugging me for a couple of days now. 
> Although I think the chances are low to find someone who experienced the 
> same issue, I would like to give it a try.

Customers of ours reported issues that look related. We consequently filed 
https://bugreports.qt.io/browse/QTBUG-49782 and reference it from our 
Knowledge Base[0].

Harri.

[0] https://kb.froglogic.com/display/KB/Automating+native+or+Qt+%28file%29+dialogs+in+Qt+applications#AutomatingnativeorQt%28file%29dialogsinQtapplications-Generalsolution%231Automatethedialogsvia{{nativeType%28%29}}

> 
> I was able to strip the interesting part of my application down to these lines:
> 
>   QString sProject = QFileDialog::getOpenFileName( this, tr( "Select Project File" ),
>     m_sProjectFile, tr( "Project File (*.prj);;" "All Files (*.*)" ) );   // (1)
> 
>   QPrinter p;   // (2)
> 
>   sProject = QFileDialog::getOpenFileName( this, tr( "Select Project File" ),
>       m_sProjectFile, tr( "Project File (*.prj);;" "All Files (*.*)" ) );   // (3)
> 
> Case 1: Running my application on Windows 10 (four different machines):
> - (1) succeeds
> - (2) succeeds
> - (3) hangs the application in QWindowsNativeFileDialogBase::doExec() at m_fileDialog->Show(owner)
> 
> Case 2: Embedding the above snippet into a small sample application on Windows 10:
> - (1) to (3) all succeed
> 
> Case 3: Setting the QFileDialog::DontUseNativeDialog option on Windows 10:
> - (1) to (3) all succeed
> 
> Case 4: Running my application on Windows 7 (several machines):
> - (1) to (3) all succeed
> 
> I took a close look at the m_fileDialog in QWindowsNativeFileDialogBase for the Windows 10 case that gets stuck:
> - at (1), m_fileDialog references a vtable in comdlg32.dll!const CFileOpenSave
> - at (3), m_fileDialog references a vtable in OneCoreUAPCommonProxyStub.dll!0x...
> 
> So it seems to be related to Windows "umbrella" libraries, but I have no idea how. I looked even deeper into where m_fileDialog is getting assigned and found
> the call to CoCreateInstance(), where CLSID_FileOpenDialog and IID_IFileOpenDialog is passed in both (1) and (3), but in (1) the comdlg32 object is returned
> while in (3) it is the OneCore... object.
> 
> My guess so far:
> 1. The constructor or destructor of QPrinter could lead to the replacement of the object behind CLSID_FileOpenDialog, but
> 2. this replacement not done always, but is triggered by the "some magic" part in my application that is executed before (1)
> 
> ​Any ideas what the "​some magic" part could be? Unfortunately my application is quite complex and can not be stripped down easily...
> 
> ​Cheers, Rainer
> 
> --
> Software Engineer | Trimble Imaging Division
> Rotebühlstraße 81 | 70178 Stuttgart | Germany
> Office +49 711 22881 0 | Fax +49 711 22881 11
> http://www.trimble.com/imaging/ | http://www.inpho.de/
> 
> Trimble Germany GmbH, Am Prime Parc 11, 65479 Raunheim
> Eingetragen beim Amtsgericht Darmstadt unter HRB 83893,
> Geschäftsführer: Dr. Frank Heimberg, Jürgen Kesper
> 
>