[Interest] Windows 10 + native QFileDialog + QPrinter + some magic = application hangs

[Rainer Wiesenfarth]

Hi all,

I have a strange problem that is bugging me for a couple of days now.
Although I think the chances are low to find someone who experienced the
same issue, I would like to give it a try.

I was able to strip the interesting part of my application down to these
lines:

  QString sProject = QFileDialog::getOpenFileName( this, tr( "Select
Project File" ),
    m_sProjectFile, tr( "Project File (*.prj);;" "All Files (*.*)" ) );
 // (1)

  QPrinter p;   // (2)

  sProject = QFileDialog::getOpenFileName( this, tr( "Select Project File"
),
      m_sProjectFile, tr( "Project File (*.prj);;" "All Files (*.*)" ) );
 // (3)

Case 1: Running my application on Windows 10 (four different machines):
- (1) succeeds
- (2) succeeds
- (3) hangs the application in QWindowsNativeFileDialogBase::doExec()
at m_fileDialog->Show(owner)

Case 2: Embedding the above snippet into a small sample application on
Windows 10:
- (1) to (3) all succeed

Case 3: Setting the QFileDialog::DontUseNativeDialog option on Windows 10:
- (1) to (3) all succeed

Case 4: Running my application on Windows 7 (several machines):
- (1) to (3) all succeed

I took a close look at the m_fileDialog in QWindowsNativeFileDialogBase for
the Windows 10 case that gets stuck:
- at (1), m_fileDialog references a vtable in comdlg32.dll!const
CFileOpenSave
- at (3), m_fileDialog references a vtable in
OneCoreUAPCommonProxyStub.dll!0x...

So it seems to be related to Windows "umbrella" libraries, but I have no
idea how. I looked even deeper into where m_fileDialog is getting assigned
and found the call to CoCreateInstance(), where CLSID_FileOpenDialog and
IID_IFileOpenDialog is passed in both (1) and (3), but in (1) the comdlg32
object is returned while in (3) it is the OneCore... object.

My guess so far:
1. The constructor or destructor of QPrinter could lead to the replacement
of the object behind CLSID_FileOpenDialog, but
2. this replacement not done always, but is triggered by the "some magic"
part in my application that is executed before (1)

​Any ideas what the "​some magic" part could be? Unfortunately my
application is quite complex and can not be stripped down easily...

​Cheers, Rainer

-- 
Software Engineer | Trimble Imaging Division
Rotebühlstraße 81 | 70178 Stuttgart | Germany
Office +49 711 22881 0 | Fax +49 711 22881 11
http://www.trimble.com/imaging/ | http://www.inpho.de/

Trimble Germany GmbH, Am Prime Parc 11, 65479 Raunheim
Eingetragen beim Amtsgericht Darmstadt unter HRB 83893,
Geschäftsführer: Dr. Frank Heimberg, Jürgen Kesper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20171220/cde9142b/attachment.html>