[Interest] QProcess unbuffered

[Thiago Macieira]

[quoting out of order]

On Tuesday, 7 March 2023 12:57:59 PST Björn Schäpers wrote:
>        startInf.cbReserved2 = sizeof(HackedHandlePasser);
>        startInf.lpReserved2 = reinterpret_cast<LPBYTE>(&handles);

These reserved fields are how the runtimes "pass file descriptors" to child 
processes. This mimics the Unix fork() behaviour that the parent's open file 
descriptors are available in the child, without implementing the full fork() 
behaviour. The fields are undocumented (AFAIK) but since both msvcrt and ucrt 
depend on the specific behaviours, they can't change.

That includes these bit fields for flags:

>        #ifndef FOPEN
> #define FOPEN 0x01
>        #endif
>        #ifndef FDEV
> #define FDEV 0x40
>        #endif
>        
>        handles.FlagsPerHandle[0] = 0;
>        std::memset(&handles.FlagsPerHandle[1], FOPEN | FDEV, 2);

According to the UCRT source code, the FDEV flag controls whether the standard 
streams will be buffered or not. The UCRT implementation of isatty() is a check 
to see if this flag is set.

The UCRT source code isn't complete (ucrtbase isn't included AFAICS), so I 
couldn't find all details when trying to give you a more specific answer.

> * I have the HANDLE32 because I start a 32 bit application from within a 64
> bit, as far as I understood the structure needs the right HANDLE size, thus
> for a 64 bit application it should be std:int64_t, but since I don't need
> that I never tested it.

The parent process wouldn't know whether the child is 32-bit or not, so I 
imagine the size must be fixed at 32 bits in order to be inheritable. If you 
have MSVC, you have access to the UCRT source code too and you can search for 
those two reserved fields to see if they hardcode to 32-bit or pointer sizes.

> So when using it with QProcess I think one should copy the handles from the
> STARTUPINFOW structure into this struct, then I think the normal QIODevice
> interface should keep working. If one should clear the STARTF_USESTDHANDLES
> flag I also don't know. I stopped my experiments when I achieved success
> for my "simple" use case.

QProcess uses CreateProcessW directly, not the _spawnv* CRT family of 
functions, which is why file descriptors aren't usually inherited via QProcess. 
I don't know either whether the STARTF_USESTDHANDLES flag must be cleared 
because they're handled at separate times.

If this flag is passed, then the three handles in STARTUPINFO[1] are passed to 
the child as its default handles, by the Win32 call itself. That means those 
handles are installed in the child before the runtime gets a chance to 
initialise and read the file descriptor table passed in those reserved fields. 
Whether the runtime overwrites the standard file descriptors or not, I couldn't 
find in the UCRT sources.

And for the same reason, I can't tell either how the FDEV flag is set for the 
standard file descriptors from outside the file descriptor table.

Moreover, please note that this is *runtime* -driven behaviour, not OS-
mandated. So it's entirely possible that this executable that Scott wants to 
run operates differently because it uses a weird runtime. (Strictly speaking, 
it's the same on Unix systems, but all libcs have the same behaviour)

[1] https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/ns-processthreadsapi-startupinfoa
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Cloud Software Architect - Intel DCAI Cloud Engineering
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5152 bytes
Desc: not available
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20230307/e1d4e955/attachment.bin>