[Development] [QIODevice]How to correctly treat/understand of the documentation?

Tue Apr 22 10:28:39 CEST 2014

Hi Thiago, Kuba,

> It should support the same waitForBytesWritten() and waitForReadyRead()
that
> QAbstractSocket and QProcess have. Those are blocking.

Yes, QSerialPort does it (at least tries to do). :)

>Let's make sure we distinguish blocking from buffered. It's quite ok for
>QSerialPort to be unbuffered -- there's a QIODevice flag for it. But it
should
>still be non-blocking.

Yes, in the future we will try to add this opportunity. But I don't see
advantage of Unbuffered mode against Buffered. Because in Unbuffered a mode
is probable loss of data, because the FIFO can be overflowed and so on. At
least this can be occured at reading of data.

 > Acceptable. It's not how QAbstractSocket and QProcess work, but they
could
> change to that someday. And QSerialPort can adopt that policy.

But this decision considerably complicates realization and increases
fragmentation of data (if we speak about Buffered mode). For me this
invention is doubtful.

For example, let's see on the non-blocking buffered mode at this scenario:

MyClass::foo()
{
    dev.write('a');
    dev.write('b');
    dev.write('с');
}

1) In the current implementation of QProcess, QAbstractSocket (I am about
the "deferred" writing, as currently implemented) everything is fine:
at shooting of the notifier (with the Tx empty event), we will send at once
a whole 'abc' package to the device.

There is only one shortcoming: is a some delay between foo() and the
triggering of notifier (can be it is critical ??).
But I don't consider it as a shortcoming.

2) An option which offers by Kuba (if I correctly understood). It is
"direct" writing to the device if it is ready for writing (e.g. when Tx
FIFO is empty).
But in this case we got an fragmentation of data. Because each of
write('a'), write('b'), write('c') will write data directly to the device
(in case the Tx FIFO can be filled).
I don't know as it will affect Unix platforms, but on Windows it is a sheer
hell. Because there each WriteFile() is separate I/O operation with
separate OVERLAPPED structure. And each completion of each overlapped
operation will be fired and led to loading of CPU.

Concerning to QSerialPort we can't refuse from overlapped operations. We
use it completion to emitting the bytesWritten() and so on.

Thus, I don't see advantage of Kuba offer's against to the current
QAbstractSocket, QProcess, etc implementation for Buffered mode. Maybe it
has sense in Unbuffered mode, I don't know (or maybe I do not understand
correctly).

UPD:
If to be honest, I don't agree with implementation with emission of the
bytesWritten() signal in QAbstractSocket. IMHO, this signal has to be
emitted when data were really transferred, i.e. after completion of
transfer, when fired the Tx-empty notifier. But not after the calling of
non-blocking write(fd, ..). Probably, for high-speed interfaces (like
sockets) it is noncritical, but for the serial port it is critical. Because
exist, for example, the hardware RS-485 devices where the direction of
transfer is switched by DTR signal. Thus, it is important to know when data
transmission is really finished to switch to reception. For this purpose it
is possible to use bytesWritten(qint64) signal. BUT, in case same
implementation as in QAbstractSocket, we got an fail, because
bytesWritten() will tell lies to us.

> Example: QWindowsPipeWriter.

It has to be re-implemented to "IO Completion Port" instead of thousands of
threads. :)

BR,
Denis

2014-04-22 11:33 GMT+04:00 Denis Shienkov <denis.shienkov at gmail.com>:

> Hi Kuba,
>
> > The `QSerialPort` API must be nonblocking,
>
> It is truth, currently it is non-blocking.
>
>
> > The writes should be executed right away if they won’t block. The
> semantics would be:
>
> There are some problems, please see below:
>
>
> > 1. Check how many bytes can be written to the OS device buffers without
> blocking.
>
> In most cases it is impossible since OS doesn't provide such API. There is
> only API by means of which it is possible to receive number of bytes in Tx
> FIFO of device (ready to transfer). But a problem what not all
> producers/vendors of drivers of serial ports implements this API. Therefore
> it is impossible to rely on this API and to be sure of it for 100%.
>
>
> > 2a. If the buffer is empty, write as many bytes as possible from the
> data given to write,
> add remainder to the buffer.
> > 2b. If the buffer is non-empty, append the data to the buffer, and drain
> as much of the buffer
> as possible.
>
> It is the continue of  the same situation from the item №1 - impossible be
> implemented.
>
>
> > The reads should be executed right away if they wouldn’t block. The
> semantics would be:
>
> This also has some limitations:
>
>
> > 1. Check how many bytes can be read from the OS device buffers without
> blocking.
>
> Also, not all vendors of devices implements in a drivers these
> possibilities of receiving number of bytes in Rx FIFO  of device (which
> ready to read). Even if we assume, that this API is implemented, then this
> system call is the additional loading on CPU, we need to avoid it.
>
> Currently at data reading is applied an some cunning. We don't get number
> of bytes in Rx FIFO (it isn't necessary for us). We simply read from Rx
> FIFO some fixed pieces (on 512 bytes) of data.
>
> For example, in Rx FIFO came 15 bytes, further, the Rx event shoots and we
> call read(512). In this case the read() will return 15 bytes which we add
> to the buffer.
> Of course, all this occurs inside of QSerialPort and is invisible for the
> user. Everything made is asynchronously and in non-blocking mode. When the
> user will call QSerialPort::read(), then the QSerialPort will return data
> from the internal buffer immediately.
>
> I will repeat once again: at present, the QSerialPort always works in
> asynchronously (in non-blocking mode) and in the buffered mode. At present,
> there I/O are no problems in principle.
>
> Problems begin at implementation of the waifForReadyRead(),
> waitForBytesWritten(), flush() methods. And also bytesWritten() signal.
> Because IMHO, there is no accurate specification and their behavior can be
> implemented in various ways.
>
> BR,
> Denis
>
>
>
>
>
>
>
>
> 2014-04-21 22:31 GMT+04:00 Thiago Macieira <thiago.macieira at intel.com>:
>
> Em seg 21 abr 2014, às 14:21:24, Kuba Ober escreveu:
>> > Essentially, there’s only *two* changes I’d like to see in QSerialPort’s
>> > implementation. These changes may not be possible to implement on all
>> > platforms, of course. The `QSerialPort` API must be nonblocking, that’s
>> the
>> > whole point of having it to start with. Doing blocking reads/writes is
>> > trivial using platform APIs - I don’t think QSerialPort should support
>> any
>> > blocking operations.
>>
>> It should support the same waitForBytesWritten() and waitForReadyRead()
>> that
>> QAbstractSocket and QProcess have. Those are blocking.
>>
>> Let's make sure we distinguish blocking from buffered. It's quite ok for
>> QSerialPort to be unbuffered -- there's a QIODevice flag for it. But it
>> should
>> still be non-blocking.
>>
>> The only constraint that puts is on non-buffered writing. The
>> waitForBytesWritten() function operates on the buffered bytes. You may
>> want to
>> add a waitForReadyWrite() for the unbuffered case.
>>
>> > Internally I use classes that behave that way and they work as well as
>> > anything in userspace can ever work, given that the code must be
>> > non-blocking.
>> >
>> > The writes should be executed right away if they won’t block. The
>> semantics
>> > would be:
>> >
>> > 1. Check how many bytes can be written to the OS device buffers without
>> > blocking.
>>
>> Acceptable. It's not how QAbstractSocket and QProcess work, but they could
>> change to that someday. And QSerialPort can adopt that policy.
>>
>> > 2a. If the buffer is empty, write as many bytes as possible from the
>> data
>> > given to write, add remainder to the buffer.
>> >
>> > 2b. If the buffer is non-empty, append the data to the buffer, and
>> drain as
>> > much of the buffer as possible.
>> >
>> > The reads should be executed right away if they wouldn’t block. The
>> > semantics would be:
>> >
>> > 1. Check how many bytes can be read from the OS device buffers without
>> > blocking.
>> >
>> > 2. Get everything from the OS queue and append to the buffer.
>> >
>> > 3. Drain as much of the buffer as requested in the argument to `read`
>> and
>> > return to the user.
>>
>> QIODevice already does that: if the buffer is non-empty, it drains the
>> buffer.
>> If the buffer becomes empty and the user still wants more data, it calls
>> readData().
>>
>> QAbstractSocket and QProcess do not read from the OS device in readData()
>> though. So that readData() used to read from a second buffer in older Qt
>> versions -- now, it doesn't do anything.
>>
>> > On platforms where there is no non-blocking API for step #1, the read
>> and
>> > write is forced to use the buffer only, as it currently does.
>>
>> Not exactly true. If you don't have a non-blocking API, you need threads.
>> Buffers alone don't help you.
>>
>> Example: QWindowsPipeWriter.
>>
>> --
>> Thiago Macieira - thiago.macieira (AT) intel.com
>>   Software Architect - Intel Open Source Technology Center
>>
>> _______________________________________________
>> Development mailing list
>> Development at qt-project.org
>> http://lists.qt-project.org/mailman/listinfo/development
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20140422/3900464e/attachment.html>