[Development] Enhancement to QIODevice?

[Andrzej Ostruszka]

Thiago, my apologies for replying that late.
For some strange reason gmail has decided to mark your reply as spam
(even though previous conversation was perfectly OK).  Since I see you
active in the mailing list I had to dig deeper to find it.

On Wed, Sep 16, 2015 at 8:17 PM, Thiago Macieira
<thiago.macieira at intel.com> wrote:
[...]
> Well, yeah, it needs to read at least one from the lower device into the
> QIODevice buffer. It will then copy the data again to your buffer. There's no
> way to avoid the double copy, unless device implementation (QSerialPort?)
> already read into the QIODevice buffer. Please talk to Alex Trotsenko to
> optimise this.
>
> But canReadLine() needs to do something similar already. If the data isn't
> buffered yet, it needs to call readData() to copy into the buffer first, then
> read it.

This is why I talked about peek() from the buffer itself since I
wanted to avoid reading from device.
My wish was to handle QSerialPort asynchronously and just peek in the
buffer (and if there is not enough data then just wait for more).

>> C. Add QIODPLB::data() function to buffer which will return "first" (I'm
>> talking in terms of QIODPLB implementation)
>
> Returning access to the internal buffer is something to consider. It would
> allow for one memcpy fewer of the data. But it's not something that we can
> trivially do.
>
[...]
>
> The internal buffer is chunked, so the delimiter may not be in the first buffer.
> And I don't want to provide access to the internal buffers without a lot of
> thought to design, since after that it becomes ABI and we will have less
> freedom to improve.

Is it really chunked?  As I've written previously I'm not wearing
"expert" hat here but QIODevicePrivateLinearBuffer seems to be a nice
and simple linear buffer (with data remaining after read() being moved
to the beginning of buffer).  This buffer is then used in
QIODevicePrivate which is then used in QIODevice.

And in case of QSerialPort on unix reading goes like:
1. Notification from QSocketNotifier ready to read is passed to
QSerialPortPrivate::readNotification()
2. There, buffer.reserve() returns pointer to write to and we read
from port to it

Similar thing is done in case of windows implementation - we schedule
asynchronous read to some internal "readChunkBuffer" and in completion
of this operation we append what was read to the
QIODevicePrivateLinearBuffer.

>From the user perspective this difference is hidden - he gets
readyRead() notification and then it has all read data available in
buffer so I don't see chunking here in the internals - do I miss
anything?

Maybe you are thinking about chunking during reading (in case of
QSerialPort we are reading in 512 chunks but the data already
buffered/available are just one linear array).

If my understanding is correct then I see no problem in adding simple:

const char* QIODevicePrivateLinearBuffer::data() const { return first; }

> QIODevice needs a complete redesign to support zero-copy. The current API
> created in Qt 4 (readData(), writeData(), etc.) is way too limited. The
> problem is that there's no way to do that without a major break in
> compatibility (so, Qt 6) and spending a *lot* of time fixing all the QIODevice
> subclasses (QBuffer, QFileDevice, QAbstractSocket, QProcess, QNetworkReply,
> QSerialPort, plus all the non-Qt code).

I don't want to suggest that significant change.  I mean I don't
object any such attempt but I think I'd be satisfied with this little
addition above.

[...]
> I understand the value of countTo(), but it can spiral out of control really
> quickly. After we add the one containing a single byte delimiter, we'll get
> people asking for:
>  - string searching (like CRLF)
>  - regular expression
>  - state machine searching (CSV: comma, unless inside a "")

That's why I prefer now just opening internal buffer implementation to
allow for read-only access to data already buffered instead of my
original countTo() suggestion.  Any such custom request can be built
by the user by subclassing QSerialPort/... and accessing data
available in buffer.

Best regards
Andrzej