[Qt-interest] QHttp upload won't stream?

Tue Oct 6 19:39:01 CEST 2009

I kinda mentioned that under SSL, it waits until the cypher block is done or the socket is closed. 

I see your point in that it is reporting bytes were sent when they aren't. But what can you do? You've got a hard block device, not a char device. It is similar to base64 encode/decode. You want to fill the block if you can. And you can until you're done. There's no easy answer.

I do both SSL and plaintext http. But I use the the signals I mentioned and it seems to work for me... nwo I am using an old Qt, maybe things changed?

________________________________
From: Info <info at misafe.com>
To: Jason H <scorp1us at yahoo.com>
Cc: qt-interest at trolltech.com
Sent: Tuesday, October 6, 2009 12:49:00 PM
Subject: Re: [Qt-interest] QHttp upload won't stream?

I've tracked it down to the fact that when
QNativeSocketEnginePrivate::nativeWrite() calls WSASend() it returns
SOCKET_ERROR and the reason is WSAEWOULDBLOCK, now this appears to be
because under normal HTTP, only chunks of 4096 are sent which are fine
but using SSL, chunks of 30kB are sent which causes the socket to still
be busy on the next attempt. This is find expect that somewhere it
seems to report that the bytes were sent and read more from the input
device.

That is as far as I've got with it. 

On 6 Oct 2009, at 14:27, Jason H wrote:

In my experience, the content-length header gets sent.
>
>What do you mean by "streaming" then? Streaming to me is you don't know the final length and you'll be using chunked encoding. 
>
>Https is interesting because its block encryption. You can't send 10 bytes and expect 10 bytes to come out without closing the transfer. It'll be buffered until it gets a whole block to encrypt, or it is closed.
>
>
>
>
>
>
________________________________
From: Info <info at misafe.com>
>To: Jason H <scorp1us at yahoo.com>
>Cc: qt-interest at trolltech.com
>Sent: Tuesday, October 6, 2009 9:23:37 AM
>Subject: Re: [Qt-interest] QHttp upload won't stream?
>
>
>But the same method works fine when not using HTTPS, surely that would also require the same informations?
>
>
>The QIODevice (output) does implement bytesAvailable() and return the correct number. Is that what you mean? I also explicitly set the content-length header before the put/post request but it makes no difference.
>
>
>
>On 6 Oct 2009, at 14:19, Jason H wrote:
>
>The problem (as I see it) is that Qt wants to know content-length for the header (by doing bytesAvailible() on the device). If you don't, you can only use chunked encoding, which is not supported from a send perspective. I rolled my own using QHttpHeader and a socket. I do have streamed transfers, however I used signals and slots.
>>
>>One signal is the start message (taking care of headers) then a signal for each data chunk, then ending with a done message which closes the socket.
>>
>>
>>
>>
>>
>>
________________________________
From: Info <info at misafe.com>
>>To: qt-interest at trolltech.com
>>Sent: Tuesday, October 6, 2009 9:00:52 AM
>>Subject: Re: [Qt-interest] QHttp upload won't stream?
>>
>>
>>Hi,
>>
>>
>>Did you get anywhere with this? I'm having exactly the same problem with QNetworkAccessManager and have tracked it back to the same place you have QSslSocket::writeData, so it seems that you will still have the issue if you switch from QHttp to QNetworkAccessManager as they both use the same underlying socket.
>>
>>
>>
>>
>>On 30 Sep 2009, at 18:02, qt-interest-request at trolltech.com wrote:
>>
>>Hi All,
>>>
>>>(Note: also posted on Qt Centre.  Just trying to widen the net.)
>>>
>>>Using QT 4.5.2 on Windows XP.
>>>
>>>I have a file upload API which uses QHttp to send a PUT request to a WebDav server.  It works fine except for one major problem:  it loads the entire file into memory before it uploads the file.  This is a problem with large files, a show stopper if the client runs out of memory.  See code example below (of course, you need a dav server for it to work).
>>>
>>>int QHttp::request ( const QHttpRequestHeader & header, QIODevice * data = 0, QIODevice * to = 0 )
>>>
>>>I'm using QHttp::request to start the request.  I can't seem to come up with a device or combination of devices which will stream or otherwise buffer the file from the disk.
>>>
>>>My working code (which doesn't stream) just passes in the QFile:
>>>
>>>_httpId = _qHttp->request(header, sourceFile);
>>>
>>>I tried passing a QDataStream (or TextStream), which is not a QIODevice.  Tried to come up with a combination of QBuffer, QFile, just to see if something would work.  No luck.
>>>
>>>I took a look at the QT source.  Interestingly, it _is_ buffering the file read in qhttp.cpp, QHttpPrivate::_q_slotBytesWritten().  The memory load looks to me like it's happening in qsslsocket.cpp, QSslSocket::writeData() when the method memcpy's the buffer.
>>>
>>>- Can I use QHttp to do what I want?  It doesn't seem like it, but maybe I'm missing something.  Is this a bug (I don't think so, yet)?
>>>- Is it worth trying QNetworkAccessManager, or am I going to run into the same problem?  (I'm working on trying this now.  Looks to me like QNetworkAccessManager is an entirely new impl.)
>>>- If I need to write custom code to support this, could someone outline a basic approach?  Will I need to create multiple requests for this kind of "chunked" transfer?
>>>
>>>Code snippet (builds, haven't tried to run.  my actual code has a bunch of classes abstracting/wrapping a lot of this.)
>>>
>>>
>>>   QUrl destUrl(QString("http://www.someDAVserver.com/someFolder/someBigFile.dat"));
>>>   QFile* sourceFile = new QFile("D:\\someLocalDir\\someBigFile.dat");
>>>
>>>   if (!sourceFile->open(QIODevice::ReadOnly)) {
>>>       //throw
>>>   }
>>>
>>>   QHttp::ConnectionMode mode = destUrl.scheme().toLower() == "https" ? QHttp::ConnectionModeHttps : QHttp::ConnectionModeHttp;
>>>
>>>   QHttp http;
>>>   http.setHost(destUrl.host(), mode, destUrl.port() == -1 ? 0 : destUrl.port());
>>>
>>>   QHttpRequestHeader header("PUT", destUrl.toEncoded(QUrl::RemoveScheme|QUrl::RemoveAuthority));
>>>   header.setValue("Host", destUrl.host());
>>>   header.setValue("Accept-Language", QLocale::system().name());
>>>   header.setValue("Content-Length", QString::number(sourceFile->size()));
>>>
>>>
>>>   http.request(header, sourceFile);
>>>
>>>
>>>
>>>Thanks in advance,
>>>
>>>Hawkeye Parker
>>>
>>
>>__________________________________________________
>>Do You Yahoo!?
>>Tired of spam? Yahoo! Mail has the best spam protection around 
>>http://mail.yahoo.com
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.qt-project.org/pipermail/qt-interest-old/attachments/20091006/d00c5a37/attachment.html 