[Qt-interest] QHttp upload won't stream?

[Info]

Getting closer to tracking this down I hope. It seems that during the  
"buffering", the calls to QSslSocket::flush() are not behaving. I've  
tracked it down to the call to WSASend() in  
QNativeSocketEnginePrivate::nativeWrite() returning -1 for almost  
every call right up until the buffer is full and then every call to  
WSASend() starts returning the correct value.

I have no idea what a return of -1 from WSASend() means but it is  
either a symptom or the cause.




On 6 Oct 2009, at 14:27, Jason H wrote:

> In my experience, the content-length header gets sent.
>
> What do you mean by "streaming" then? Streaming to me is you don't  
> know the final length and you'll be using chunked encoding.
>
> Https is interesting because its block encryption. You can't send 10  
> bytes and expect 10 bytes to come out without closing the transfer.  
> It'll be buffered until it gets a whole block to encrypt, or it is  
> closed.
>
>
>
> From: Info <info at misafe.com>
> To: Jason H <scorp1us at yahoo.com>
> Cc: qt-interest at trolltech.com
> Sent: Tuesday, October 6, 2009 9:23:37 AM
> Subject: Re: [Qt-interest] QHttp upload won't stream?
>
> But the same method works fine when not using HTTPS, surely that  
> would also require the same informations?
>
> The QIODevice (output) does implement bytesAvailable() and return  
> the correct number. Is that what you mean? I also explicitly set the  
> content-length header before the put/post request but it makes no  
> difference.
>
>
> On 6 Oct 2009, at 14:19, Jason H wrote:
>
>> The problem (as I see it) is that Qt wants to know content-length  
>> for the header (by doing bytesAvailible() on the device). If you  
>> don't, you can only use chunked encoding, which is not supported  
>> from a send perspective. I rolled my own using QHttpHeader and a  
>> socket. I do have streamed transfers, however I used signals and  
>> slots.
>>
>> One signal is the start message (taking care of headers) then a  
>> signal for each data chunk, then ending with a done message which  
>> closes the socket.
>>
>>
>>
>> From: Info <info at misafe.com>
>> To: qt-interest at trolltech.com
>> Sent: Tuesday, October 6, 2009 9:00:52 AM
>> Subject: Re: [Qt-interest] QHttp upload won't stream?
>>
>> Hi,
>>
>> Did you get anywhere with this? I'm having exactly the same problem  
>> with QNetworkAccessManager and have tracked it back to the same  
>> place you have QSslSocket::writeData, so it seems that you will  
>> still have the issue if you switch from QHttp to  
>> QNetworkAccessManager as they both use the same underlying socket.
>>
>>
>> On 30 Sep 2009, at 18:02, qt-interest-request at trolltech.com wrote:
>>
>>> Hi All,
>>>
>>> (Note: also posted on Qt Centre.  Just trying to widen the net.)
>>>
>>> Using QT 4.5.2 on Windows XP.
>>>
>>> I have a file upload API which uses QHttp to send a PUT request to  
>>> a WebDav server.  It works fine except for one major problem:  it  
>>> loads the entire file into memory before it uploads the file.   
>>> This is a problem with large files, a show stopper if the client  
>>> runs out of memory.  See code example below (of course, you need a  
>>> dav server for it to work).
>>>
>>> int QHttp::request ( const QHttpRequestHeader & header, QIODevice  
>>> * data = 0, QIODevice * to = 0 )
>>>
>>> I'm using QHttp::request to start the request.  I can't seem to  
>>> come up with a device or combination of devices which will stream  
>>> or otherwise buffer the file from the disk.
>>>
>>> My working code (which doesn't stream) just passes in the QFile:
>>>
>>> _httpId = _qHttp->request(header, sourceFile);
>>>
>>> I tried passing a QDataStream (or TextStream), which is not a  
>>> QIODevice.  Tried to come up with a combination of QBuffer, QFile,  
>>> just to see if something would work.  No luck.
>>>
>>> I took a look at the QT source.  Interestingly, it _is_ buffering  
>>> the file read in qhttp.cpp, QHttpPrivate::_q_slotBytesWritten().   
>>> The memory load looks to me like it's happening in qsslsocket.cpp,  
>>> QSslSocket::writeData() when the method memcpy's the buffer.
>>>
>>> - Can I use QHttp to do what I want?  It doesn't seem like it, but  
>>> maybe I'm missing something.  Is this a bug (I don't think so, yet)?
>>> - Is it worth trying QNetworkAccessManager, or am I going to run  
>>> into the same problem?  (I'm working on trying this now.  Looks to  
>>> me like QNetworkAccessManager is an entirely new impl.)
>>> - If I need to write custom code to support this, could someone  
>>> outline a basic approach?  Will I need to create multiple requests  
>>> for this kind of "chunked" transfer?
>>>
>>> Code snippet (builds, haven't tried to run.  my actual code has a  
>>> bunch of classes abstracting/wrapping a lot of this.)
>>>
>>>
>>>    QUrl destUrl(QString("http://www.someDAVserver.com/someFolder/someBigFile.dat 
>>> "));
>>>    QFile* sourceFile = new QFile("D:\\someLocalDir\ 
>>> \someBigFile.dat");
>>>
>>>    if (!sourceFile->open(QIODevice::ReadOnly)) {
>>>        //throw
>>>    }
>>>
>>>    QHttp::ConnectionMode mode = destUrl.scheme().toLower() ==  
>>> "https" ? QHttp::ConnectionModeHttps : QHttp::ConnectionModeHttp;
>>>
>>>    QHttp http;
>>>    http.setHost(destUrl.host(), mode, destUrl.port() == -1 ? 0 :  
>>> destUrl.port());
>>>
>>>    QHttpRequestHeader header("PUT", destUrl.toEncoded 
>>> (QUrl::RemoveScheme|QUrl::RemoveAuthority));
>>>    header.setValue("Host", destUrl.host());
>>>    header.setValue("Accept-Language", QLocale::system().name());
>>>    header.setValue("Content-Length", QString::number(sourceFile- 
>>> >size()));
>>>
>>>
>>>    http.request(header, sourceFile);
>>>
>>>
>>>
>>> Thanks in advance,
>>>
>>> Hawkeye Parker
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam? Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.qt-project.org/pipermail/qt-interest-old/attachments/20091006/71d8df99/attachment.html