[Qt-interest] QHttp upload won't stream?

[Jason H]

I guess the http request is just doing a iodev->readAll()


I think QNetworkAccessManager (and friends) will chunk it.

QNAM has issues in the past (slow transfers, 1/2 rate of my QHttp impl) so I didn't use it. It is supposed to be fixed. 
I just used QHttpHeaders and and socket.




----- Original Message ----
From: Hawkeye Parker <hawkeye.parker at autodesk.com>
To: QT Interest List <qt-interest at trolltech.com>
Sent: Wednesday, September 30, 2009 1:01:16 PM
Subject: [Qt-interest] QHttp upload won't stream?

Hi All,

(Note: also posted on Qt Centre.  Just trying to widen the net.)

Using QT 4.5.2 on Windows XP.

I have a file upload API which uses QHttp to send a PUT request to a WebDav server.  It works fine except for one major problem:  it loads the entire file into memory before it uploads the file.  This is a problem with large files, a show stopper if the client runs out of memory.  See code example below (of course, you need a dav server for it to work).

int QHttp::request ( const QHttpRequestHeader & header, QIODevice * data = 0, QIODevice * to = 0 )

I'm using QHttp::request to start the request.  I can't seem to come up with a device or combination of devices which will stream or otherwise buffer the file from the disk.

My working code (which doesn't stream) just passes in the QFile:

_httpId = _qHttp->request(header, sourceFile);

I tried passing a QDataStream (or TextStream), which is not a QIODevice.  Tried to come up with a combination of QBuffer, QFile, just to see if something would work.  No luck.

I took a look at the QT source.  Interestingly, it _is_ buffering the file read in qhttp.cpp, QHttpPrivate::_q_slotBytesWritten().  The memory load looks to me like it's happening in qsslsocket.cpp, QSslSocket::writeData() when the method memcpy's the buffer.

- Can I use QHttp to do what I want?  It doesn't seem like it, but maybe I'm missing something.  Is this a bug (I don't think so, yet)?
- Is it worth trying QNetworkAccessManager, or am I going to run into the same problem?  (I'm working on trying this now.  Looks to me like QNetworkAccessManager is an entirely new impl.)
- If I need to write custom code to support this, could someone outline a basic approach?  Will I need to create multiple requests for this kind of "chunked" transfer?

Code snippet (builds, haven't tried to run.  my actual code has a bunch of classes abstracting/wrapping a lot of this.)


    QUrl destUrl(QString("http://www.someDAVserver.com/someFolder/someBigFile.dat"));
    QFile* sourceFile = new QFile("D:\\someLocalDir\\someBigFile.dat");

    if (!sourceFile->open(QIODevice::ReadOnly)) {
        //throw
    }

    QHttp::ConnectionMode mode = destUrl.scheme().toLower() == "https" ? QHttp::ConnectionModeHttps : QHttp::ConnectionModeHttp;

    QHttp http;
    http.setHost(destUrl.host(), mode, destUrl.port() == -1 ? 0 : destUrl.port());

    QHttpRequestHeader header("PUT", destUrl.toEncoded(QUrl::RemoveScheme|QUrl::RemoveAuthority));
    header.setValue("Host", destUrl.host());
    header.setValue("Accept-Language", QLocale::system().name());
    header.setValue("Content-Length", QString::number(sourceFile->size()));


    http.request(header, sourceFile);



Thanks in advance,

Hawkeye Parker

_______________________________________________
Qt-interest mailing list
Qt-interest at trolltech.com
http://lists.trolltech.com/mailman/listinfo/qt-interest