[Qt-interest] large xml over the network - problem
franki
franki at franki.eu.org
Wed Apr 27 18:47:59 CEST 2011
Wednesday 27 of April 2011 17:43:35 Jeroen De Wachter napisał(a):
> Hey Marek,
>
> I just remembered I once talked about the QDataStream on the qt IRC
> channel and it might be important for you.
thanks for reply I'll try quint32 or uint32_t.
as to QDataStream I belive I'm using it now, aren't I ?
QByteArray block;
QDataStream out(&block, QIODevice::ReadWrite);
out.setVersion(QDataStream::Qt_4_0);
out << (quint16)0;
out << xml.toString();
out.device()->seek(0);
out << (quint16)(block.size() - sizeof(quint16));
tcpSocket->write(block);
best regards
Marek
>
> If you send a large chunk of data through the QDataStream, if might
> cause trouble with buffering (in combination with the tcp connection) or
> if the connection shuts down (and you may have no way of knowing sending
> the data failed). I don't remember the specifics, I just recall there
> was some discussion.
>
> Maybe you could test the QDataStream stuff and just see if it works (I
> think it's more elegant than breaking up a string into size and data
> fields yourself), but keep your current solution in mind as a fallback.
>
> Just wanted to give you a heads-up...
>
> Kind regards,
>
> Jeroen
>
> On Wed, 2011-04-27 at 17:20 +0200, Jeroen De Wachter wrote:
> > Hey Marek,
> >
> > Whenever the size of a data type is important, I try to use types that
> > specify the size in no uncertain terms (this avoids surprises on other
> > platforms/with other compilers), so I'd use quint32 or uint32_t
> >
> > Your solution with sending the size first should work.
> >
> > However, Qt also provides some serialization/deserialization
> > functionality all on its own (which takes care of communicating the
> > length before sending the type)
> > You should take a look at QDataStream:
> >
> > http://doc.trolltech.com/4.7/qdatastream.html
> >
> > Kind regards,
> >
> > Jeroen
> >
> > On Wed, 2011-04-27 at 16:50 +0200, franki wrote:
> > > Wednesday 27 of April 2011 16:23:22 Jeroen.DeWachter at elis.ugent.be
napisał(a):
> > > > Hi,
> > > >
> > > > You do know quint16 is limited to 65536, right?
> > > > So if you're using strings that are longer than that (and you seem to
> > > > do just that, judging from your example), the length can't be
> > > > specified correctly in the two bytes you are trying to use for it...
> > >
> > > Yes... in my example block.size() on server side is 69496 and quint16
> > > is limited to 65536 so... 65536 - 69496 =- 3960 and in my example quint
> > > size was 3958 (very close), so If I change from quint16 to int (which
> > > is four bytes, or at least I think so?) should it be OK ?
> > >
> > > And answering to Jason, I putted the size of packet at the beginning
> > > because sometimes there are lots of packets emitted from server to
> > > client, each with some unique data, how do I know if last message xml
> > > has ended before I start parsing it? I'm only beginner but wouldn't be
> > > a problem when a few xml's are sended one after another and client
> > > hasn't finished processing of first xml command when two or more has
> > > arrived. You know, what can separate xml's on client side? to process
> > > then one by one? I thought that I can ensure myself that I'm reading
> > > these xml's one by one by putting size at the beginning? (besides it
> > > was in fortune client example)
> > >
> > > best regards
> > > Marek
> > >
> > > > That would certainly be an explanation for the huge discrepancy in
> > > > your block size and the quint size...
> > > >
> > > > Kind regards,
> > > >
> > > > Jeroen
> > > >
> > > > Quoting franki <franki at franki.eu.org>:
> > > > > Hi all,
> > > > >
> > > > > I have client-server application which exchange data using xml's
> > > > > over the net.
> > > > > With relatively small xml's everything goes fine, but when I try to
> > > > > send big ones there is problem with receiving it (not always just
> > > > > from time to time), and if I double the size of xml, they are
> > > > > always lost. Snippet of code on server side:
> > > > >
> > > > > void NetThread::sendToClient(QDomDocument xml) {
> > > > > QByteArray block;
> > > > > QDataStream out(&block, QIODevice::ReadWrite);
> > > > > out.setVersion(QDataStream::Qt_4_0);
> > > > > out << (quint16)0;
> > > > > out << xml.toString();
> > > > > out.device()->seek(0);
> > > > > out << (quint16)(block.size() - sizeof(quint16));
> > > > > qDebug() <<" quint size:"<<(quint16)(block.size() -
> > > > > sizeof(quint16)) qDebug() <<" block size:"<<block.size();
> > > > > qDebug() <<" string size:"<<xml.toString().size();
> > > > > tcpSocket->write(block);
> > > > > }
> > > > >
> > > > > code on client side look's like this (dataSize is quint16):
> > > > > void NetManager::readData() {
> > > > > QDataStream in(socket);
> > > > > in.setVersion(QDataStream::Qt_4_0);
> > > > > if (dataSize == 0) {
> > > > > if (socket->bytesAvailable() < (int)sizeof(quint16))
> > > > > return;
> > > > > in >> dataSize;
> > > > > }
> > > > > if(socket->bytesAvailable() < dataSize)
> > > > > return;
> > > > >
> > > > > QString text;
> > > > > in >> text;
> > > > > qDebug() << " dataSize:"<<dataSize<<" received:"<<text;
> > > > > dataSize=0;
> > > > > this->parseData(text);
> > > > > if(socket->bytesAvailable())
> > > > > this->readData();
> > > > > }
> > > > >
> > > > > On the server side, during transmission debug says:
> > > > > quint size: 3958
> > > > > block size: 69496
> > > > > string size: 34745
> > > > >
> > > > > On client side when there is successful transmision I got:
> > > > > NetManager::readData dataSize: 3958 received: "very
> > > > > long_xml_string"
> > > > >
> > > > > When there is problem with transmission I got:
> > > > > Debug: NetManager::readData bytes: 3958 received: ""
> > > > > Debug: NetManager::parseData parse error
> > > > > Debug: NetManager::readData bytes: 61 received: ""
> > > > > Debug: NetManager::parseData parse error
> > > > > Debug: NetManager::readData bytes: 51 received: ""
> > > > > Debug: NetManager::parseData parse error
> > > > > Debug: NetManager::readData bytes: 56 received: ""
> > > > > Debug: NetManager::parseData parse error
> > > > >
> > > > > So in second example the packet length is the same 3958 but the
> > > > > function is called 4 times and I'm quite sure that application on
> > > > > server side doesn't send anything except for this xml.
> > > > > So please can someone point me on my errors?
> > > > >
> > > > > best regards
> > > > > Marek
> > > > > _______________________________________________
> > > > > Qt-interest mailing list
> > > > > Qt-interest at qt.nokia.com
> > > > > http://lists.qt.nokia.com/mailman/listinfo/qt-interest
> > > >
> > > > ----------------------------------------------------------------
> > > > This message was sent using IMP, the Internet Messaging Program.
> >
> > _______________________________________________
> > Qt-interest mailing list
> > Qt-interest at qt.nokia.com
> > http://lists.qt.nokia.com/mailman/listinfo/qt-interest
More information about the Qt-interest-old
mailing list