[Qt-interest] large xml over the network - problem
Jeroen De Wachter
jeroen.dewachter at elis.ugent.be
Wed Apr 27 17:43:35 CEST 2011
Hey Marek,
I just remembered I once talked about the QDataStream on the qt IRC
channel and it might be important for you.
If you send a large chunk of data through the QDataStream, if might
cause trouble with buffering (in combination with the tcp connection) or
if the connection shuts down (and you may have no way of knowing sending
the data failed). I don't remember the specifics, I just recall there
was some discussion.
Maybe you could test the QDataStream stuff and just see if it works (I
think it's more elegant than breaking up a string into size and data
fields yourself), but keep your current solution in mind as a fallback.
Just wanted to give you a heads-up...
Kind regards,
Jeroen
On Wed, 2011-04-27 at 17:20 +0200, Jeroen De Wachter wrote:
> Hey Marek,
>
> Whenever the size of a data type is important, I try to use types that
> specify the size in no uncertain terms (this avoids surprises on other
> platforms/with other compilers), so I'd use quint32 or uint32_t
>
> Your solution with sending the size first should work.
>
> However, Qt also provides some serialization/deserialization
> functionality all on its own (which takes care of communicating the
> length before sending the type)
> You should take a look at QDataStream:
>
> http://doc.trolltech.com/4.7/qdatastream.html
>
> Kind regards,
>
> Jeroen
>
> On Wed, 2011-04-27 at 16:50 +0200, franki wrote:
> > Wednesday 27 of April 2011 16:23:22 Jeroen.DeWachter at elis.ugent.be napisał(a):
> > > Hi,
> > >
> > > You do know quint16 is limited to 65536, right?
> > > So if you're using strings that are longer than that (and you seem to do
> > > just that, judging from your example), the length can't be specified
> > > correctly in the two bytes you are trying to use for it...
> >
> > Yes... in my example block.size() on server side is 69496 and quint16 is
> > limited to 65536 so... 65536 - 69496 =- 3960 and in my example quint size was
> > 3958 (very close), so If I change from quint16 to int (which is four bytes,
> > or at least I think so?) should it be OK ?
> >
> > And answering to Jason, I putted the size of packet at the beginning because
> > sometimes there are lots of packets emitted from server to client, each with
> > some unique data, how do I know if last message xml has ended before I start
> > parsing it? I'm only beginner but wouldn't be a problem when a few xml's are
> > sended one after another and client hasn't finished processing of first xml
> > command when two or more has arrived. You know, what can separate xml's on
> > client side? to process then one by one? I thought that I can ensure myself
> > that I'm reading these xml's one by one by putting size at the beginning?
> > (besides it was in fortune client example)
> >
> > best regards
> > Marek
> >
> > >
> > > That would certainly be an explanation for the huge discrepancy in your
> > > block size and the quint size...
> > >
> > > Kind regards,
> > >
> > > Jeroen
> > >
> > > Quoting franki <franki at franki.eu.org>:
> > > > Hi all,
> > > >
> > > > I have client-server application which exchange data using xml's
> > > > over the net.
> > > > With relatively small xml's everything goes fine, but when I try to send
> > > > big ones there is problem with receiving it (not always just from time to
> > > > time), and if I double the size of xml, they are always lost.
> > > > Snippet of code on server side:
> > > >
> > > > void NetThread::sendToClient(QDomDocument xml) {
> > > > QByteArray block;
> > > > QDataStream out(&block, QIODevice::ReadWrite);
> > > > out.setVersion(QDataStream::Qt_4_0);
> > > > out << (quint16)0;
> > > > out << xml.toString();
> > > > out.device()->seek(0);
> > > > out << (quint16)(block.size() - sizeof(quint16));
> > > > qDebug() <<" quint size:"<<(quint16)(block.size() - sizeof(quint16))
> > > > qDebug() <<" block size:"<<block.size();
> > > > qDebug() <<" string size:"<<xml.toString().size();
> > > > tcpSocket->write(block);
> > > > }
> > > >
> > > > code on client side look's like this (dataSize is quint16):
> > > > void NetManager::readData() {
> > > > QDataStream in(socket);
> > > > in.setVersion(QDataStream::Qt_4_0);
> > > > if (dataSize == 0) {
> > > > if (socket->bytesAvailable() < (int)sizeof(quint16))
> > > > return;
> > > > in >> dataSize;
> > > > }
> > > > if(socket->bytesAvailable() < dataSize)
> > > > return;
> > > >
> > > > QString text;
> > > > in >> text;
> > > > qDebug() << " dataSize:"<<dataSize<<" received:"<<text;
> > > > dataSize=0;
> > > > this->parseData(text);
> > > > if(socket->bytesAvailable())
> > > > this->readData();
> > > > }
> > > >
> > > > On the server side, during transmission debug says:
> > > > quint size: 3958
> > > > block size: 69496
> > > > string size: 34745
> > > >
> > > > On client side when there is successful transmision I got:
> > > > NetManager::readData dataSize: 3958 received: "very long_xml_string"
> > > >
> > > > When there is problem with transmission I got:
> > > > Debug: NetManager::readData bytes: 3958 received: ""
> > > > Debug: NetManager::parseData parse error
> > > > Debug: NetManager::readData bytes: 61 received: ""
> > > > Debug: NetManager::parseData parse error
> > > > Debug: NetManager::readData bytes: 51 received: ""
> > > > Debug: NetManager::parseData parse error
> > > > Debug: NetManager::readData bytes: 56 received: ""
> > > > Debug: NetManager::parseData parse error
> > > >
> > > > So in second example the packet length is the same 3958 but the function
> > > > is called 4 times and I'm quite sure that application on server side
> > > > doesn't send anything except for this xml.
> > > > So please can someone point me on my errors?
> > > >
> > > > best regards
> > > > Marek
> > > > _______________________________________________
> > > > Qt-interest mailing list
> > > > Qt-interest at qt.nokia.com
> > > > http://lists.qt.nokia.com/mailman/listinfo/qt-interest
> > >
> > > ----------------------------------------------------------------
> > > This message was sent using IMP, the Internet Messaging Program.
> >
> >
>
>
> _______________________________________________
> Qt-interest mailing list
> Qt-interest at qt.nokia.com
> http://lists.qt.nokia.com/mailman/listinfo/qt-interest
More information about the Qt-interest-old
mailing list