[Qt-interest] large xml over the network - problem

[Jeroen De Wachter]

Hey Marek,

Whenever the size of a data type is important, I try to use types that
specify the size in no uncertain terms (this avoids surprises on other
platforms/with other compilers), so I'd use quint32 or uint32_t

Your solution with sending the size first should work.

However, Qt also provides some serialization/deserialization
functionality all on its own (which takes care of communicating the
length before sending the type)
You should take a look at QDataStream:

http://doc.trolltech.com/4.7/qdatastream.html

Kind regards,

Jeroen

On Wed, 2011-04-27 at 16:50 +0200, franki wrote:
> Wednesday 27 of April 2011 16:23:22 Jeroen.DeWachter at elis.ugent.be napisał(a):
> > Hi,
> >
> > You do know quint16 is limited to 65536, right?
> > So if you're using strings that are longer than that (and you seem to do
> > just that, judging from your example), the length can't be specified
> > correctly in the two bytes you are trying to use for it...
> 
> Yes... in my example block.size() on server side is 69496 and quint16 is 
> limited to 65536 so... 65536 - 69496 =- 3960 and in my example quint size was 
> 3958 (very close), so If I change from quint16 to int (which is four bytes, 
> or at least I think so?) should it be OK ?
> 
> And answering to Jason, I putted the size of packet at the beginning because 
> sometimes there are lots of packets emitted from server to client, each with 
> some unique data, how do I know if last message xml has ended before I start 
> parsing it? I'm only beginner but wouldn't be a problem when a few xml's are 
> sended one after another and client hasn't finished processing of first xml 
> command when two or more has arrived. You know, what can separate xml's on 
> client side? to process then one by one? I thought that I can ensure myself 
> that I'm reading these xml's one by one by putting size at the beginning? 
> (besides it was in fortune client example)
> 
> best regards
> Marek
> 
> >
> > That would certainly be an explanation for the huge discrepancy in your
> > block size and the quint size...
> >
> > Kind regards,
> >
> > Jeroen
> >
> > Quoting franki <franki at franki.eu.org>:
> > > Hi all,
> > >
> > > I have client-server application which exchange data using xml's
> > > over the net.
> > > With relatively small xml's everything goes fine, but when I try to send
> > > big ones there is problem with receiving it (not always just from time to
> > > time), and if I double the size of xml, they are always lost.
> > > Snippet of code on server side:
> > >
> > > void NetThread::sendToClient(QDomDocument xml) {
> > >     QByteArray block;
> > >     QDataStream out(&block, QIODevice::ReadWrite);
> > >     out.setVersion(QDataStream::Qt_4_0);
> > >     out << (quint16)0;
> > >     out << xml.toString();
> > >     out.device()->seek(0);
> > >     out << (quint16)(block.size() - sizeof(quint16));
> > >     qDebug() <<" quint size:"<<(quint16)(block.size() - sizeof(quint16))
> > >     qDebug() <<" block size:"<<block.size();
> > >     qDebug() <<" string size:"<<xml.toString().size();
> > >     tcpSocket->write(block);
> > > }
> > >
> > > code on client side look's like this (dataSize is quint16):
> > > void NetManager::readData() {
> > >     QDataStream in(socket);
> > >     in.setVersion(QDataStream::Qt_4_0);
> > >     if (dataSize == 0) {
> > >         if (socket->bytesAvailable() < (int)sizeof(quint16))
> > >             return;
> > >         in >> dataSize;
> > >     }
> > >     if(socket->bytesAvailable() < dataSize)
> > >              return;
> > >
> > >     QString text;
> > >     in >> text;
> > >     qDebug() << " dataSize:"<<dataSize<<" received:"<<text;
> > >     dataSize=0;
> > >     this->parseData(text);
> > >     if(socket->bytesAvailable())
> > >         this->readData();
> > > }
> > >
> > > On the server side, during transmission debug says:
> > > quint size: 3958
> > > block size: 69496
> > > string size: 34745
> > >
> > > On client side when there is successful transmision I got:
> > > NetManager::readData dataSize: 3958  received: "very long_xml_string"
> > >
> > > When there is problem with transmission I got:
> > > Debug: NetManager::readData bytes: 3958  received: ""
> > > Debug: NetManager::parseData parse error
> > > Debug: NetManager::readData bytes: 61  received: ""
> > > Debug: NetManager::parseData parse error
> > > Debug: NetManager::readData bytes: 51  received: ""
> > > Debug: NetManager::parseData parse error
> > > Debug: NetManager::readData bytes: 56  received: ""
> > > Debug: NetManager::parseData parse error
> > >
> > > So in second example the packet length is the same 3958 but the function
> > > is called 4 times and I'm quite sure that application on server side
> > > doesn't send anything except for this xml.
> > > So please can someone point me on my errors?
> > >
> > > best regards
> > > Marek
> > > _______________________________________________
> > > Qt-interest mailing list
> > > Qt-interest at qt.nokia.com
> > > http://lists.qt.nokia.com/mailman/listinfo/qt-interest
> >
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> 
>