[Interest] QWebSocketServer - server randomly stops accepting connections

[alexander golks]

i was going to switch some codebase to websocket server, too,
but reading this lets me think again...

thanks ;)

Am Thu, 4 Apr 2019 16:33:19 +0200
schrieb "Jason H" <jhihn at gmx.com>:

> I looked at the private implementation of QWebSocketServer, QWebSocketServerPrivate.
>  
> I haven't tested it, but there are a few concerns:
>  
> 1. That for errorString(), it first checks if it doesn't have an error then returns the QTcpSocket errorString(). But if you just look at serverError() it only reports it's own error and nothing at the QTcpSocket level. This means that if you only check serverError() you won't catch QTcpServer error. This is probably a bug. QWebSocketProtocol::CloseCode needs to be able to express QAbstractSocket::SocketErrors as well.
>  
> 2. QWebSocketServerPrivate::handshakeReceived() has a check of pTcpSocket->bytesAvailable()).endsWith(QByteArrayLiteral("\r\n\r\n") Such position-reliant checks are bad form. There's no reason for it to end with, it should probably be using contains()
>  
> 3. There are multiple `if` blocks that don't set an error. This isn't wrong, as the handhake bytes may come in slowly and take multiple re-parsings, (something to add to your fuzzer?) but you can wind up in an uncaught error state.
>  
>  
>  
> Sent: Thursday, April 04, 2019 at 10:07 AM
> From: "Jason H" <jhihn at gmx.com>
> To: "Narolewski Jakub" <izowiuz at gmail.com>
> Cc: interest at qt-project.org
> Subject: Re: [Interest] QWebSocketServer - server randomly stops accepting connections
> So, yeah, I was thinking of QTcpServer::incommingConnection(). That function is used in multithreadding because you use the descriptor to make a socket on the proper thread. Looks like QWebSocketServer is not multithreaded. 
>  
> I've used Qt's websockets, and I've used QTcpServer extensively on 5-nines stuff. Never saw anything like this. 
>  
> Next steps: 
> - QWebSocketServer::setMaxPendingConnections() set this to 0, 1, or 100, see what (if anything) changes.
> - Use QTcpServer to and QWebSocketServer::handleConnection(QTcpSocket*) to handle the connection yourself just to get QWebSocketServer out of the initial connection stuff.
>  
> Whatever the cause, I think you're in bug territory, either in Qt or the linux kernel. Both of those is unlikely.  There is one more possibilty, if you're using secure sockets, maybe something is failing at the SSL level? Like your SSL implementation stops working for whatever reason?
>  
> Sent: Thursday, April 04, 2019 at 9:13 AM
> From: "Narolewski Jakub" <izowiuz at gmail.com>
> To: "Jason H" <jhihn at gmx.com>
> Cc: interest at qt-project.org
> Subject: Re: [Interest] QWebSocketServer - server randomly stops accepting connections
> That's the thing. I already handle connections and disconnections in my code - including logging relevant information.
> When I implemented this I heavily based on the example that you linked to.
> WSS communication is single threaded - only way to communicate with the outside world from different thread is to send custom QEvent onto the Server instance.
> QWebSocketServer does not seem to have the method incomingConnection in it's public API or maybe I already went mad :P I react to the QWebSocketServer::newConnection signal.
> 
> Last RST from client at 33 second mark is our client-side timeout.
> This is the code that handles websocket stuff, I have stripped some parts that are less important:
> 
> // server connects
> connect(m_wsServer, &QWebSocketServer::acceptError, this, [this](QAbstractSocket::SocketError socketError) {
>         qCritical() << "QWebSocketServer - acceptError:" << socketError;
> });
> 
> connect(m_wsServer, &QWebSocketServer::peerVerifyError, this, [this](const QSslError& error) {
>         qCritical() << "QWebSocketServer - peerVerifyError:" << error;
> });
> 
> connect(m_wsServer, &QWebSocketServer::serverError, this, [this](QWebSocketProtocol::CloseCode closeCode) {
>         qCritical() << "QWebSocketServer - serverError:" << closeCode;
> });
> 
> connect(m_wsServer, &QWebSocketServer::sslErrors, this, [this](const QList<QSslError>& errors) {
>         qCritical() << "QWebSocketServer - sslErrors:" << errors;
> });
> 
> connect(m_wsServer, &QWebSocketServer::closed, this, [this]() {
>         qCritical() << "QWebSocketServer - closed, serverError():" << m_wsServer->errorString();
> });
> 
> connect(m_wsServer, &QWebSocketServer::newConnection, this, [this]() {
>         QWebSocket* clientSocket = m_wsServer->nextPendingConnection();
>         qInfo() << "Got connection from client:" << clientSocket->peerAddress();
> 
>         [ client initialization stuff ]
> 
>         connect(clientSocket, &QWebSocket::binaryMessageReceived, this, [this](const QByteArray& message) {
>                 auto client = sender()->property("client").value<Client*>();
> 
>                 [ message deserialization ]
>         });
> 
>         connect(clientSocket, &QWebSocket::disconnected, this, [this]() {
>                 auto client = sender()->property("client").value<Client*>();
>                 qInfo() << client->sayHello() << "is disconnected.";
> 
>                 client->handleDisconnection();
> 
>                 [ clinet cleanup stuff ]
>         });
> });
> 
> Client's handleDisconnection() function does:
> 
> void i9ms::Client::handleDisconnection()
> {
>         m_isConnected = false;
>         m_websocket->deleteLater();
>         m_websocket   = nullptr;
> }
> 
> Nothing from this lands in system logs. It seems that QWebSocketServer::newConnection() is just not emited.
> As I wrote earlier I have a timerEvent that logs QWebSocketServer's state - once every ten minutes.
> It happily reports that server is listening, has no pending connections and it's internal 'errorString' is empty - even if it is mute from the outside.
> 
> It's hard for me to pinpoint exactly what causes this. 'Time' is my best guess.
> I'm also not sure if quantity of connections plays a role here.
> 
> I have a 'fuzzer-bot' that spams hundreds of connections and messages at the server in short period of time.
> Malformed messages, out-of-order client state changes, ugly words - you name it we have it.
> QWebSocketServer handles it well enough for us but then, at some time in future WHILE being completely idle, server goes silent.
> On the other hand I can start the server, connect - disconnect one client, leave it for a while in idle and same thing happens, leaving me one step closer to dementia.
>  
> On Wed, 3 Apr 2019, 22:32 Jason H, <jhihn at gmx.com> wrote:
> Addendum, 
> >  
> > How much does you code diverge from the example? https://doc.qt.io/qt-5/echoserver.html
> > Can you hack that to receive your traffic or augment your code to to do what it does?
> > Note that the QWebSocket server does not take ownership of the QWebSocket, you have to track that manually. Using the code they provide, I would also log your connected clients. See the slot EchoServer::socketDisconnected()
> > _______________________________________________ Interest mailing list Interest at qt-project.org https://lists.qt-project.org/listinfo/interest
> 


-- 
/*
 *  printk(KERN_ERR "happy meal: Transceiver BigMac ATTACK!");
 *    linux-2.6.19/drivers/net/sunhme.c
 */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: Digitale Signatur von OpenPGP
URL: <http://lists.qt-project.org/pipermail/interest/attachments/20190404/99e29c96/attachment.sig>