[Qtwebengine] Proof of concept - changing the order of http request headers

CJ Ess zxcvbn4038 at gmail.com
Mon May 11 22:49:43 CEST 2015


Hello,

I originally sent this to the downstream PhantomJS project, but they
thought it was more of a QT issue so I should send it here. I would like to
change the order of the HTTP request headers - technically the order
doesn't matter, however because the order which PhantomJS sends them (which
is the way that QTWeb sends them) is fairly unique, it's one of the things
that Web Application Firewalls (layer 7 firewalls) key off of to keep out
screen scrapers.

The attached diff demonstrates that the order of the headers can be
controlled reliably at the point they are serialized to the outgoing
buffer. The next step would be to make the order configurable and expose
that in the API someplace however I think thats beyond what I can
contribute. I'm hoping someone else can run with it from here.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/qtwebengine/attachments/20150511/dc6fd215/attachment.html>
-------------- next part --------------
--- a/src/qt/qtbase/src/network/access/qhttpnetworkrequest.cpp
+++ b/src/qt/qtbase/src/network/access/qhttpnetworkrequest.cpp
@@ -137,6 +137,7 @@ QByteArray QHttpNetworkRequestPrivate::header(const QHttpNetworkRequest &request
 {
     QList<QPair<QByteArray, QByteArray> > fields = request.header();
     QByteArray ba;
+    QByteArray value;
     ba.reserve(40 + fields.length()*25); // very rough lower bound estimation

     ba += request.methodName();
@@ -149,14 +150,34 @@ QByteArray QHttpNetworkRequestPrivate::header(const QHttpNetworkRequest &request
     ba += QByteArray::number(request.minorVersion());
     ba += "\r\n";

-    QList<QPair<QByteArray, QByteArray> >::const_iterator it = fields.constBegin();
-    QList<QPair<QByteArray, QByteArray> >::const_iterator endIt = fields.constEnd();
-    for (; it != endIt; ++it) {
+    QList<QByteArray> hdrord;
+    hdrord << "Host" << "Connection" << "Accept" << "User-Agent" << "Accept-Encoding" << "Accept-Language";
+    // Headers with preferred order, allowing duplicates
+    foreach(QByteArray item, hdrord) {
+      QList<QPair<QByteArray, QByteArray> >::const_iterator it = fields.constBegin();
+      QList<QPair<QByteArray, QByteArray> >::const_iterator endIt = fields.constEnd();
+      for (; it != endIt; ++it) {
+        if (it->first != item)
+          continue;
         ba += it->first;
         ba += ": ";
         ba += it->second;
         ba += "\r\n";
+      }
+    }
+    // Custom Headers or w/o preferred order, allowing duplicates
+    QList<QPair<QByteArray, QByteArray> >::const_iterator it = fields.constBegin();
+    QList<QPair<QByteArray, QByteArray> >::const_iterator endIt = fields.constEnd();
+    for (; it != endIt; ++it) {
+      if (hdrord.indexOf(it->first) > -1)
+        continue;
+      ba += it->first;
+      ba += ": ";
+      ba += it->second;
+      ba += "\r\n";
     }
+    hdrord.clear();
+
     if (request.d->operation == QHttpNetworkRequest::Post) {
         // add content type, if not set in the request
         if (request.headerField("content-type").isEmpty()) {


More information about the QtWebEngine mailing list