[Web] Restrictive robots.txt file

David Boddie david at boddie.org.uk
Wed Oct 12 00:01:21 CEST 2016


In an attempt to recover as much of the Qt Quarterly content as possible, I'm
accessing the pages linked to from

https://doc.qt.io/archives/qq/index.html

Since some of those pages were hosted on qt-project.org I now have to try and
find their contents via archive.org and this is where I fall foul of the
robots.txt file for that site:

https://web.archive.org/web/20130702155050/http://qt-
project.org/quarterly/view/using_cmake_to_build_qt_projects

Please could someone could tweak the file to make it possible for us to
access those pages again?

Alternatively, or even additionally, could I get access to the repository
that held the Qt Quarterly documents and examples? That would be even better
than having to scrape old archives of the site.

Thanks,

David



More information about the Web mailing list