[Web] Restrictive robots.txt file
David Boddie
david at boddie.org.uk
Wed Oct 12 00:01:21 CEST 2016
In an attempt to recover as much of the Qt Quarterly content as possible, I'm
accessing the pages linked to from
https://doc.qt.io/archives/qq/index.html
Since some of those pages were hosted on qt-project.org I now have to try and
find their contents via archive.org and this is where I fall foul of the
robots.txt file for that site:
https://web.archive.org/web/20130702155050/http://qt-
project.org/quarterly/view/using_cmake_to_build_qt_projects
Please could someone could tweak the file to make it possible for us to
access those pages again?
Alternatively, or even additionally, could I get access to the repository
that held the Qt Quarterly documents and examples? That would be even better
than having to scrape old archives of the site.
Thanks,
David
More information about the Web
mailing list