[Web] [Development] Search engine indexing of qt-project.org

Jonas Gastal jgastal at profusion.mobi
Wed Jun 12 15:14:45 CEST 2013

On Tue, Jun 11, 2013 at 1:30 PM, Jonas Gastal <jgastal at profusion.mobi>wrote:

> Interestingly, doing wget I get the same as Giuseppe, using the
> browser(chromium) I get the same as Robin.
> On further analysis the reason I was getting the disallow everything
robots.txt in chromium was because of https everywhere.
I don't see much of a reason to block https  crawling, on the other hand I
doubt that's what's causing bad results for google searches.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/web/attachments/20130612/faa4a172/attachment.html>

More information about the Web mailing list