[Hpr] We're [in]famous

Ken Fallon ken at fallon.ie
Wed Jul 10 23:57:15 PDT 2013


On 2013-07-11 07:51, ken.fallon at gmail.com wrote:
> ______________________________________________
> Hpr mailing list
> Hpr at hackerpublicradio.org
> http://hackerpublicradio.org/mailman/listinfo/hpr_hackerpublicradio.org
> We may have found the problem :)
>
>
>     Fetch as Google
>
> This is how Googlebot fetched the page.
>
> *URL:* http://hackerpublicradio.org/
>
> *Date:* Wednesday, July 10, 2013 at 10:48:01 PM PDT
>
> *Googlebot Type:* Web
>
> *Download Time (in milliseconds):* 114
>
> |HTTP/1.1 403 Forbidden
> Date: Thu, 11 Jul 2013 05:48:01 GMT
> Server: Apache/2.0.63 (Unix) mod_ssl/2.0.63 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4
> Content-Length: 465
> Keep-Alive: timeout=15, max=100
> Connection: Keep-Alive
> Content-Type: text/html; charset=iso-8859-1
>
> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>403 Forbidden</title>
> </head><body>
> <h1>Forbidden</h1>
> <p>You don't have permission to access /
> on this server.</p>
> <p>Additionally, a 403 Forbidden
> error was encountered while trying to use an ErrorDocument to handle the request.</p>
> <hr>
> <address>Apache/2.0.63 (Unix) mod_ssl/2.0.63 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 Server at hackerpublicradio.org Port 80</address>
> </body></html>|
>
>
>
> _______________________________________________
> Hpr mailing list
> Hpr at hackerpublicradio.org
> http://hackerpublicradio.org/mailman/listinfo/hpr_hackerpublicradio.org

Using the " Fetch as Google" the url showed up as shown above. Yet
accessing the site using wget and Googles user agent showed the site up
fine.
wget --user-agent="Googlebot/2.1 (+http://www.google.com/bot.html)"
http://hackerpublicradio.org/ -O -

So what's happening ?

Well, some time ago (long enough that I had forgotten about it) I
implemented a ban users doing suspicious stuff script. Said script
blocks ip addresses that go to suspicious url's on the site, have too
many 404 errors etc. Exactly the sort of things malicious hackers do,
'course it's also exactly the sort of thing GoogleBot does as well. (OK
you can stop laughing now)

Anyway it turns out not to be a global conspiracy by Google against HPR,
so order is now restored to the universe. But keep those incoming links
coming.

Sorry for the stupidity.

Ken.






More information about the Hpr mailing list