[Hpr] Slow downloads again

Claes Wallin (韋嘉誠) hackerpublicradio at clacke.user.lysator.liu.se
Wed Sep 6 07:08:24 PDT 2017

On Sep 6, 2017 4:09 PM, "Ken Fallon" <ken at fallon.ie> wrote:

That thread is so funny. People really don't bother to read the full tread
before they post, do they?

 - Using robots.txt didn't help, so here's how I detect and redirect that
 - Dude use robots.txt
 - IP block!
 - I redirected instead.
 - Dude you really should be using robots txt.
 - Here's how I IP block!


Some people seem to believe that text file has magical powers.

I didn't get this part:

> One more thing: it identifies as client, not as bot !! Google and Yahoo
identified as bots !

What are they talking about? Is there anything aside from the user agent
that indicates what the client is?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://hackerpublicradio.org/pipermail/hpr_hackerpublicradio.org/attachments/20170906/cfc8f8bc/attachment-0001.html>

More information about the Hpr mailing list