[Hpr] Slow downloads again
jknapp85 at gmail.com
Wed Sep 6 07:14:03 PDT 2017
They mean that some times it didn't identify as yandex, sometimes it would
fake as a browser. I saw the same thing in the logs after I started trying
to block the bot by user agent.
On Sep 6, 2017 7:09 AM, "Claes Wallin (韋嘉誠)" <
hackerpublicradio at clacke.user.lysator.liu.se> wrote:
> On Sep 6, 2017 4:09 PM, "Ken Fallon" <ken at fallon.ie> wrote:
> That thread is so funny. People really don't bother to read the full tread
> before they post, do they?
> - Using robots.txt didn't help, so here's how I detect and redirect that
> - Dude use robots.txt
> - IP block!
> - I redirected instead.
> - Dude you really should be using robots txt.
> - Here's how I IP block!
> Some people seem to believe that text file has magical powers.
> I didn't get this part:
> > One more thing: it identifies as client, not as bot !! Google and Yahoo
> identified as bots !
> What are they talking about? Is there anything aside from the user agent
> that indicates what the client is?
> Hpr mailing list
> Hpr at hackerpublicradio.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Hpr