I know. They are still damned pesky blighters.It's companies trying to harvest data to load into their AI chatbots.
I haven't had issues with Tencent yet, but AliBaba bots seem to have fallen in love with me. They do not respect robots.txt directives either.
Someone pointed me at this yesterday: https://arstechnica.com/tech-policy/202 ... obots-txt/
Repo: https://zadzmo.org/code/nepenthes/
I have to say this (or something similar) is sounding very appealing.
Statistics: Posted by Gumboots — Mon Apr 28, 2025 9:19 pm