[ad_1]
*** PUBLIC SERVICE ANNOUNCEMENT ***
I have thoroughly investigated the bots, Bytedance and Bytespider. As a result, I highly recommend blocking them. Main reasons:
- They do not respect robots.txt rules.
- They are using well-known hosting services to bypass normal blocking channels or methods. For example, they are using AmazonAWS services to crawl websites and use the results for potentially malicious purposes. Details: https://prnt.sc/7Zst7eUZaxZ3
- Their crawling rates are extremely high.
- Blocking Bytedance and Bytespider via AS138699 and AS396986 does not help much.
In short, I recommend blocking the User Agents, Bytedance and Bytespider, for best results.
Hope this helps a bit.
Cheers 🙂