[ad_1]
Hi,
we’ve just found a nasty problem with Google.The added parameters ?nab=0 etc. can easily exhaust the Google crawl budget, since it tiples the amount of sites that can be crawled. So, I added a rule to the robots.txt to block urls containing this parameter like so:
Disallow /*?nab=
This seems to me a perfectly fine rule. But suddenly, new “errors” appeared in the Search Console that some page were in the index but blocked by the robots.txt like:
domain.de/page/
I checked these pages in the Google URL tester, and they are, of course, not blocked. This is a typical bad Google joke, but we have to deal with it.
I can remove the parameter from the URL. Is this safe without compromising the data? How much will it affect load time?