[ad_1]
I recently installed YOAST on a brand-new website, and not a single page was indexed within a week. Trying to figure out what was happening, I discovered that my robots.txt had been rewritten by YOAST. For what reason? Why? Now, it’s not even valid.
https://preview.redd.it/vuuz9k5wdxsc1.png?width=1952&format=png&auto=webp&s=53192a4c5a0364ddcf006ceda44331d1fb7bb60b
https://preview.redd.it/ljhf9o5wdxsc1.png?width=1138&format=png&auto=webp&s=55b56ee41457123dc75fe9af8d891ae56d567268
[ad_2]
Original WordPress robots.txt looks like this btw:
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://website.com/wp-sitemap.xml
*and it’s not the first time I have this problem!
Yoast is just the worst plugin honestly use SEOPress or Rank math.
I think you may be misunderstanding what this robots.txt means. I notice that happening a lot in this subreddit. I suggest learning about robots.txt
Opinions on Yoast aside your robots file (their default one) is 100% valid and this is not Yoast’s fault.
Top post is correct, it’s not blocking your website, it’s blocking bots.
That example looks 100% correct and is doing exactly what it’s supposed to do. Your understanding of robots.txt isn’t correct.
Are you sure you site isn’t set to not be indexed in Settings? Once you confirm you are disallowing search engine indexing:
Ping your sitemap. [Read This](https://trevorfox.com/2018/09/ping-sitemaps-search-engines/)