[ad_1]
Hi, I’ve been trying to make my site findable online, but, Google cant index or crawl my page because it is “blocked by robots.txt” (according to Google search console). I have no clue whats blocking it and no robot.txt tester can find any problems.
Anyone know what I’m doing wrong?
User-agent: *
Allow: /wp-admin/admin-ajax.php
Disallow: /wp-admin/
Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
Sitemap:
Sitemap:
Check your header, for this..
<meta name=’robots’ content=’noindex,follow’ />
Also check settings > reading for discourage search engines check box.
Check your seo plugins. They dynamically generate the robots.txt file.
open this url in a browser
https://**put your domain name here**/robots.txt and see what you get.
If you block wp-content you are also blocking plugins and themes. This can keep content from rendering properly which is a problem for indexing.