facebook debugger not scraping my website-might be an issue with robots.txt

[ad_1]

When I try to scrape any blog links to get an open graph image to change, I get the following error.

"This response code could be due to a robots.txt block. Please allowlist facebookexternalhit on your sites robots.txt config to utilize Facebook scraping"

I edited the robots.txt file from the squirrly seo plugin I'm using. I added something to allow facebookexternalhit on it. The robots.txt file showed what I added yesterday. When I check it again today, it's reverted to the one I had on a few days ago.

Has anyone had a similiar issue?

https://escapingtherabbithole.com/robots.txt

https://escapingtherabbithole.com/how-to-transform-conspiracy-beliefs-with-little-known-language-hacks/

https://preview.redd.it/5fy3ad2b69nd1.png?width=1827&format=png&auto=webp&s=24bbdda888e706f62285eb9dfa61122b985a179c

[ad_2]

 

This site will teach you how to build a WordPress website for beginners. We will cover everything from installing WordPress to adding pages, posts, and images to your site. You will learn how to customize your site with themes and plugins, as well as how to market your site online.

Buy WordPress Transfer