Our site is a high-traffic blog that has video embeds on 1500 posts using a custom AMP-stack CMS from 2008. We are converting it to WordPress and I used Elementor to build it. Along the way, I discovered that Elementor seems to have severe performance problems.
​
I got a lot of great suggestions for improving TTFB, load speed, etc. And I have posted other questions about this same website redesign project previously. However, this post is asking a very specific question I’ve never though about until now.
​
The temp staging site (technically a prod site, but used as if it was staging) has high-speed hosting on WPEngine and there is caching and Litespeed and other optimizations that WPE is doing by default. Once I’ve visited a page, it responds super-fast (miliseconds?). But upon first click, it takes 4-7 seconds to load.
​
That’s an unacceptable TTFB for user experience.
​
My question is this — It seems like if a web-spider crawls my site it will automatically trigger WPE’s page caching mechanism. … Is that right?
​
I’m trying to find a sort “page-preloader” or “cache warmup”. I was investigating plugin options, but most that reference “preload” are about sliders and carousels. I’m trying to preload the whole page … every page. Is that possible?
​
URL: streamsair dot org
I think your question is actually, “can I rely on a search engine crawler to preload my cache”? No. They don’t visit your site often enough to be reliable for this. Plus, if you use a decent SEO plugin, the crawlers will use the site map published by your SEO plugin, and hit your pages even less often.
You could preload your cache yourself. A script or bat file with a lot of wget commands for your most important content URLs will do the trick. Or you could use wget to ” mirror” your site once in a while. You’ll need to read about that, it’s beyond the scope of a reddit comment.
Plus, when you’re live this problem will be less serious, especially if you set a long duration on your cache. That is, once any user takes the 4-7 second hit for a page, nobody else will need to.