I have been working on a directory website for a long time. Countless hours trying to figure out things with a bunch of custom code. It's 100% bootstrapped. Almost ready for launch, but there are a few major issues I need to fix before it's ready. Almost all of the pages are dynamically generated.
Main issues
- Cannot update ANY Posts/Pages/CPTs without seeing a 524 error code. I update as much as I can using php and MySQL queries because WP All Import is very very slow. Still trying to figure out how to bulk update taxonomies.
- Users cannot submit any new listings/reviews, etc. without seeing the same error code. Everything is manually verified, so this part if VERY important.
- Loading Speed when logged is so slowwww. OMG I cannot figure this out.
Solutions so far
- Perfmatters really helped me get loading speed down to about .5 seconds for desktop. Mobile is a PITA and is currently at 2.5-3 seconds. I still need to go through the Script Manager to figure out that stuff.
- Setup custom unique tables when needed that don't require a ridiculous amount of joins. Works perfectly for the CPTs I'm not managing the usual WordPress way. Why doesn't WordPress allow one table per CPT?
- Created a master table that has every single Post/CPT/Term. Takes about 5 hours to populate everything with a very long MySQL procedure. I try to use this table when possible.
Current setup
- 63gb 16 CPU server – 250 SSD (Nocix-use about half the drive).
- Cloudpanel- Varnish Enabled with Nginx
- Cloudflare free plan
- Matomo self hosted
- Various external tools set up on Vultr VPS.
- Voxel Directory Theme
Current Active Plugins
- aaa-option-optimizer
- clp-varnish-cache
- elementor
- elementskit-lite
- extensions-leaflet-map
- index-wp-mysql-for-speed
- leaflet-map
- master-addons
- mycred
- perfmatters
- permalink-manager
- premium-addons-for-elementor
- premium-addons-pro
- query-monitor
- redis-cache
- seraphinite-accelerator-ext (currently testing-seems very promising- might take a few weeks to cache everything)
- google-site-kit
- updraftplus
- wp-security-audit-log
- wp-crontrol
- wp-review-slider-pro
- google-sitemap-generator
Inactive/As needed plugins
- wp-asset-clean-up
- category-import-reloaded
- imagify (currently testing)
- plugins-garbage-collector
- seo-by-rank-math (only use for schema and monitor 404s)
- seo-by-rank-math-pro
- safe-svg
- shortpixel-image-optimiser (currently testing)
- wp-cloudflare-page-cache (doesn't seem to help much)
- user-role-editor
- wp-optimize (only option that won't crash unlike wp sweep, etc.)
- wp-all-export-pro
- wpai-voxel-reviews-addon
- wpai-voxel-addon
- wp-all-import-pro
- wpdatatables
- wp-githuber-md
- wp-rss-aggregator
- wp rocket (could never get this to work after a month going back and forth with support about bugs)
I'm really not sure about how to handle the issues. Cloudflare Enterprise would help, but from what I've read that's very expensive. I've been trying to find tutorials to help, but coming up short. My server is nowhere near capacity.
I know my way around servers and MySQL, as I'm completely self taught pre-ChatGPT days. But I'm not an expert and continue to spend many hours on Stack Overflow researching.
I would LOVE to find a tool like Airtable that can handle 2M-4M records. To help with cache problems I came across this article, which leads me to believe I need to consider Rocket.net. A bit expensive, and I'm not even sure if it'll help.
ChatGPT told me I should consider looking into asynchronous processing or queue jobs to handle the tasks in the background while allowing the server and Cloudflare to finish the initial request. Started to look into that until it gave me a headache lol.
One idea I came up with was to use an external form for review submissions and then I could parse that information and dump into the database. But that doesn't help with the issues of manually publishing articles, posts, etc.
I'm open to any and all suggestions.