tl;dr: We are working to solve the infrastructure challenges of distributing plugins, themes and core for WordPress users. Matt has highlighted a vulnerability, which we should all care about, regardless of who we think is right. But the effort needs help.
Hello. This may be my first post to r/WordPress but it's not my first engagement with the WordPress or PHP communities. In fact, I've been a PHP and WordPress developer for more than two decades, and I owe my livelihood and my success to WordPress and PHP. I'm thankful to Matt for his work on WordPress, and I'm also thankful for the hard work of core PHP developers who support WordPress. This post isn't about battling on who is wrong or who is right – it's about the future of WordPress.
The Problem
This week, Matt exposed a huge challenge for the WordPress community – centralized distribution of assets.
Whether you think Matt was right or wrong to cut off WP Engine, is not the point.
What *is* the point is this: WordPress.org serves as *the single source of truth* for automatically updating WordPress.
This is a problem on many fronts. It's tremendously expensive for WordPress.org. It's resource-intensive. It limits the options and choices for WordPress users. And it creates a serious community vulnerability.
The vulnerability should be apparent: if WordPress.org goes down, for any reason, millions of sites stop updating. A coordinated attack (zero-day implementation coupled with a DDoS attack that prevents updates from going out from the zero-day) could be a disaster the world over. And, if the Foundation ever decided to get out of the update business, or ran into financial difficulty, or Matt decides to retire to Aruba and quit WordPress entirely – whatever the case may be – there’s no Plan B.
The Solution
PHP has used Composer for years to distribute PHP code. It has several advantages: it’s designed to allow distribution of code from repositories, and requires no centralized server to store the code (Packagist.org does serve as a clearinghouse for FINDING packages, but it can be replaced with a line of code). And it relies on third-party delivery mechanisms for delivering the code – GitHub, GitLab, whatever – rather than being responsible for the bandwidth itself.
We need to decentralize the model in much the same way – not to punish Matt or hurt the Foundation, but to strengthen the community and HELP the Foundation.
We need a backup option, and we need opportunity to distribute plugins in new ways that match modern infrastructure and available tooling. This is a future goal – right now the idea is to provide a backup to WordPress.org so that the community can defend itself against attack, and we avoid a single-point-of-failure.
How It Works
I’ve been working on a solution to solve these problems.
First, I developed a plugin to rewrite the download URLs from the hard-coded WordPress API to a new endpoint. This is working.
Next, I started working on an API that can respond to requests for updates in the format that WordPress expects, and present options for downloading plugins from a CDN. This effort is ongoing and could use help (more on that later).
The way it works is simple: requests for data from the API are sent to the API. If the API implements a custom solution or has the data available, it provides it. If it doesn’t, it proxies the request to WordPress.org to allow for a complete API without having to implement the full API. I plan to implement the most basic API endpoints first – core, plugin and theme updates – and focus on things like the browsing the theme directory second. This way, if WordPress.org ever went down, we could still distribute critical updates (even if you couldn’t install new plugins).
For the community, not for profit
Distributing assets on behalf of WordPress users is expensive. But this isn’t intended to be a profit-generating enterprise – it’s intended to protect the community and take some of the load off the Foundation.
To that end, the intent is to provide access completely free to individual users.
If you’re a website hosting provider who has more than 1,000 sites under your management, we would ask that you negotiate a license for access, as a way of paying for the infrastructure and bandwidth. Individual users can still gain access on their own, without the license.
Any funds collected would be distributed equitably, first to pay for the bandwidth and infrastructure, and next split 50-50 between donation to the Foundation and paying developers to work on the infrastructure and code. The books of the project will be made public. Any remaining profit would be paid to a charity decided between the Foundation and us.
Current Status
Complete:
* A plugin to rewrite api.wordpress.org and download.wordpress.org to another set of custom endpoints.
* Negotiations with a CDN provider for bandwidth at an affordable rate.
In Progress:
* Development of a plugin API that can respond to requests.
Not Started:
* A website for issuing API keys
* Collecting a copy of all plugins and themes in some kind of storage
How you can help
This is a project that is open for the community to participate in. Though the code of the API is currently closed source, the plugin is open source and I’m happy to have others work on the API that have experience and desire.
Future goals
In the future, this API could function as a secondary source of plugin content, potentially even a marketplace or the like. It would also allow for us to improve distribution opportunities, similar to how Composer distributes code, rather than hosting all code centrally. There are security, legal and financial risks associated with these endeavors; they need to be thought through. But the future is bright.
Why should I trust you?
You shouldn’t – at least not yet. After all, this is my first post to Reddit, and you don’t know me from Adam. So watch what I do.
Asking people to give access to their updates is like giving up the keys to the kingdom. A compromised update can result in hacks far greater than an unpatched version of WordPress or a vulnerable plugin. So it’s important that people trust that we’re honest, and that we are working in the best interests of the community.
To that end, you can expect that we will build in public and be transparent. We will explain our thinking clearly, and take as long as necessary to clarify it. We will be transparent about security and reliability. And we will work to earn your trust.
Closing Thoughts
I want to reiterate the fact that this is not about who is right and who is wrong. It’s about what is good for the community, and how we can help the community.
WordPress.org is a single-point-of-failure for critical updates and patches. Asking millions of website owners to update their sites manually during an emergency would wreak havoc.
I’m hopeful that Matt and the WordPress Foundation will support this effort as focused on the community and intended for the good of WordPress, not to harm Matt or the Foundation in any way. It’s neither fair nor reasonable to expect that Foundation to bear the brunt of the bandwidth and storage costs. And it’s not reasonable to expect them to be the sole sources of innovation in the plugin ecosystem. We must, as a community, help.
It is also my hope that the WordPress Foundation will support the mirroring of their assets and proxying of certain requests to them as we build up the API to eventually replace their endpoints entirely. This doesn’t add extra bandwidth to WordPress.org, but does require cooperation from them to not block OUR requests.
My DMs are open. Feel free to reach out if you’d like to contribute. Feel free to share your thoughts.
You’re proposing to migrate from WordPress.org to Composer which is another host? I don’t fully understand how this makes it decentralized?
And if it’s just about changing the format, why is that necessary for others to host another plugin library?
Nice try Silver Lake
Take a look at satis / satispress.
Sounds good. I’d like to know more, and maybe put my shoulder to the wheel and help out. But you didn’t include a call to action other than “DMs are open” in your post. So, I’ll wait for that.
The vulnerability is real, and unfortunately a likely next step is that an entity like WPEngine attempts to do a hard fork of the community or “help” by introducing their own proprietary infrastructure.
I think you should get in touch with an organization like Software in the Public Interest or the Apache Software Foundation. These are organizations that are genuinely committed to the letter and spirit of open source, they are not as cowboy as Matt about their governance, and they have funding, expertise and resources. With the number of websites running it, WordPress could really be considered essential to the architecture of the web these days. I’m not sure they would be ready to get involved at this stage but it’s very much worth seeing if they want to talk. If one of these organizations was to support a mirror of the WPorg repositories it could be very powerful.
P2P plugin and core delivery API
Hmmm.
You should ask WP Engine to build this and open source it.
What’s stopping us from putting up multiple mirrors of the repositories?
The problem is that there are so many single points in the community.
You don’t have to host your plugins/themes at W.org. I have 3 of my own sites, all three use themeforest domains.
When it comes to the community, it is usually a few people “taking control”.
Using my city: Toronto…………..The last WordCamp in Toronto was in 2018. The organizers got “tired” and no 2019 WCTO occurs, they could could of handed the reins to others in the community. Obviously 2020 to let’s say 2022 or 2023 had not in-person WCTO, online WCTO could of occurred.
All the WPTO monthly meetup group does is “let’s fix your site” online meetings. We had great community monthly meetups before the pandemic.
A small group decides who comes to speak at the WordCamps, most of the times it’s someone who is buddy buddy with one of the organizers and not someone from the community. If you are having a WCTO, WCMIA, WCLA……..the speakers should be from Toronto, Miami, Los Angeles areas respectively. But a lot is usually that certain crowd that goes from WordCamp to WordCamp. I’d understand WCUS/Canada/EU/Asia but not the WCcities ones.
We need a true community and just a small loud minority.
An example: PodCamp Toronto, speakers apply to speak, then people who will attend PCTO…favourite/bookmark the sessions they want to attend….they top favourite/bookmarked get on the schedule. WordCamps should be like that. Not just a small single point of organizers decide. Community decides.
You can’t decentralize plugin/theme/core updates without spending millions on traffic. I did my fair share of research and turns out that the traffic will eat you alive.