Finally managed to delete this plugin by going to my host’s cpanel and deleting it from my file manager (as trying to delete it from my WordPress install gave a critical error message).
I consider any rollback plugin as one that you can use, then delete it when you’ve done the rollback.
This plugin though does not allow that and needs to be installed and live to go back to an older version.
However, trying to delete it is where the fun starts:
On deactivating PlugVersions, ALL of the versions of plugins, and themes that have been rolled back now appear as NEW plugins that need to be removed and deleted.
These plugins also appear in WordFence – but when you see which plugins need updating, nothing is there – they just appear in your admin bar as plugins that need updating.
I needed to delete 21 versions of plugins and themes and as already mentioned, delete Plugversions from my hosting cpanel.
Probably the worst plugin that I’ve ever installed.