I'm a system admin in a small company. We have a few WordPress websites (mostly static pages), and are hosted on AWS on Linux servers with EFS as file share and RDS MYSQL as database. Currently we do not have a control panel, and all the file and database updates are done by system admins for the website developers. Copying sites from stage to prod, prod to stage etc are also manual takss by copy and restore files and databases.
I would like to have a control panel like software to be installed in front (like the wp engine ones) so developers can manage the websites themselves.
Is there anything out there for this? Or do i have to go for external hosting and service providers like WP Engine.

Get hosting with control panel? Or install mainwp on a hosting and manage all your website from there
You can do it with MainWp plugin create another website a private one just install WordPress and mainwp the on the other site just install the mainwp child plugin from there you will have all site conected in one place and manage updates and so on.
One of options: https://www.cloudpanel.io/ and how to use it with AWS: https://www.cloudpanel.io/docs/v2/getting-started/amazon-web-services/installation/ami/
And for managing more WP sites: https://managewp.com/
I use RunCloud.io to do this. However the sites need to be set up RC originally. For WP updates, I use ManageWP.