Thanks for the report. Am I right that if you click the link you’ll see an error message and not be able to switch into the original account?
If so, it sounds like there’s a page caching mechanism in place on your site which is caching the page while a user is switched off. The cached page is then subsequently shown to other users with the “Switch back” link in place.
This is either a bug or misconfiguration in the cache plugin or cache mechanism, because the page should not get cached while an authentication cookie is present (which it is while you’re switched off).
If you’re using a page caching plugin your best bet is to report this to its author. If you’re using an edge cache service such as Cloudfront, Cloudflare, etc, then you’ll need to look at configuring it correctly to not cache pages when a WordPress authentication cookie is present.