I saw I got notification about 21 updates available in my dashboard, but when I clicked on it, it shows that update-core.php doesn't exist. It's not just this website btw, the other 2 website I manage also have this same problem. When I tried to add a new plugin, it also shows that plugin-install.php doesn't exist. I checked on cPanel, and indeed there are none of that file names.
So what is this? Is this an error, bug, or something? Or someone might accidentally delete those file? Or was it never there in the first place? And more importantly, how do I fix this?

Fix it by downloading WordPress from WordPress.org and installing that over the top of your current install. Unzip and extract the files.
Though the bigger issue here is why they were deleted in the first place. My guess would be malware.