The path represented by
Application.persistentDataPath is a path on the user's computer. You can't really prevent the user from seeing and manipulating files on his or her own machine.
Application.persistentDataPath writes to locations that are hidden by default on macOS and Windows (I'm not sure about Android). In theory you could set the "hidden" attribute on the files you write, as well (if you access to an API to do so). But that doesn't prevent the user from toggling "show hidden files" or otherwise deleting them.
If you really want files to be written to a secure location where the user cannot tamper with them, you will need to write them on a machine you control (that is, a server), which is a whole new can of worms.
I suspect you're best off not worrying too much about this. Write a bit of code to handle the cases where the files don't exist (so for example your game doesn't crash), and that should be sufficient.