You cannot prevent it (in situations which are equivalent to running arbitrary native code supplied by others).
You can:
- curate the mods
- screen submissions with a scanner
- restrict your API as much as possible
- select a language that is safety oriented
- sandbox foreign code
All of these increase safety to varying degrees. But none of them are a guarantee.