I'm currently working for a websites client development team and need guidance on where to go for anti-cheat with unity.
We have scripting, where a user can code in their own features via a editor we have made in lua, with a scripting api that allows them to interact with parts of the game via said scripting, others can exploit this with tools and execute scripts on games that didnt have said scripts, via "hacking". I'm trying to stop script execution exploit(eg scripts being executed that weren't put there by the author) and general hacks such as speedhacks, assembly injection, etc. We have tried the anti-cheat toolkit but it hasn't worked well for us. In terms of execution we have the lua5.2 library in a dll, of course they can hook this library and execute anything via luas execution functions built in, statistcally linking the library would help defend againist this but that is not a option that is easily done in unity, so anyone could just hook it and execute any script/lua they want. we have server-side scripts(executed on server) and client/local-side scripts(executed on client/locally), we've done our best to secure these but we're looking into more ways to help protect us, such as anti-cheat.
We have plans to possible invest into paid anti-cheats in the future such as battleeye, easyanticheat, etc.
we currently don't make a lot of capital and i have no experience making any type of anti-cheat nor anti cheat in unity, where do i start and what do i do?