In my current project, Lua scripts are called by the C++ functions on the server side. After that, the scripts again call the C++ functions still in that solution. Why should we do such things and not call the C++ function directly? What are the situations in which scripts are needed?
Scripts are usually compiled at run time, while the host language will be compiled at compile time. This means that we don't need to recompile if the script changes. Recompiling a full game can take minutes to hours, which implies a big productivity hit.
Usually, the critical code or backend code will not be scripted. This code should run fast and often memory management is crucial.
In games, game logic and configuration are typically contained in script files. These scripts can easily be updated by non-programmers (like the designer) to tweak the gameplay. Script languages are easy and act in a forgiving manner for that purpose.
Often, a script language is also used to do scripting at real time. This comes in handy for tweaking some gameplay elements or even for debugging. Many games provide a console for this (mostly in-house) purpose.
It is very well possible that you create a game using an existing game engine, just by scripting. The game engine layer is thus fully decouple from the game logic layer. Modern engines can usually be used to create FPS or RTS games easily like this, but it is not possible for any genre. An MMO would probably require another type of engine.
So the bottom line is decoupling. The benefits listed above often outweight the extra work to create or integrate a scripting language.
We use "scripting" languages (which really means, very high-level, garbage collected, and usually some form of looser typing and dynamic compilation) because they're generally easier languages for anyone - programmers included - to write code in. There's less stupid stuff like remembering to free after you malloc or make sure your code is exception-safe, or remembering to make all your destructors virtual. If our computers were infinitely fast, we would be using "scripting" languages for everything.
Scripting languages for game logic is a very good example of the software architecture pattern Alternate Hard and Soft Layers. There's a good discussion on that site (and others I'm sure) on the benefits of doing so.
- Each new lua class is two lines. Each new C++ class is pain.
- No whining about types when all you want is shuffle values around.
- Garbage collection.
- Script code is nicely isolated in virtual machines, away from all those nasty wandering segfaults and array overflows.
Script changes are easy to deploy. For example, you can keep scripts in a database which means that instead of a full binary redeploy and possible service restart, you just issue a single SQL UPDATE statement, possibly followed by a signal to your running service to reload the script.
Also script languages are often simple to understand and easy to program in so you don't need the 'hardcore' devs (the ones dealing with Memory Management/Pointers and cpu-level optimizations) for the majority of code (if a large RPG, the scripts for AI, Spells, Item Effects and the world itself are often bigger than the engine code). Script languages allow a lot more focusing on the 'what' rather than the 'how' due to Garbage Collection (in case of Lua) and a higher level of abstraction.
One scenario where the scrips are useful is when we want our engine to be extensible by plugins/addons. These extensions are created in many cases by non-professional people (like advanced gamers and enthusiasts). Scripts are safer and easier for this purpose. By using scripts, they don't need to use compilers, and don't need to be aware of pointers...
A great example of this is the game World of Warcraft. It has thousands of addons created by the community. These addons are written using LUA + XML.