How does Lua work as a scripting language in games?

Question

I'm a little hazy on what exactly Lua is and how a game that is programmed in C++ would use it. I'm asking primarily about how it is compiled and run.

For instance when you use a program written in C++ which uses Lua scripts: does the code in Lua just call to functions in the main program written in C++ and act as a uncompiled class waiting to be compiled and added to the memory heap of the C++ program?

Or does it act like a bash script in Linux where it just executes programs completely separate from the main program?

Answer 1

Scripting is a programming abstraction in which you (conceptually) have a program (the script) running inside another program (the host). In most cases, the language in which you write the script is different from the language in which the host is written, but any program-inside-a-program abstraction could be considered scripting.

Conceptually, the common steps to enable scripting are the following (I will be using pseudo-c for the host and pseudo-lua for the script. These are not exact steps, but more like the general flow in which you enable scripting)

1. Create a virtual machine in the host program:

    VM m_vm = createVM();
   

2. Create a function and expose it to the VM:

    void scriptPrintMessage(VM vm)
    {
        const char* message = getParameter(vm, 0); // first parameter
        printf(message);
    }
   
    //...
   
    createSymbol(m_vm, "print", scriptPrintMessage);
   

Notice that the name in which we exposed the function (print) does not have to match the internal name of the function itself (scriptPrintMessage)

3. Run some script code that uses the function:

    const char* scriptCode = "print(\"Hello world!\")"; // Could also be loaded from a file though
    doText(m_vm, scriptCode);
   

That's all there is to it. The program then flows in the following manner:

1. You call doText(). Control is then transferred to the virtual machine, which will execute the text inside scriptCode.

2. The script code finds a previously exported symbol print. It will then transfer control to the function scriptPrintMessage().

3. When scriptPrintMessage() finishes, control will transfer back to the virtual machine.

4. When all of the text in scriptCode has been executed, doText() will finish, and control will be transferred back to your program on the line after doText().

So in general, all you're doing is running a program inside another program. Theoretically speaking, there is nothing you can do with scripts that you can't do without them, but this abstraction allows you to do some interesting things really easily. Some of them are:

• Separation of concerns: it is a common pattern to write a game engine in C/C++ and then the actual game in a script language like lua. Done right, the game code can be developed completely independently from the engine itself.

• Flexibility: scripting languages are commonly interpreted, and as such, a change in a script will not necessarily require a rebuild of the entire project. Done right, you can even change a script and see the results without even a program restart!

• Stability and security: since the script is running inside a virtual machine, if done right, a buggy script won't crash the host program. This is specially important when you allow your users to write their own scripts to your game. Do remember that you can create as many independent virtual machines as you like! (I once created an MMO server in which each match ran on a separate lua virtual machine)

• Language features: when using scripting languages, based on your choice for host and script languages, you can make use of the best features each language has to offer. In particular, lua's coroutines are a very interesting feature that is very difficult to implement in C or C++

Scripts are not perfect though. There are some common disadvantages to using scripting:

• Debugging becomes very difficult: usually, the debuggers included in common IDEs are not designed to debug the code inside scripts. Because of this, console trace debugging is much more common than I'd like.

  Some scripting languages like lua have debugging facilities which can be taken advantage of in some IDEs like Eclipse. Doing this is very difficult, and I've honestly never seen script debugging working as well as native debugging.

  By the way, the extreme lack of interest the Unity developers have in this matter is my main criticism of their game engine, and the main reason I don't use it anymore, but I digress.

  Update from 2023: I now use a Lua debugger for Visual Studio Code called LuaPanda, which allows you to attach the debugger to a Lua VM by just adding one line of code to the script (no changes to the host program). It's a bit clunky to install, but it works really well once you're up and running with it.

• IDE integration: It is unlikely that your IDE will know which functions are being exported from your program, and as such, features such as IntelliSense and the like are unlikely to work with your scripts.

• Performance: Being commonly interpreted programs, and meant for an abstract virtual machine whose architecture may be different from the real hardware, scripts are commonly slower to execute than native code. Some VMs like luaJIT and V8 do a really good job though. This may be noticeable if you do some very heavy usage of scripts.

  Usually, context changes (host-to-script and script-to-host) are very expensive, so you may want to minimize them if you're having performance problems.

How you use your scripts is up to you. I've seen scripts used for things as simple as loading settings, to as complex as making entire games in the scripting language with a very thin game engine. I once even saw a very exotic game engine that mixed lua and JavaScript (via V8) scripting.

Scripting is just a tool. How you use it to make awesome games is completely up to you.

Answer 2

Generally, you bind or expose some native functions to Lua (often using a utility library to do so, although you can do it by hand). That allows Lua code to make calls into your native C++ code when your game executes that Lua code. In this sense, your assumption that the Lua code just calls into the native code is true (although Lua has its own standard library of functionality available; you do not need to call into your native code for everything).

The Lua code itself is interpreted by the Lua runtime, which is C code that you link as a library (usually) in your own program. You can read more about how Lua works at the Lua homepage. In particular, Lua is not "an uncompiled class" as you surmise, especially if you are thinking of a C++ class, because C++ is almost never dynamically compiled in practice. However, the Lua runtime and objects created by Lua scripts that your game runs do consume space in your game's pool of system memory.

Answer 3

Scripting languages like Lua can be used in several ways. As you said you can use Lua to call functions in the main program but you can also just have Lua functions called from the C++ side if you want. Generally you build up an interface to allow some flexibility with the scripting language of your choice so you can use the scripting language in a series of scenarios. The great thing about scripting languages like Lua is they are interpreted instead of compiled so you can modify the Lua scripts on the fly so you don't have to sit around waiting for your game to compile only for you to re-compile if what you've done doesn't suit your tastes.

Lua commands only get called when the C++ program wants to execute them. As such, the code will only be interpreted when it is called. I guess you can consider Lua as a bash script that executes separately from the main program.

Generally you want to use scripting Languages for things you may want to update or iterate further down the line. I've seen a lot of companies use it for the GUI so it provides a lot of customizability to the interface. Providing you know how they have made their Lua interface you can also modify the GUI yourself. But there are numerous other paths you can take to using Lua, such as AI logic, information about weapons, dialogue of characters, etc.

Answer 4

Since no one mentioned this, I will add it here for those interested. There is a whole book on the subject called Game Scripting Mastery. This is a fantastic text that was written quite a while ago, but it remains completely relevant today.

This book will not only show you how scripting languages fits into native code, it also teaches you how to implement your very own scripting language. While this will be overkill for 99% of users, there is no better way to understand something than actually implementing it (even in a very basic form).

If you ever want to write a game engine yourself (or you work with a render engine only), this text is invaluable for understanding how a scripting language can best be incorporated into your engine/game.

And if you ever want to create your own scripting language, this is the one of the best places to start (as far as I know).

Answer 5

Most scripting languages, including Lua, operate on a Virtual Machine (VM), which is basically a system to map a script instruction to a "real" CPU instruction or function call. The Lua VM normally runs in the same process as the main application. This is specially true for games that use it. The Lua API provides you with several functions that you call in the native application to load and compile script files. E.g. luaL_dofile() compiles the given script into Lua bytecode and then runs it. This bytecode will then be mapped by the VM running inside the API into native machine instructions and function calls.

The process of connecting a native language, such as C++, with a scripting language is called binding. In the Lua case, its API provides functions that help you expose native functions to the script code. So you can for example define a C++ function say_hello() and make this function callable from a Lua script. The Lua API also provides methods for creating variables and tables via C++ code that will be visible for the scripts when they are run. By combining these features you can expose entire C++ classes to Lua. The opposite is also possible, the Lua API allows the user to modify Lua variables and call Lua functions from the native C++ code.

Most if not all scripting languages provide APIs to facilite the binding of script code with native code. Most are also compiled into bytecode and run in a VM, but some may be interpreted line-by-line.

Hope this helps clarifying some of your questions.

Answer 6

Firstly, scripting languages are USUALLY not compiled. That is a large part of what generally defines them as scripting languages. They are often, instead, "interpreted." What this essentially means is that there is another language (one that is compiled, more often than not) that is reading in the text, in real time, and performing operations, line-by-line.

The difference between this and other languages is that scripting languages tend to be simpler (commonly referred to as "higher level"). However, they also tend to be a bit slower, as compilers tend to optimize a lot of the issues that come with the "human element" of coding, and the resulting binary tends to be smaller and faster to read, for the machine. Additionally, there is less overhead from another program needing to be run to read the code being run, with compiled programs.

Now, you might be thinking, "Well I get that it's a bit easier, but why on earth would someone give up all that performance for a little bit extra ease of use?"

You would not be alone in this assumption, however the level of ease you tend to get with scripting languages, depending on what you're doing with them, can be well worth the sacrifice in performance.

Basically: For instances where speed of development is more important than speed of the program being run- use a scripting language. There are PLENTY of situations like this in game development. Especially when dealing with trivial things like high-level event handling.

Edit: The reason lua tends to be rather popular in game development is because it is arguably one of the fastest (if not the fastest) publicly available scripting languages on earth. However, with this extra speed, it has sacrificed some of its convenience. That being said, it's still arguably more convenient than working with straight up C or C++.

Important Edit: Upon further research, I've found that there's a lot more controversy over the definition of a scripting language (see Ousterhout's dichotomy). The primary criticism of defining a language as a "scripting language" is that it is neither significant to the syntax nor the semantics of the language that it is interpreted or compiled.

While languages that are usually considered "scripting languages" are usually traditionally interpreted, rather than compiled, the long and short of their definition as "scripting languages" is really up to a combination of how people view them, and how their creators defined them.

Generally speaking, a language could be easily considered a scripting language (assuming you agree with Ousterhout's dichotomy) if it fulfills the following criteria (according to the article linked, above):

• They are typed dynamically
• They have little or no provision for complex data structures
• Programs in them (scripts) are interpreted

Additionally, it is often accepted that a language is a scripting language if it is designed to interact and function alongside another programming language (usually one that is not considered a scripting language).