Let's say I have a game written in C++. But I want to add some modding or scripting functionality to it. How would one go about adding a scripting functionality to your game?
First of all, you should decide what part of your game is scripted. One option is to have a fully scripted game in the sense that while the time-critical backend operations are coded in C++, all the game logic is in the scripting language. Designers use the backend as a an API called from the high level scripting language. On the other extreme, you can have few specific places where scripts are used, such as the user interface or scripted sequences, with the majority of the game code is still in C++. There are advantages to each approach (speed, flexibility, compilation time, scope of game, etc.), but you need to decide that beforehand.
Once you know how you want to use scripting, you now need to decide if you're going to use an existing scripting language or your own domain-specific language. Today there are many scripting languages to choose from with different design goals and target audiences, so I'm not sure if it's worth creating your own anymore. If you decide to implement your own, here are some resources:
Lua is a popular lightweight and easy to embed scripting language. It uses a stack for communication between the host and embedded language and it was successfully used in many professional games. Sol2 simplifies the process of binding Lua and C++. If you're not fond of Lua's syntax, Moonscript is a language that compiles to Lua and adds a nice set of features.
I wrote a post about embedding GameMonkey script to a game using DragonFire SDK here.
Basically the idea is to expose your C\C++ functions to your chosen scripting language and use them from your script. In my tutorial, I exposed 2 functions from DragonFire SDK. At game start, I call onStart function from script and on update, I call onTimer function from script.
Hope that helps!