0
\$\begingroup\$

I would like to create an educational game, part of which involves allowing the user to modify code for procedurally generating geometry on the CPU and modify shader code and see the changes they make reflected in real-time.

Unlike How can I create a shader at runtime in Unity, I can't use a workaround with if statements, since one of the learning goals is to learn how to code shaders/procedural geometry, and I want users to be able to enter genuinely arbitrary code (ideally sandboxed) to explore the space of possibilities.

My current workflow uses javascript, THREE.js, and GLSL (through webGL), which can certainly manage this kind of process, but I'm considering learning to use a large game engine (such as Unity or Unreal) to take advantage of the conveniences they offer (and so I can learn how to use a large game engine).

How can I allow runtime shader authoring in the context of a larger game engine?

\$\endgroup\$
1
  • 1
    \$\begingroup\$ Unity has deprecated runtime compilation of shaders. You can use native plugin to do that, A bit cumbersome in my opinion: See this and this \$\endgroup\$
    – Mangata
    Commented May 11, 2022 at 21:28

2 Answers 2

2
\$\begingroup\$

This answer is about Godot. Since Godot games are running on the same core as the editor, they can do almost everything the editor can do. This also means that powerful modding and meta-programming is possible with Godot.


Godot

This is quite an advanced topic to be the entry point for Godot. I don't think I can make a proper introduction to the engine here, so please pick up a tutorial. Yet some concepts are going to be necessary for the answer, so I feel compelled to provide a quick and dirty explanation of some of Godot idiosyncrasy, mainly similarities and differences with other engines.

First of all, there is an SceneTree, it has Nodes, and each node can have a Script attached. Some kinds of Node include Viewport, Camera, RigidBody, MeshInstance, ImmediateGeometry, and so on.

You can have serialized branches (serialized Nodes including children), and to that we refer as "scene". So you can instance a "scene" inside the SceneTree. And you can also instance a "scene" inside another "scene", because it all one SceneTree. So there is no "prefab"/"scene" distinction.

Also, Godot is not component based, instead of adding components to a Node, you add specialized Nodes as children. Thus, again, it is all one SceneTree.

Well, we are actually going to challenge the idea that there is only one SceneTree in this answer. So, let us say it is all - usually - one SceneTree.

It is also useful to be aware of Resources. Resources are Objects that Godot knows how to serializase and deserialize. So, an "scene" is a Resource, of type PackedScene to be precise. Scripts are also Resources, and so are meshes and so on.

By the way, I've been told that Resources are similar to "ScriptableObjects" or "DataAssets", but more versatile.

Godot has both reference types and value types. All reference types ultimately inherit from Object. Among them, some are reference counted, which inherit from Reference. The Node does not inherit form Reference. Instead they are manually freed. Which also means you could have a reference to a freed Node, which is not a null pointer. You can check with is_instance_valid.

A GDScript script is a GDScript class. Or to put it another way, GDScript scripts can inherit from other GDScript scripts. They can also inherit from built-in classes. If we don't specify, they inherit from Reference. So all GDScript scripts ultimately inherit from some built-in type.

Finally, when we attach a script to a Node, we need a script that inherits from the class of the Node. In fact, when we call new on a script (and I remind you that the script is class as far as other scripts are concerned), Godot will instance an Object of the type the built-in type the script ultimately inherits from and attach the script to it.


Sandboxing?

Godot does not have proper sandboxing… And by that I mean there is no permission system or similar that restricts what the code can do. So the fact that Godot lets you do so much at runtime is a double edged sword.

However, I want to point out that you can create a separate SceneTree from the one you get by default. It is easy, actually:

var scene_tree := SceneTree.new()

We can call init on it:

func _ready() -> void:
    scene_tree.init()

The _ready function will run when the Node and dependencies has been loaded in the SceneTree, I mean, the main one in this case… The one in which the Node with script we are talking about is.

And with that the secondary SceneTree will start rendering over the main one. You can disable that if you don't want it to, with:

scene_tree.get_root().set_update_mode(Viewport.UPDATE_DISABLED)

Be aware that the root of the SceneTree is a Viewport.

Or if you prefer to have both, perhaps we can split the screen. The following code will set then side by side:

func resize() -> void:
    var window_size := OS.window_size
    var half_window_size := Vector2(window_size.x * 0.5, window_size.y)
    var left := Rect2(Vector2.ZERO, half_window_size)
    var right := Rect2(Vector2(half_window_size.x, 0.0), half_window_size)
    scene_tree.get_root().set_attach_to_screen_rect(right)
    get_tree().get_root().set_attach_to_screen_rect(left)

We want to run that code every time the window/screen resizes, so let us connect it to the "screen_resized" signal (and by signal I mean what you would call an "event"):

func _ready() -> void:
    scene_tree.init()
    get_tree().connect("screen_resized", self, "resize")

And while we are at it, make sure to call it at least once:

func _ready() -> void:
    scene_tree.init()
    get_tree().connect("screen_resized", self, "resize")
    res

Be aware that this is not a typical split screen. Usually when we split the screen we want to render the same SceneTree on each side, perhaps form a different Camera. But we are not doing that here. Instead here we are rendering two entirely different SceneTrees. Of which, one controls the other.


You can change the the scene of the SceneTree, for example:

scene_tree.change_scene("res://scenes/test/scene.tscn")

And we need to call into the secondary SceneTree each frame:

func _process(delta: float) -> void:
    scene_tree.idle(delta)

func _physics_process(delta: float) -> void:
    scene_tree.iteration(delta)

The _process method will run each graphics frame. And the _physics_process will run each physics frame. You can establish the rate of physics frames in Project Settings.

Finally call scene_tree.finish() when you are done with it.

I remind you that this secondary scene_tree is like the main one. It has a root and you can add Nodes to it.

Then you can put all the stuff made at runtime in that other SceneTree. It does not give you control over what that code can do (because, again, no proper sandboxing). So it can still make network requests, mess with the file system and stuff like that. But it will be isolated form your own (the code on the secondary SceneTree does not have a way to get a reference to the main one).

ERRATA: There is a way: Engine.get_main_loop will give you the default MainLoop which is the SceneTree (the SceneTree class extends MainLoop). By the way, you can change the main loop from project settings, under Application -> Run -> Main Loop Type.


Ultimately you would probably want to parse the code from the user. This will allow you to provide pretty printing, and also making sure it does not call what it should not. You can start with the GDScript reference grammar. The documentation does not include the equivalent for the shading language, however it is basically GLSL.


Creating shaders at runtime

I'll point out that Godot has VisualShader but the UI to create them is not included in games. You could make your own UI for them (Based on GraphEdit and GraphNode). But this answer will cover text based shaders instead.

To create a shader in runtime in Godot, we can follow the following approach:

  1. Create a new ShaderMaterial object:

    var shader_material := ShaderMaterial.new()
    
  2. Create a new Shader object:

    var shader = Shader.new()
    
  3. Then set the shader code for the shader:

    shader.code = """
    shader_type spatial;
    render_mode unshaded;
    
    void fragment()
    {
        ALBEDO = vec3(UV, 0.0);
    }
    """
    

    Here I'm using """ for a multiline string literal. The example is an a spatial shader (so one you would use for a MeshInstance (3D)) that output the UV coordinates as color.

  4. Set the shader to the shader material:

    shader_material.shader = shader
    
  5. And set the shader material to where you need it:

    $MeshInstance.set("material/0", shader_material)
    

    Here it will be the first material (0) of the mesh.

    Oh, by the way, the $ syntax is used in GDScript to reference a child Node by name.

Just because you have already set the material does not mean you cannot replace the code. So, if I have already set a shader material, I can just replace the code, no hassle:

var shader_material = $MeshInstance.get("material/0")
shader_material.shader.code = """
shader_type spatial;
render_mode unshaded;

void fragment()
{
    ALBEDO = vec3(1.0, 1.0, 0.0);
}
"""

This shader outputs pure yellow.

Beyond that, if you want to allow users to write shader code, you could provide a text editor, take what the write, and perhaps do some string manipulation before setting it such as shown above.

Caution:

  • Make sure you have a ShaderMaterial with a Shader before you try to modify its code, otherwise you are going to get an error.
  • If the code is invalid you will also get an error.

In this error conditions you are going to get the default material. So they are not fatal. However, I haven't found a good way to detect them from GDScript. What the editor does is output the errors to the console. We could use logging (you can configure it on project settings, it is enabled by default) and monitor the logs, but that is not practical at all.

  • When the shader compiles (when it is first seen), Godot may skip a frame. Godot 3.4.5 (currently in beta) should allow for async compilation of shaders.

Creating meshes at runtime

The simplest way to generate a mesh at runtime is using a Node of type ImmediateGeometry. Its usage is similar to old school OpenGL's immediate mode.

For example, this would draw a segment (I'm assuming this is running in a script attached to the ImmediateGeometry):

clear()
begin(Mesh.PRIMITIVE_LINES)
add_vertex(Vector3.ZERO)
add_vertex(Vector3.FORWARD)
end()

However, despite the name and the similarity to the immediate mode, it is actually creating a mesh under the scenes. Which is why we need to call clear to reset it, and also why this code does not have to be in _process or any particular method. In fact, the coordinates of the vertex are local, and you can position the ImmediateGeometry like any other node.

We can, of course, make a triangle mesh with Mesh.PRIMITIVE_TRIANGLES instead of Mesh.PRIMITIVE_LINES.

And we can set:

  • Vertex color with set_color.
  • Normal with set_normal.
  • Tangent and binormal with set_tangent. It takes a Plane, the normal of the plane is the tangent. The distance to the plane is the scale of the binormal (you probably want to pass 1.0).
  • UV with set_uv.
  • UV2 with set_uv2.

You can assign a material to the ImmediateGeometry by setting its material_override. If you need more than one material, or some other features, we need to move to SurfaceTool.


To use SurfaceTool, create an instance:

var surface_tool := SurfaceTool.new()

Then we use SurfaceTool pretty similar to ImmediateGeometry above. So we can call begin, add_vertex and so on. Furthermore, you can set a material with set_material and even define bones and weights. Taking full advantage of SurfaceTool is beyond me, but…

What you are making is a surface. You are going to add these surfaces to a mesh. So have a mesh:

var mesh := ArrayMesh.new()

And then add the surface to the mesh:

surface_tool.commit(mesh)

Do that for as many surfaces as you need. Each surface can have its own material.


You may also be interested in MeshDataTool which provides some capacity to query and edit existing meshes (instead of authoring new ones).


Creating and running scripts at runtime

Godot also support visual scripting ("blueprints" style), but similar to what happens with VisualShader, the UI to create them is not included in games. But we have text based scripting, and that is what I'll talk about.

You can create that a script in runtime too. Create a GDScript object:

var script:= GDScript.new()

Then set its source_code:

script.source_code = """
extends Node

func method() -> void:
    print("hello")
"""

Yes, you could also read the source_code of existing GDScript scripts. On a similar note, you could have the script created at runtime inherit from another script.

Then reload it… This time checking if it went OK is readily available:

if OK != script.reload():
    print("Error loading script")

And now you could attach it to an Object (e.g. a Node) with set_script. Or you can instance it directly, and use it:

if script.can_instance():
    var instance := script.new() as Node
    instance.method() # should print "hello"

I also want to mention Expression which will let you execute - as the name suggest - an expression.

Start by creating the Expression object:

var expression := Expression.new()

Then parse the string expression:

var parameters:PoolStringArray = ["p"]
if OK != expression.parse("2 + p", parameters):
    print(expression.get_error_text())

In this case the expression will have one parameter "p" which we need to pass when we want to execute it.

And, of course, execute it:

var argument := [5]
var result = expression.execute(arguments, null, true)
if not expression.has_execute_failed():
    print(result) # should print 7

The arguments are passed to the parameters according to their position in the array.

The null I'm passing is the self reference. And the true is enabling error output.

So Expression can be a way to execute something that might fail and check if it failed. And, no, it does not work for the shader, as the error happens when Godot compiles it not when we set its code.

\$\endgroup\$
0
\$\begingroup\$

Polygonjs may be interesting for this situation. It is nowhere near as complete as Unity or Unreal, but it provides a layer on top of threejs, where you can create your own shaders as well as procedural geometries with a node-based editor.

(disclaimer, I am building it)

Its core library is also open-source, so you could easily extend it if needed.

And you can edit the shaders live from the editor. But by default, when you export a scene, the shaders are not editable via Polygonjs api anymore, only via threejs api. The reason is to make sure the export scene is as optimized as possible. Happy to discuss options if that was something interesting to add.

\$\endgroup\$
4
  • 3
    \$\begingroup\$ Moderator note: I see some users have flagged this answer as spam. Note that the flag reason says "Exists only to promote a product or service, does not disclose the author's affiliation." However, the answer above does disclose the author's affiliation with this software/service. The suggestion is also relevant to answering this question. So I don't think this answer merits deletion. I do think the answer could be improved by editing it to explain in more detail how to implement these features with this tool. \$\endgroup\$
    – DMGregory
    Commented May 16, 2022 at 17:48
  • \$\begingroup\$ thank you. I'm hoping to find the time to add details and code samples.I've also just received a message from the moderators in my inbox, but it is truncated and it looks like I do not have permission to view the full message. \$\endgroup\$
    – polygonjs
    Commented May 19, 2022 at 13:48
  • \$\begingroup\$ I don't believe any moderator on GameDev.StackExchange has sent you a moderation message, but I cannot speak for other sites in the network. \$\endgroup\$
    – DMGregory
    Commented May 19, 2022 at 13:54
  • \$\begingroup\$ ah, apologies, it was from stackoverflow. I'm still getting my head around the different sites. Thank you for pointing me in the right direction. \$\endgroup\$
    – polygonjs
    Commented May 19, 2022 at 20:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .