0
\$\begingroup\$

I'm designing a kind of simple open world with ECS. The whole world is too large to be loaded at once, so I load and unload chunks according to player's position. Nothing fancy, pretty much standard nowadays.

In this world, I have entities that I need to update frequently. There are a couple of thousands of them. These entities share some components and a tag to retrieve them, but can refer to different archetypes. These entities are placed by hand, before runtime. No entity of this kind is created on runtime.

I need to update these entities every five minutes. I do not need them to update in a single frame, it can be scheduled over multiple frames (so performance is not exactly critical here but as always, the more performant the better).

Problems:

  1. I need to update every entity, even if not loaded according to player's position
  2. I need to save and load the state of these entities

Currently, the position of these entities is guaranteed to remain the same and it is guaranteed too that two entities can't share a same position. However, it may change.

Considered solutions:

Solution A

I create a singleton "manager" that holds a container. Keys are the position of the entities. I iterate trough the container to update the values, so I don't need to load entities.

I update the entities currently loaded with the new values retrieved from the manager.

Pros:

  • Fast
  • Easy to work with
  • No need to load entities (and meshes and so on)
  • Trivial to serialize and save
  • Do not require additional work on the world creation

Cons:

  • Not flexible as the position is now a key (could be a problem if I want the system to evolve)
  • May cause issues when parallelizing
  • Having positions as keys is probably a very bad idea
  • Defeats the principles of ECS as data is not "owned" by the entities, but is centralized

Solution A (alternative):

This time, with a script I execute before runtime, I set an ID to each entity in my world as part of a component. I use this ID to store the data in my container (which is now a simple array and not some sort of a dictionary).

Pros:

  • Very fast
  • Easy to work with
  • No need to load entities (and meshes and so on)
  • Trivial to serialize and save

Cons:

  • Does require additional work on the world creation
  • Still not very flexible (but better than positions)
  • May cause issues when parallelizing
  • Still defeats the principles of ECS as data is not "owned" by the entities, but is centralized

Solution B

This time, all the data remains in the entities. I have to separate my chunks into two parts: a part with the entities I need to update (let's call that an update-chunk), a part with the rest of my entities (a regular-chunk).

I first have to create a save for my entities: I load every update-chunk, I update the entities, I serialize them then unload them.

Then, for each update: I deserialize my entities, I update the entities, I serialize them again.

When a chunk is loaded: I deserialize my entities and I load the regular-chunk only.

Pros:

  • I'm using ECS, yeah!
  • ... Nothing much

Cons:

  • Ridiculously inconvenient and complicated
  • I need to serialize over and over again
  • Probably very, very bad performance
  • I save a lot of data I already "saved" in my base configuration (so many duplicates)
  • Difficult to set up as I have to create twice as much chunks
  • I have to load meshes/textures associated with the entities even if I do not need to render them (I can't change this behavior in the framework I use, or I need to code another system and this is getting far too complex)
  • Probably more but that's way enough to disqualify this solution

Conclusion:

I'm pretty much convinced I'm doing something wrong and that I missed something really important.

Solution A may perfectly work (after all, I have no need to absolutely use ECS for everything) but it bothers me that I can't figure out how to solve this problem with ECS.

\$\endgroup\$
5
  • \$\begingroup\$ I would say that a principle of ECS is that data is not owned by the entities, but that you can manage it in whichever way is best for your application. Is this Unity? \$\endgroup\$
    – user253751
    Jul 13 at 9:13
  • \$\begingroup\$ Well... multiple entities sharing "references" to a same component seems like a very questionable choice of design, I would say. Probably there are some very specific use cases, but that's way outside the scope of my question since I don't even mention some kind of shared values or references between entities. In most cases the data is still owned in a way by an entity, since a component may be attached to a single entity. And to answer your second question, this is not Unity but I don't understand why it is relevant since the problem is pretty much framework/game engine agnostic. \$\endgroup\$
    – Sisyphe
    Jul 13 at 9:58
  • \$\begingroup\$ what do you mean, multiple entities? There is one entity, the thing that updates. Sometimes it has different components attached to it. All of them belong to it and aren't shared... I asked if it's Unity because apparently Unity has a thing called "ECS" that isn't actually ECS \$\endgroup\$
    – user253751
    Jul 13 at 10:08
  • \$\begingroup\$ If the data is not owned by an entity, then it is shared among multiple entities. Owned by design I mean, not constrained in the code or anything else. And ok for Unity, I was not aware of that since I never tried their DOTS thing. \$\endgroup\$
    – Sisyphe
    Jul 13 at 10:17
  • \$\begingroup\$ but how is it not owned by an entity? The data about the NPC (let's say it's an NPC) is owned by the NPC... \$\endgroup\$
    – user253751
    Jul 13 at 10:18
1
\$\begingroup\$

Solution A (Alternative) seems reasonable given the constraints you mention in your question and I suggest giving it a try. Regarding your hesitancy to stray from an ECS; notice that there is a fundamental conflict between the chunk system, ECS and these updating entities: Your components are stored on a per-chunk basis (as far as I can tell), the chunk system makes unloaded components inaccessible via serialization, and yet the entities must be accessible to be updated. This is a hint that a solution outside of your current framework may be reasonable.

How radically different your solution might end up from your current systems is highly dependent on how many invariants you can place on your updates. Lets say each update will only ever affect the entity itself and the effects of the update are only needed when the player is within loading distance of the entity (for example a light that changes colour every five minutes), now you can create a solution that simply stores the amount of time spent unloaded with each chunk and between deserialization and rendering let the entities within chunks consume this time to appear as though this unloaded time had passed. On the other end if the updates that need to occur are much more depthful, lets say some physics interactions need to happen like a rube goldberg machine, then we arrive closer to solution B where loading and updating distant chunks may be necessary for correct behaviour.

Importantly, notice that since your updates are fundamentally global, most solutions must circumvent your chunk system. The unloaded time queue adds data that never gets serialized and deserialized as the player moves around, the deserialize-update solution must undo the work of the chunk system to make components accessible for updates, and your solution A stores a global list that is unchanged by player position.

\$\endgroup\$
3
  • \$\begingroup\$ Thank you for your thorough answer! My components are indeed stored on a per-chunk basis and the update must affect every entity, even those which are not loaded. You pointed out how a solution outside my current framework could be the best way to go... and you are probably right. I will give A (alternative) a try with some stress tests, I just have to unsure that goes nicely with some parallelization, which is pretty much needed in my case for decent performances. I will update (and accept your answer) in a few days if that goes well. \$\endgroup\$
    – Sisyphe
    Jul 12 at 18:46
  • \$\begingroup\$ @Sisyphe In a "pure" ECS it doesn't make sense to say an entity is not loaded, because an entity is just an ID number - if you have the number then you have the entity. But some of its components might not be loaded. And you are allowed to store different components of the entity in different places. \$\endgroup\$
    – user253751
    Jul 13 at 9:14
  • \$\begingroup\$ @user253751 Of course, but you understood what I meant by "entity not loaded". It doesn't change anything in the problem. Yes, I could have write "entity's components" but since I obviously need to update the values of its components (and not the entity itself... since there is pretty much nothing to update in an entity itself), just saying "entity" is probably crystal clear for everyone. \$\endgroup\$
    – Sisyphe
    Jul 13 at 9:48
1
\$\begingroup\$

We had to solve a similar problem when developing Starlink: Battle for Atlas. The game takes place in a system of seven planets, and although only a small chunk of the current planet around the player is fully-loaded at any given time, the allied and enemy factions continue to battle it out on the far side of the planet and all the other planets.

We did this by distinguishing between two forms of entity:

  • a "fully realized" entity with its full AI brain, animation and rendering features, physics, etc. and...

  • a "sim unit" that's stripped down to just the bare minimum state we need to tick the away-from-player simulation, like a position, health, faction ID, etc.

You could employ a similar strategy within your ECS. It might look something like this:

At authoring time, you attach a SimUnitMapping component to each entity that needs this kind of persistence. This component associates the fully-realized entity with a specific SimUnit ID (like a serial number or GUID), and optionally some metadata about what needs to be persisted in its de-rezzed form.

When an entity with a SimUnitMapping component is about to be unloaded, it's processed by a dedicated system. That system reads the components on that entity and spawns a new SimUnit entity with the corresponding ID, populating it with the correct components and values to persist the state you need persisted. (These components could be the very same ones you use on the fully-realized version, or specialized slimmed-down versions with their own low-fi systems for the background sim)

Similarly, when you enter a new chunk and need to load the entities there, loading an entity with a SimUnitMapping component tells you to look up the SimUnit with the matching ID, and copy over its persistent state. Then you can delete the SimUnit, transferring responsibility for its updates to the fully-realized copy, until it is again unloaded.

In this way you keep using entities throughout - you never have to leave the ECS paradigm. You just pass data back and forth from one collection of entities to another, and can specialize each collection to the needs of either background simulation or foreground player interaction.

If you need to be ticking these SimUnit entities even before the player loads their realized versions in the first place, you can do so by generating a list of all SimUnits as a build process step, and instantiating all those entities when starting the game, before loading the first chunk.

\$\endgroup\$
3
  • \$\begingroup\$ Thank you for answer! It's close to what I designed in my solution B, but I didn't think of stripping out what I didn't need to make two separate entities. I will definitely try this along with Solution A. However, I feel like this one is still more complicated and I'm not sure at this moment of the benefits to remain in the ECS paradigm for this part of the game. Also, how would you retrieve efficiently the corresponding entity? Iterating through all the SimUnit entities simply to find the matching one seems like a very bad idea permformance-wise... or am I misunderstanding something? \$\endgroup\$
    – Sisyphe
    Jul 13 at 7:23
  • \$\begingroup\$ You only need to do that iteration when loading new entitles, not every frame, and the usual speedups using either binary search or hashmaps can be applied here. \$\endgroup\$
    – DMGregory
    Jul 13 at 10:28
  • \$\begingroup\$ Hum... so I still need to store some kind of reference for the SimUnit (or store the unit directly in a container). This is getting closer to Solution A then and I'm not sure of the benefits of ECS here. I will try that and compare. Thanks a lot for your answers! \$\endgroup\$
    – Sisyphe
    Jul 13 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.