0
\$\begingroup\$

How do games handle arrays/objects of billions of lines to load every gameobject/players positions of big maps for example (server side, for authoritative servers)?

I understand it could be split into chunks like in minecraft. As i imagine it, the client request nearby chunks data from the server. And the server load/unload chunk from separate json files when a player arrive near or inside the chunk. But is it the right approach ? It could also be loaded from a database but it doesn't look like the right approach performance wise (but opening local files neither...i guess)

This lead me to the idea of loading the entire map in a variable at runtime, but it will just explode my memory usage too and eventually even crash nodejs if there is too much data to load in memory.

Edit: i have just remembered that world Of Warcraft private servers running on CMANGOS are just loading all data in memory at server start, so maybe it is not that crazy to do

How do Minecraft or Eve online/echoes handle this part?

Improve this question
\$\endgroup\$
5
  • 1
    \$\begingroup\$ Why would these lists of objects take up a lot of memory? The server doesn't need textures or sounds or often even animations, so that removes most of the memory hogs. Instead it mostly needs positions, orientations, and IDs. That's a matter of bytes per data point, so even a billion of them is only a few gigabytes - well within the range of server hardware. And they don't need every object that's displayed on the client, only the ones that affect authoritative validation - so, maybe navmesh for the world, and sparse interactive objects, not every vertex of every 3D model. \$\endgroup\$
    – DMGregory ♦
    May 19, 2022 at 15:08
  • \$\begingroup\$ @DMGregory i agree but you still have to load an array containing all game objects positions that are needed for authoritative validation, and when it's a very big map, it still create a very big array to load at runtime. I have made tests with 1 million lines from a json file just for the sick of it and it 100% crash \$\endgroup\$
    – Owow
    May 19, 2022 at 16:40
  • 1
    \$\begingroup\$ JSON is designed for human-readable data interchange & thus isn't a great choice for transferring large amounts of data that only needs to be machine-readable. Of course it doesn't need to be one or the other. Minecraft's NBT format has some JSON like tagging, but uses binary where it makes sense in order to efficiently store/transfer data. \$\endgroup\$
    – Pikalek
    May 19, 2022 at 17:12
  • 1
    \$\begingroup\$ If the real issue is how to handle big maps in your game, you're far better off asking about that here rather than 'how did game X do thing Y'. While the latter isn't strictly off-topic, those questions don't typically do well here. Furthermore, some (possibly all) of the games you've asked about weren't implemented in JS, so their solutions are probably not as well suited to your situation as an Q&A about your particular project. \$\endgroup\$
    – Pikalek
    May 19, 2022 at 17:20
  • 1
    \$\begingroup\$ Those games you listed all work entirely different so handle their maps entirely different. Minecraft maps are designed to change and grow in all directions, WoW maps don't, the server doesn't need the map, it only needs to have the collision data, EVE Online doesn't even have a map. \$\endgroup\$
    – tkausl
    May 20, 2022 at 12:54

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .