In Dwarf Fortress, nearly everything that happens during world generation is logged and viewable in legends mode. An XML file containing that information can be exported and loaded into a utility such as Legends Viewer which can show specific events, relationships between two recorded people, what kingdoms people belonged to, etc.

My naive approach would be to add each event associated with a person to a database with a unique ID then filtering using the ID in order to view the correct events. However, this approach seems to be inflexible and adds more dependencies and complexity to my project.

My second approach would be to store these events in a JSON file in order to be able to treat each individual as an object, although this approach will probably be either be too memory intensive and not scale well since I would have to load a large file into memory or be too slow since I would have to open and modify many files at a time.

Assume the game has a simple system where each individual is tracked by a name and a UUID, belongs to a kingdom, and has some type of relationship to other individuals. Events are tracked by recording the location, date, and individuals involved. What type of file, data structure, library, etc. is generally used for something like this that is efficient enough to store up to a million or so of these simple entries?

  • \$\begingroup\$ I'm sorry, but this is a very broad question where the answers are usually very specific to the data structures of the game and how it needs to access that data. Can you be more specific? What are the requirements of your game? What would be your naive approach and where do you see issues with it? \$\endgroup\$ – Philipp May 26 at 10:45
  • \$\begingroup\$ This question is extremly broad and depending on so many details of your implementation. LIke if you use ECS, what type of entities do you need to keep track of etc. \$\endgroup\$ – PSquall May 26 at 10:46
  • \$\begingroup\$ Thanks for pointing out what needed to be changed. I have updated the question. \$\endgroup\$ – Austin May 27 at 19:14
  • \$\begingroup\$ This looks like a lot of speculation over problems you have not actually encountered yet, demonstrated via profiling, nor proven will be an issue with a back-of-napkin calculation. I think you might be worrying over the wrong thing here. Storing or iterating a graph of a few million entity nodes, relationship edges, and event timestamps is not the hard part of a system like Dwarf Fortress's legends, and either of the approaches you've described should work reasonably well to get you started. The hard part is the generator logic that makes interesting entities with sensible relationships/events \$\endgroup\$ – DMGregory May 27 at 19:53
  • \$\begingroup\$ I'd strongly recommend diving in with the most simple, even naive solution you can think of. Put everything into a big ol' list or CSV file - it can be hideous in your first pass. The point is to get it functional enough that you can start iterating on your entities and relationships, so you understand what kinds of queries and operations you need to do on the data to make them interesting and suited to your game. Then and only then, if some of those operations are slow with the structure you're using so far, you can look at ways to optimize them, once you know which ones matter & need it. \$\endgroup\$ – DMGregory May 27 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.