Should references between assets be handled by name or ID or something else?

Question

How should content or asset items be referenced in their serialized form?

For example, a material might need to reference several textures. The simplest way would be to use a path, relative to the base directory of the assets.

Alternatively, all content items could be referenced by an ID. I implemented it this way (using GUIDs), which allows content items to be renamed or moved without breaking references. However, this makes it more difficult to replace a content item (delete the original, then rename something else to the same name... requires copy/pasting GUIDs around). It also makes debugging missing content trickier.

Is there a clear winning option here?

Answer 1

String-keying / Hashmaps

• Are fast, as read time is amortized O(1), meaning that read access is usually very fast, but in worst cases (rare, but not unheard of), it can be quite slow. Worst case results from hash collisions.

• Implementations sometimes have to be built, or found (for instance, in C). Writing / finding a performant string-keyed map implementation can be non-trivial.

• String comparisons can be expensive, though whether this is pertinent depends on your language's implementation of both hashmap read ops, and string equality implementation.

• May make it possible for you to inadvertently create duplicates or overwrite existing values (depends on implementation).

Numeric keying / arrays

• Are very fast, because the structure you're adding to maps directly to memory (as per arrays in most languages)

• Are very fast, because numeric equality is checked rapidly (at least for sufficiently small bitwidths)

• Are very fast, because long, contiguously-allocated arrays can have superb cache performance (caveat: this depends very much on your choices of language and implementation).

• Require that your "GUID" is not too large to act as an array index. However, fast options can be found to work around this, such as breaking up your ID (via bit ops) to use as two sufficiently-short indices per element in a 2D array.

• Are inconvenient, for the reasons you've mentioned. If the IDs are compile time values that you need to be able to develop with, it's less of a problem, since they can be assigned to appropriately-named ALL CAPS constants and used that way.

• Can reference structures that contain a human readable name. This offers some convenience, even if they can't be accessed by this name.

Conclusion

Unless you are going to be processing many elements (say, tens of thousands) per 16-20ms tick using either of these options, I wouldn't worry about numeric IDs, and particularly if you are using a language where efficient hashmaps are already available, just opt for a hashmap.

Otherwise, the numeric ID / array approach is good for conserving CPU cycles, but go for this in cases where resources come at a premium.

Answer 2

I would strongly advocate the use of IDs, and GUIDs especially. GUIDs have a number of excellent advantages:

• They can be generated locally and still preserve their uniqueness. Simple numeric IDs require a central authority to guarantee uniqueness, which makes them immediately more cumbersome to use once you have more than a single person working a project.

• Altering the name of an asset does not require the expensive operation of locating all the references to that old name and updating them. Not only is this error-prone, it also has severe problems in a world where many people are collaborating on a data set (if somebody is working on a file referencing the one you are changing, then either you are forced to wait or they are forced to resolve the merge).

• Moving assets around within the filesystem or logical hierarchy of the content store does not require updating any references either.

GUIDs do have disadvantages, but they are all solvable with a little effort.

• They are hard to remember and type, which means it can be difficult to collaborate with another user ("Look at object {CD0B6FBB-1156-4BCF-A5EB-82967376090E}" is a bit cumbersome). However, you can build into your asset editing toolchain a way to look up objects by name, or even better, build a URL scheme for your tools that allow you to generate clickable links that load up the tools viewing the desired asset.

• They are big. This is generally not as much of a concern during development, but is for shipping the code. However, it's very easy to support a data-compile-time step that produces a remapping from GUIDs to integer IDs when baking final content for the game.

On Guild Wars 2, we used named-references to content for several years early in development. It was the wrong decision and hamstrung the performance of our tools for quite some time before we were able to transition to GUIDs. Every project I've ever worked on that used named-references had trouble with them, so I would absolutely recommend against them. The effort it takes to build tool support around GUIDs is less, in the long run, that it will take to cope with named-references.

If you have a small project, just by yourself or maybe a handful of others you can get away with using IDs (giving each person a dedicated "block" of IDs, for example, can easily avoid the clashing problem for some time). But if you have a small project the overhead of GUIDs isn't likely to be problematic either, so I'd just say you should use those for the best future scalability.

Answer 3

It is easier to think about it when you make a distinction between external identifiers and internal identifiers. An asset has a unique external identifier that is referenced in configuration files, makefiles, documentation, etc. Once an asset has been loaded into your application the assets might get an efficient internal identifier for internal referencing.

For the external identifier, I would use a consistent naming scheme, following the convention over configuration paradigm. So you might want to use a path-based scheme such as 'resources/textures/walls/concrete.png'. This identifiers tells you a lot: that it is a resource, that it is a texture, that it belongs to the texturegroup 'walls' and even where it can be found on the disk. Plus it is human-readble, much more so than a GUID.

For the internal identifier, you can choose to use either memory locations (pointers), increasing integers or GUID's. Ofcourse you can also opt to use the external identifier as your idental identifier, at the cost of higher memory-usage and slower performance.

To get an internal identifier from your external identifier you will most likely need a central mapping/repository in your application. Typically the resourcemanager that loads your resources can be used for this.

Finally, about the point of missing references: as resources are typically lazy-loaded at runtime, you should have some kind of test-suite or integrity check for your application anyway, so I don't think it will matter much what kind of identifier scheme you will use.

Answer 4

There's no best approach, because it always depends on the actual use case and your overall architecture.

Using IDs or GUIDs won't necessarily be easier to use or replace. Just remember that you'll have to copy and use those as well. Especially GUIDs are typically rather long and hard to remember without copypasting. Same can be true for raw numbers of some kind.

As such, "talking" names are probably the best approach, even if this involves some more writing or replacing.

What you can do, is using some fixed length identifier. For example, you could use 4 alphanumeric characters to identify everything. This also has the added advantage, that you'll be able to easily cast the ID (string) to an integer (just keep byte order differences in mind) and compare it the fast way.

This could also be extended to the file name as well, so you won't have to refer to full paths either. For example, if I use "NMY1" for my first enemy, it's data would be stored in the file "NMY1.dat", with the texture sitting in "NMY1.png", script code in "NMY1.lua", etc.

Another thing would be modability: If you want users to be able to easily mod your game, try to avoid overoptimizing it. I.e. prefer talking names over abbreviations, don't use raw IDs or GUIDs and use something easier to read/understand/copy instead, even if that means you might have to update lots of references as well.

If your data gets too complex to easily update/check, write tools to do it for you (and to verify integrity as well) or rethink your approach.