There's a lot of variety in how you could do this; I'm going to suggest the “obvious to me” choices, but there are lots of variations that could be designed. Disclaimer: I haven't actually implemented anything like this.
First, you need a data structure that covers your world. If you are doing NPC movement in a 3D space, then you probably have, or will eventually need, such a structure for pathfinding purposes — say, a navigation mesh. So, let's assume we can add a field for scents to that.
So, what do we put in that field? I propose a list of records (scent, strength, time). This list is kept at or below a maximum length, and sorted by strength — so weak scents will be discarded. The scent could be either something explicitly defined for each entity or entity type, or it could simply be the entity type — depending on what you want to be able to track distinctly. The time is a timestamp of when this scent record was last updated.
When an entity passes through an area (e.g. a given triangle of the navigation mesh), it's time to update the scent list. First, decrease all the strengths according to the amount of time passed according to the time value vs. the current time — exponential decay is probably a reasonable choice here. Then add the current entity's scent to the list, perhaps with a strength dependent on the entity type. Then if the list just got too long, discard the lowest strength.
To obtain a tracking result, find the scent in the list for the current location, then do the same for all of its neighbors, and go in the direction of the strongest scent (that isn't the direction the tracker just came from).
For extra realism:
Diffusion: periodically transfer a fraction of the scent in each location to its neighbors. This confuses trails, but also means that stationary things can be sniffed out (finding food, dead bodies, etc). (It's even a recognized sort of AI to base actions entirely on this type of information — the landscape provides information about which way to go to obtain a particular resource, etc. I forget the name of it.) The main disadvantage is the time spent calculating diffusion everywhere.
Strong scents should prevent the detection of weak scents; divide the strength of the sought scent by the strength of the strongest scent, and fail if it is too small. This could allow deliberately confusing one's scent trail.