When I read discussions about state machines vs. behavior trees, it is often in favor of using one over the other. However, this seems odd to me that you wouldn't still need a state machine to manage the player character and static game entities (weapons, equipment, food, money, etc.). Do behavior trees translate to those concepts as well, even though they aren't technically AI?
A behavior tree is essentially a sub-type of a (possibly non-deterministic) state machine. Behavior trees are used not because they can do something FSMs can't, but because they are simpler to follow and understand for us humans.
Generally, AI has tons of states, and behavior trees are used to make them understandable. Other management, on the other hand, usually uses few states, and is perfectly clear even with most brain-dead FSM implementation.
If you, for some reason, have really complex behavior outside the AI - for example, you're making a game about alchemy, with tons of interlocked recipes, item properties, etc - using behavior trees instead of state-machines might be helpful.
Look at it like this, an enemy entity (clearly something in need of AI) is something that needs a behavior of some sort. We need this behavior to be conditional and not totally predictable. A behavior tree (with weighted-random chances when making a choice) is a good fit for something like this.
Now let's take a look at a gun. This clearly is not an item that is in need of a complex behavior, frankly I would not even suggest a (complex) state machine. All the gun needs is a few rules:
- Trigger pulled + clip is not empty = fire bullet
- Trigger pulled + clip is empty = make click sound
So that's basically how I see how someone could make an educated guess on what system best to use, of course there are always exceptions but always first try the simplest solution imaginable, often simple rules are the most fun (since people can understand them and mechanics do not become a black-box).