I think there are three aspects of this:
- Meaningful decisions.
- Balance.
- Mechanic complexity.
On the meaningful decisions aspect... well, if the answer to armor is more damage, then that is not a decision. It is good for there to exist multiple ways to deal with a situation. I mean, multiple different in kind (not just in raw numbers) ways. Not simply picking the attack with more dps.
If just more damage is always the answer, then deciding what attack to use is NOT interesting. It might aswell be automatic. There might aswell be a single kind of attack (and unique equipment).
Ideally we would have unique mechanics instead of more stats. However, from the point of view of meaningful decision, the option of bypassing armor is valid.
The answer is often not obvious.
It should not be. From the point of view of balance, if there is one single way that is clearly the best, then the options are not balanced. Au contraire, the use of armor penetration or just more damage should be primordially of preferred game style (and then of tactics and availability).
It also makes sense for there to be ways to counter or avoid armor penetration. It can be something that negates damage, something like evasion (if it makes sense in the game, e.g. turn based tactics) or user agility (if it makes sense in the game, e.g. action rpg), it can be attacking from greater distance, whatever.
To me this seems needlessly complex.
That goes for the particular case of each game, how exactly was the mechanic implemented and what other mechanics it synergy with. Armor penetration can be straight forward: it does this amount of damage ignoring armor. Done.
It can be complex, if the way to counter armor penetration is with armor that have some sort of resistance to armor penetration… then it isn’t really armor penetration, is it? I mean, designer can come up with odd logic for these mechanics.
If the player has to...
look up definitions and statistics on the web
... to be able to use it. Then we have a problem of clarity. The mechanics are not clear, not easy to understand. It could be that the mechanics should be reevaluated, or it could be just a matter of better UI (if it is not that bad), in fact if the UI designers or the writers are having trouble explaining the mechanic, the game designers probably should have another look at it.
On the other hand, if the player has to...
look up definitions and statistics on the web
To decide which is better, despite both using armor penetration and more damage (and hopefully other options) being perfectly viable options and easy to use... we have an optimizer player. It is good to have of those. And it is good to have mechanical depth for them. There is people who enjoy figuring out that stuff. There is people who take pride in having or using the optimal solution.
Also, web sites, video explanations, wikis, forums, and other online communities spawning around the game are a good thing.
If more damage cant scale with more armor, then shouldnt damage be buffed or armor be nerfed?
Buffs and nerfs are often not trivial. You need to consider other mechanics. Is there a mechanic that reduces armor? What about one that deals damage overtime? Are there invulnerability mechanics? etc...
While it is true that developers can create a mess, perhaps by good old accretion. That does not mean that armor penetration by itself is a bad design.
It could be it is a bad idea in whatever game you had in mind when writing the question, it does not mean it is in general.