One option is to create a damage type called Reaction Damage, and construct your rules so that Reaction Damage does not trigger Reaction Damage.
The Thorn effect then becomes "When another player deals non-Reaction Damage to you, deal X Reaction Damage to that player"
This way you can cleanly separate initial effects from follow-up effects, and strictly restrict the depth of recursion for this class of abilities.
Once this is a keyword that you use consistently, it opens up new avenues for other cards to interact with these reaction abilities, like...
Careful Strike: take no Reaction Damage this turn
Counter Master: whenever you deal Reaction Damage, double the amount of Reaction Damage you deal
Reactive Vampire: heal 1 health point any time you deal Reaction Damage
Another strategy is to use diminishing returns, so each invocation of Thorn or similar effects does less damage than the last (eg. by subtraction or division), and eventually the recursion "bottoms-out" at zero.
This can be more brittle than the hard recursion limit described above though. If there are enough damage-boosting/vulnerability effects in play, those could potentially counteract the diminishing returns and get us back into an infinite regress.
Usually in such a case one player or the other would eventually be defeated, and we could terminate the loop that way (halt all card effect events emitted by defeated players), but again there could be exotic effects in play that somehow interfere with this soft limit (eg. a zombie effect that lets the player finish one last turn after being reduced to 0 health).
So I think the hard limit on recursion is more robust to other comboing effects.