There is no "industry standard" ( why do you think there are so many different systems? DnD/PF/d20, WoD, d6, GURPS, etc... ) when it comes to comparing two players and balancing the game.
@hustlerinc has a pretty good answer that focuses on the difference in player level, and handles it fairly between two people only taking into account their level.
I would revise your equations to something more like this:
Battle starts If player.dex > enemy.dex Accuracy = ((player.dex - enemy.dex)/2)*20
The equation itself looks alright, but you're doing extraneous math.
(player.dex-enemy.dex)*10 gets you the same result. This now shows us the problem: If player.dex is ever 10 above enemy.dex, he will always hit. If your stats are reasonably bound ( something like 8-25 ), should keep more or less balanced. Additionally, what if they have equal stats? neither could ever hit the other.
If player.dex < enemy.dex Accuracy = ((enemy.dex*player.dex)-enemy.dex)/player.dex
This equation just looks terrible. Without knowing the range for stats, this equation makes absolutely no sense.
Accuracy is a percentage tested against a roll of a 100 sided die.
If accuracy > roll Hit success
If accuracy < roll Hit fails
Standard type stuff. I would actually suggest a different system. Start at 50% chance to hit. Use some sort of curved algorithm to determine the change in chance for one side or the other. For instance, 10v10 = 50%, 11v10 = 59%, 12v10 = 66%, 13v10 = 71%, etc. etc. For every point above the opponent, give them a bonus, but have it be decreasing return per point. Again, it depends on what your proposed ranges are.
If hit success All of the attacker's damaging items (spiked helms and shields, weapons, etc.) are added together and applied as such: Damage = (player.str + item.damage)-(enemy.dex + enemy.armor)
This could go extremely wrong. Suppose your player is using a ranged weapon ( which is usually based off of dex ), and the enemy is using a melee weapon ( based off of str ). This gives an inherent advantage to melee weapons, mechanically speaking.
If the player is fighting a monster, I have a base amount of gold and experience which is modified according to the player: Gold gained = (enemy.gold*rand(1,10))/player.level Exp gained = (enemy.exp*rand(1,3))/player.level
This I disagree with vehemently. Never divide by player level - you end up making a logarithmic ( reciprocal of x ) curve. Simple put, upon moving from level 1 to level 2, the character now gains half experience and half gold. From level 2 to level 3, he gains two-thirds of what he gained at level 2. Then 3/4, 4/5, 5/6, etc. The difference in gain from one level to the next approaches zero.
There are a couple of ways to deal with experience and gold. I find I tend towards the low multiplier method. Essentially, each level above 1, the amount of experience you need for the next level increases by a low multipler, something in the .125-.25 neighbourhood. However, this is cumulative. You end up with an equation for XP like such:
baseXP * ( ( 1 + XPmod)^level )