I'd like to pay an artist to create art assets for my game. Is it common for the client (myself in this case) to exclusively own all the artwork that I pay the artist to create and deliver? How can I make sure of this?
I am not a lawyer; this is not legal advice.
As long as the contract you agree states that you own the artwork and you make it clear to the artist that this is the case, you will have no problems.
I believe it is common for the client to own it. But with everything: explicit is better than implicit. If you are not 100% certain you will own the assets, make sure it is written in the agreement.
Dependant on the country you or your artist are in, the artist can still have certain rights to reproduce and use the work as he sees fit. The law applied will obviously depend on the country and hence there can be conflicting laws. Some countries stipulate that any creation remains the intellectual property of a person, and they cannot sell their intellectual property (at least not in it's entirety). I would recommend seeking a lawer's advice to make sure your contract is waterproof, and try to contract people in the same country as you.
Common law: employer owns the copyright in work created by employees
Civil law: employer enjoys an exclusive licence to the economic rights in work created by employees
What's common isn't nearly as important as what you and the other party agree to. It may be important, however, because an artist may wish to earn more if they also have to give all their rights away to their works.
You might also consider some sort of joint ownership agreement where there are some restrictions on what the author does with their work, such as a limit on who they can sell to in order to prevent them from selling their same work to a competitor of yours while also leaving them free to sell it for uses that are not in the same industry (e.g., for packaging on some instant noodles), but this can get very complicated (e.g., the instant noodles are targeted at gamers -- this may or may not be beneficial to you).
Hiring a competent lawyer with suitable experience in contract law would certainly be a wise choice on your part.