This was going to be a comment, but it's really long enough to be an answer.
The answer is (unfortunately) you can't avoid or minimize low-level work. It's just impossible.
Why? Because high-level is built on low-level. There are layers of abstractions upon abstractions; but at some point, somewhere, abstractions leak or are imperfect, and you need to dig down. How far down depends on the problem. You may find CLR bugs through some third-party's application on top of another third-party's library.
Let me give you an example. You might have a web app which uses ActiveRecord, which in turn uses NHibernate, which in turn uses C#. It's possible to find issues that depend on the way the CLR was implemented on a particular architecture (eg. x64).
Plus, most likely as a developer, if there' s a low-level bug, you will need to trouble-shoot and fix it, because it will impede your high-level work. You can't just say "here, ask X to fix this" because you're a "high-level" developer. YOu have to get the job done, whether it's your code or some other layer of the cake that has the problem.
If I were you, I would pick a fairly high-level platform to specialize in (like XNA, or even XNA specific to X platform) and focus on improving your debugging skills. That will carry you very, very far.
Sorry if this is not the answer you want to hear! This is just the (sometimes sad) reality of code. It's layers, and imperfect abstractions.