I'm at the early stages of developing a 2D side scrolling open ended platformer (think Metroidvania) and am having a bit of difficulty at enemy design inspiration for something of a scifi, nature, fantasy setting that isn't overly familar or obvious. I haven't seen too many articles, blogs or books that talk about the subject at great length. Is there a fair rule of thumb when coming up with enemy art with respect to keeping your player engaged?
You should look at your design document. It should contain background information about the world your game plays in. You do have a design doc, do you?
If not, start by thinking about the setting...
Medival european fantasy? Arabian nights fantasy? Cyberpunk megacity with a hint of magitech?
What kind of places would you find in such a world?
Who would live in such a place?
Rats and mutants in abandoned sewers?
Why are they living in the sewers? Poverty? Social Stigma? Choice?
Asking questions about the world your game is set in can guide you towards certain design decisions, or might even imply certain art choices, e.g. sewers - dim & gritty, metropolis - clean and shiny.
Where does the enemy come from might allude to what they look like. E.g. soldiers from kingdom X might wear wooden armor because they don't know smelting.
The design might also be influenced by the needs of the story, e.g. you might want easy enemies for the start of the game, so you might want to limit enemies in the first stages to grunts, while in the later stages, you can go wild with ubergrunts and elite troops with shields and katanas and rolleskates (or jetpacks, or wings) or something.
For more on creating worlds, see http://en.wikipedia.org/wiki/Worldbuilding
There are a lot of recurring 'tropes' you could use, too: http://tvtropes.org/pmwiki/pmwiki.php/Main/Trope e.g. http://tvtropes.org/pmwiki/pmwiki.php/Main/UniqueEnemy
WARNING: Browsing tvtropes.org can lead to excessive wikiwalking!
Finally, you could get inspiration from other games. I watched some speedruns of the contra series on youtube lately... lots of different enemies in small 30 minute packages.
I found this to be a good source, in an attempt to address similar challenges that I faced.
Hope this helps.