TL;DR
is there any good reason to let monsters drop things
It is a cheap mechanism (in terms of development time) to keep your players playing and motivated. Quests and storylines are complicated to integrate/come up with and are no longer of interest to a player once he has completed and replayed them 2-3 times. On the other hand, the need to kill 200 monsters to get a single item that you need 100 times to get your favorite armor design takes not much development time and keeps your players playing for hours and weeks.
Full answer
Why do Monsters drop things in so many games? It creates all sorts of problems, one problem being for example ingame gold hyper-inflation.
Monster killing giving XP is equally bad
You are questioning the whole game design of Diablo and similar games with those two sentences. However, they are quite successful.
I think the answer is related to what kind of game you are making and what your target audience is. Games like Diablo target an audience that gets "addicted" by finding and collecting stuff to develop their characters. If you want to be one of the top players, you need the best gear and a lot of XP. Each killed monster brings you closer to this goal. This is enough motivation for many players to come back and keep on playing. You don't need to develop a complex quest or storyline for that (who played Diablo for the story/quests? - I didn't :) ). Hyperinflation is no problem in these games since your gear gets useless equally fast until you get closer to the level cap. It is actually a feature of the game. The inflation will slow down the higher you climb on the ladder.
The thing here is, that it is an easy mechanism to keep players playing your game. You don't need much development time to add stronger monsters with better loot. Recolor existing ones and adjust some numbers.
On the other hand, developing and integrating smart and interesting quests/storylines is much harder. You must keep track of all possible outcomes and the effects on your game world. --> What happens to the story and other quests if character XYZ dies during quest ABC? What happens if the player does something that wasn't expected - how to prevent him from being stuck in the story? How do we connect all storylines to make sense from a global point of view?
I am thinking about giving reward for fulfilling missions but not for each and any monster. Like that it might be equally good to avoid detection. Also, Monsters keep their ingame value longer if player figures can't level up so much.
Isn't it enough reward if a monster stops killing you?
Just think about how long do you play a purely story-based game? Resident Evil for example. Monsters don't drop anything there (except for some key items), stay equally strong and you have to follow the story to finish the game. I played the last one for about 40 hours (have to check my steam account to get the exact number). Now how long is the average time players spend in those grinding games? I can't give you any numbers here, but I think it is much higher. Also, the chances of players returning after years to replay a game are much higher for those grinders. Don't know how often I restarted playing Diablo 2 or Path of Exile.
The point here is, that you can generate much more play-time with the same development effort by just letting the player grind monsters for loot or XP than with adding quests. Additionally, the whole grinding loot/XP stuff tends to let your players compete for the best gear/character which lets them play even more.
I have to admit, that I enjoy playing story driven games more than games that focus on loot/XP driven character development. However, once I finished the story and explored the game world, I rarely keep on (re-)playing these games. On the other side, I kept on playing some grinders for weeks and started replaying them year over year.
"You know... I just needed that super cool looking level 90 armor but I had to kill 10.000 goblins to get it... took me 2 weeks... was totally worth it!"
game-design
questions to help make a focused question that draws useful, constructive answers. \$\endgroup\$