It really depends on your game, but three considerations you need to keep in mind are:
- Does the positioning in the inventory have any game-mechanical effect? When it does, you need to handle the inventory positions server-sided.
- When the player organizes their inventory how they see fit, logs out and back in again, their inventory will be a mess again. You could prevent this by storing the order on the client. But what if the player regularly accesses their account from different machines?
- Is there a limitation of inventory slots? When there is, item stack splitting and merging becomes a game mechanic, so you won't get around handling it server-sided.
However, when the player does a drag&drop, I would not recommend you to remove the item when they start dragging and put it back when they stop. There is a high risk that when a bug occurs while the user is dragging that the dragged item could get lost. There are also mechanical problems which might get hard to solve. What would happen, for example, when the player starts dragging an item, receives a new item while dragging which occupies the slot the item came from and then drops the first item at an invalid location? You can't revert it, because that would overwrite the new item.
Instead, wait for the player to drop the item, check where they dropped it, check if it's valid and then perform the swap-operation. While the user is dragging, the item still occupies the same place.