This is a non-trivial problem. See http://www.reddit.com/r/gamedev/comments/1w92dm/2d_collision_detection_and_resolution_solving_the/, https://www.iforce2d.net/b2dtut/ghost-vertices .

If all your blocks are axis-aligned, the easiest solution would be to prioritize collisions with normals along the y-axis.

One common solution is to use "rounded" corners for your character box. Another could be to use line segments instead of blocks (if the blocks are supposed to be static) - google "box2d chain-shapes" .