Take into account that Applying a force is not the same as changing the velocity of the object.
Without friction, the result would be very similar, but when an object is standing in a surface, and there is friction between them, the behaviour could be very different.
In the videos you uploaded, the cube is rolling because you are pushing it from a side, just like a regular cube would roll if the friction was strong enough.
If you want the cube to glide as you say, you have to remove this friction. To do so you must assing a new Phycics material to your cube, and remove friction from it.
- Create a Physics material
- Select it, and in the inspector, remove friction:
- Assign the new material to the Box Collider
Also, I would highly recommend update Physics behaviour from FixedUpdate()