Can I make a character solid but not lose non-physics behaviour?

My character can walk around in the world, but walks through other objects, for example a cylinder (stand-in for a cannon). To fix this I added a capsule collider to each of the character and cylinder, but that wasn't enough. Apparently I have to add a Rigidbody as well. [1] [2]

When I add a Rigidbody to either one, they collide as expected. However, when I add a Rigidbody to my character, it falls over when I move forward. When I add a Rigidbody to my cylinder, other functionality fails (it is supposed to shoot cannonballs, and they act strangely). If I enable Kinematics on either, they stop colliding.

Is there any other way to make the character and cylinder solid with respect to each other, without breaking existing non-physics functionality?

You can freeze rotations of a Rigidbody.