bullet is referring to the bullet prefab you're copying to make your new instance to fire. (I like naming my prefab variables things like
bulletPrefab to make this extra-obvious)
So you're assigning its velocity based on the prefab's forward direction, not on the direction the gun is pointing. If you happened to save your prefab pointing straight up, then that's where every bullet will go, regardless of the gun's facing direction.
You probably want something more like:
Vector3 velocity = shootPoint.transform.forward * firingSpeed;
shootPoint transform is oriented with its local z+ axis pointing in the direction you want to shoot.
The other thing you'll want to watch for is if both the player/gun and the bullet have colliders, and they overlap when the bullet is spawned at
shootPoint. The physics engine might see this and try to apply an impulse to separate the overlapping colliders, knocking your bullets off-course. To fix this, you can...
Use physics layers so the player does not interact with their own bullet colliders
Use IgnoreCollision to selectively ignore collisions between the gun and its bullets, as you fire each bullet
Make the bullet collider a trigger so it doesn't participate in collision resolution (but can still detect collisions with