Lets imagine the simple situation of a ship with just one engine. In order to move the ship, we simply point the thruster away from the direction we want to go, and thrust out. That pushes us in the direction we want to go. It's not possible to rotate a ship composed of only one thruster.
Lets scale our ship to three engines to make it more interesting:
If we wanted to move this ship somewhere up and to the left, we'd just point all our thrusters in the opposite direction and fire:
Same solution as the single thruster that takes care of linear movement.
But what about if we're missing a thruster? We have an unequal load?
Following the same strategy will cause the ship to rotate. In many situations, this will be mostly unavoidable, and in some cases preferable since it will orient the ship for better use of the available thrusters.
One strategy for mitigating this would be equalization. Split the ship down the axis of desired movement:
Then ensure that the right side and the left side equal each other. If they don't scale the thruster output until they do. This can easily result in no thrust being applied. It's up to you to decide if you want to instead thrust out causing rotation or force the player to redesign their ship by re-organizing their thrusters.
How about rotation?
This question describes the idea very well.
You'll need to calculate the center of mass. Then calculate the torque provided by each thruster. Lucky for you, your thrusters rotate, so you can rotate them to provide maximum torque. Maximum torque will come from a right angle to the moment arm. Applying the sum of the torques will rotate the body around the center of mass. You can also choose some other arbitrary rotation point like the bridge of the ship.
If you want to ensure that you rotate in place you can scale back the thrust of the thrusters that will cause your ship to move linearly. You can do that by adding up all the thrusters that have a positive torque and all those with a negative torque, then compare their absolute values. The group with the larger absolute value should be scaled back to match the absolute value of the other group.
You may also find a very helpful resource in Amit Patel.
He's done a 4 part series on configurable space ships here: part 1, part 2, part 3, part 4