The length units, for 3D graphics, are whatever. As long as they are internally consistent, it does not matter. For graphics.
For physics units matter.
However, you do not set the units. There is nothing different in game engine or its settings. In fact, the units for most engine only exist in documentation and in the mind of people. Except time, because time is time.
Your game will have real units of time. You need to make your physics engine be consistent with that. In your engine, I suppose you would measure the elapsed time between frames, your delta
time is some units. What units is that?
Figure out what is your unit for time. Then pick a unit for length. You can write it down in comments, or in some means of communication to other developers, or in a piece of paper. You are going to express everything with the units you picked.
For example, if you are trying to mimic earth gravity, and you set your gravitational acceleration to 9.8. What units is that? Well, that is meters over seconds squared. This means you are working with meters and seconds. You would have 980, if you were working with centimeters and seconds. 9800 for millimeters and seconds (also nanometers and milliseconds). 0.0000098 is meters and milliseconds. 0.0098 is millimeters and milliseconds. I'll reluctantly mention 32.15 is feet and second. And so on.
Remember that time is time. So getting speed and acceleration right, is important.
Of course, it makes sense to create models with whatever scale you are using for physics. What do I mean by that? Well, if you make a human, how many units tall should it be? Ah, units are whatever, but humans are around 1.6 meters tall. So, if you are working with meters, you make your human model 1.6 units. If you are working with centimeters, then it is 160 units. And so on. The graphics don't care, but remember that the the physics do.
If you create your models with the physics units, that will save you scaling the models. However, if you have to scale the models, that is not the end of the world. You would be using matrix transformations to place the models in the world anyway, it cost you nothing, nothing, to have those matrix transformations also scale the models appropriately.