A few friends and I are working on a home-brewed engine for a selection of games we wish to produce in the future. We're making it to satisfy a few key requirements that we haven't found were properly handled in professional game engines. But we were wondering, how can we test how effective our engine is at solving the problems we set out to solve?
For example, our games require a very specific internal OOP architecture, and the engine facilitates that, making it easy for us to do what we want. How are we supposed to make sure the engine actually helps us?
The reason I ask is because I've written unit tests before, but they've usually been to test whether or not something is correct. Does the engine properly determine collisions between convex polygons? Does the event propagator properly respond to user input? Is the implementation of Bresenham's line algorithm for rendering line primitives correct? Things like this.
When it comes to testing effectiveness of the application as a whole, however, there is no notion of right or wrong. It's fuzzy, it could be pretty effective, very effective, or not effective at all. Is the only solution just to make a game using the engine, and determine the engine's effectiveness from there? I want to say no, because the point of testing is to catch errors early on right, so if we make a game to test effectiveness, we have to build the entire engine beforehand.
How would tests like these even be included? If we integrated "test games" using the same program we use to integrate unit tests, they'd get run every time the engine builds. But we don't want our stupid test games to start running every time we build.
So is there another solution?