I was wondering why today programmers place so much emphasis on API like DirectX, and even OpenGl. As far as I know, you'd need thousands of lines of code to make a simple 3D game using one of the aforementioned. But with game engines like Unity, the process is simplified and you can pretty much drag around shapes to create the graphics in lieu of programming it. So what I wanted to know: why is knowing APIs DirectX more preferable than being knowledgeable in Unity, other than the fact that with an API you can code it to be portable on other systems. To me it's like choosing Assembly over a high level language like C. Sure, with assembly I'll really understand what's happening at machine level, but it's take a lot longer to write programs using it and it would be a lot harder to debug.
Using your own analogy: Why did people make COBOL? Why did people make C++ if C was around? Why did they make PHP? Why did they make Python? Ignoring the fact that some of those implementations are written in C themselves, this is the same thing.
Some people want to make their own languages for new purposes and to do that you need to start with the graphics APIs. Other people want to learn how they work -- similar to implementing your own C compiler.
In some cases it's a personal case of NIH: "My engine will be better than Unity's!". In other it's appropriateness -- building a 2d game like Monkey Island in Unity3D is probably more difficult than rolling your own simple engine. And finally you need to remember that Unity is a very large API to build upon with a lot of functionality. It's a lot of effort to learn it all, and some people feel that effort is better spent making their own simpler engine.
I can speak only for my self, but I implement my engine for sake of implementing it. It is fun for me. Plus the time spent learning an engine is roughly equal to implementing it yourself.
It also gives you absolute control over the engine and you can specialize it for your task more easily (you would have to read pretty much everything about an engine, read and understand all code to be able alter it).
Sometimes for compactness a self-made engine ( Despite thousands of lines of code ) can be more compact than a pre-made engine due to the fact that unused features can be removed.
Also keep in mind that if you write your own engine from scratch, you'll know all the ins and outs of it. If you ever want to add a feature or change something, you can!
Some engines can also be very restrictive of what you can do with them or even if you can publish it for money.
The best part about creating your own engine is the knowledge that you can look back on a game and say made it 100% yourself.
What is the point in designing your own car (e.g. to take part in an F1 race) if there are existing car manufacturers who will sell you cars that already work?
I looked over some other answers here and in the proposed dupe questions and one point not really being championed is that creating your own system affords you more technical (and, as a consequence, creative) control.
The car analogy falls well short because cars are all hardware plus the user, here we're talking about middleware software. Which is just a part of the software. Which usually has to accommodate combinatorial variations of hardware, as well as completely unskilled users.
Anyway, the point is that ABC game engine, no matter how attractive its feature set, was not designed with you in mind. For example, an Unreal engine was designed with an Unreal Tournament game in mind. Unless it is open source, usually you will have no idea how it works.
Of course that may be just fine and dandy, depends on what you're setting out to do.
BTW this is not any sort of a jab at the Unreal engine (the latest iteration of which these days can rightfully be said to be unreal. It's freaking beautiful). Modern engines like Unreal and Unity (and many others) are making huge leaps and bounds toward supporting anything you could realistically expect to build.
The fundamental idea for my answer remains the same and I expect this to be true for a while until paradigms start to shift in game development and computer programming. At the end of the day, writing your own game engine allows you to (in a simple straightforward way) generate machine code which is is not littered with tons of dynamic branches to support dynamic, general capabilities. Call it bloat or call it a feature, this is just how it works.
I envision software middleware in the distant future that has deep compiler integration and we will continue to be able to enhance software optimization capabilities while retaining ease of use for content artists and developers.