"Do not optimise early" doesn't mean "pick the worst possible way to do things". You still need to consider performance implications (unless you're just prototyping). The point is not to cripple other, more important things at that point in development - like flexibility, reliability etc. Pick simple, safe optimisations - choose the things you limit, and the things you keep free; keep track of the costs. Should you use strong-typing? Most games did and work fine; how much would it cost you to remove that if you found interesting uses of the flexibility for gamemplay?
It's much harder to modify optimised code, especially "smart" code. It's always a choice that makes some things better, and others worse (for example, you might be trading CPU time for memory usage). When making that choice, you need to be aware of all the implications - they might be disastrous, but they can also be helpful.
For example, Commander Keen, Wolfenstein and Doom were each built on top of an optimized rendering engine. Each had their "trick" that enabled the game to exist in the first place (each also had further optimizations developed over time, but that's not important here). That's fine. It's okay to heavily optimize the very core of the game, the think that makes the game possible; especially if you're exploring new territory where this particular optimized feature allows you to consider game designs that weren't much explored. The limitations the optimization introduces may give you interesting gameplay as well (e.g. unit count limits in RTS games may have started as a way to improve performance, but they have a gameplay effect as well).
But do note that in each of these examples, the game couldn't exist without the optimization. They didn't start with a "fully optimized" engine - they started with the bare necessity, and worked their way up. They were developing new technologies, and using them to make fun games. And the engine tricks were limited to as small part of the codebase as possible - the heavier optimizations were only introduced when the gameplay was mostly done, or where it allowed an interesting new feature to emerge.
Now consider a game you might want to make. Is there really some technological miracle that makes or breaks that game? Maybe you're envisioning an open-world game on an infinite world. Is that really the central piece of the game? Would the game simply not work without it? Maybe you're thinking about a game where the terrain is deformable without limit, with realistic geology and such; can you make it work with a smaller scope? Would it work in 2D instead of 3D? Get something fun as soon as possible - even if optimizations might require you to rework a huge chunk of your existing code, it may be worth it; and you might even realize that making things bigger doesn't really make the game better.
As an example of a recent game with lots of optimisations, I'd point to Factorio. One critical part of the game are the belts - there are many thousands of them, and they carry many individual bits of materials all around your factory. Did the game started with a heavily-optimised belt engine? No! In fact, the original belt design was almost impossible to optimise - it kind of did a physical simulation of the items on the belt, which created some interesting things you could do (this is the way you get "emergent" gameplay - gameplay that surprises the designer), but meant you had to simulate every single item on the belt. With thousands of belts, you get tens of thousands of physically-simulated items - even just removing that and letting the belts do the work allows you to cut the associated CPU time by 95-99%, even without considering things like memory locality. But it's only useful to do that when you actually reach those limits.
Pretty much everything that had anything to do with belts had to be remade to allow the belts to be optimised. And the belts needed to be optimised, because you needed a lot of belts for a large factory, and large factories are one attraction of the game. After all, if you can't have large factories, why have an infinite world? Funny you should ask - early versions didn't :) The game was reworked and reshaped all over many times to get where they are now - including a 100% ground-up remake when they realized Java isn't the way to go for a game like this and switched to C++. And it worked great for Factorio (though it was still a good thing it wasn't optimised from the get-go - especially since this was a hobby project, which might have simply failed otherwise for lack of interest).
But the thing is, there are lots of things you can do with a limited-scope factory - and many games have shown just that. Limits can be even more empowering for fun than freedoms; would Spacechem be more fun if the "maps" were infinite? If you started with heavily optimised "belts", you would pretty much be forced to go that way; and you couldn't explore other design directions (like seeing what interesting things you can do with physics-simulated conveyor belts). You're limiting your potential design-space. It may not seem like that because you don't see a lot of unfinished games, but the hard part is getting the fun right - for every fun game you see, there's probably hundreds that just couldn't get there and were scrapped (or worse, released as horrible messes). If optimisation helps you do that - go ahead. If it doesn't... it's likely premature. If you think some gameplay mechanic works great, but needs optimisations to truly shine - go ahead. If you don't have interesting mechanics, don't optimise them. Find the fun first - you will find most optimisations don't help with that, and are often detriminal.
Finally, you have a great, fun game. Does it make sense to optimise now? Ha! It's still not as clear as you might think. Is there something fun you can do instead? Don't forget your time is still limited. Everything takes an effort, and you want to focus that effort on where it matters most. Yes, even if you're making a "free game", or an "open source" game. Watch how the game is played; notice where the performance becomes a bottleneck. Does optimising those places make for more fun (like building ever bigger, ever more tangled factories)? Does it allow you to attract more players (e.g. with weaker computers, or on different platforms)? You always need to prioritise - look for the effort to yield ratio. You'll likely find plenty of low-hanging fruit just from playing your game and watching others play the game. But note the important part - to get there, you need a game. Focus on that.
As a cherry on top, consider that optimisation never ends. It's not a task with a little check mark that you finish and move on to other tasks. There's always "one more optimisation" you can do, and a big part of any development is understanding the priorities. You don't do optimisation for optimisation's sake - you do it to achieve a particular goal (e.g. "200 units on the screen at once on a 333 MHz Pentium" is a great goal). Don't lose track of the terminal goal just because you focus too much on the intermediate goals that might not even be pre-requisites for the terminal goal anymore.