Start by identifying what you actually need out of the "wrapper" part of the API. It's generally very, very simple: you need the basic resources (buffers, shaders, textures, pipeline state) and a way to use those resources to construct a frame by submitting some draw calls.
Try to keep any high-level logic out of the wrapper portion of the API. If you implement a clever scene culling technique in this portion of the API, well now you are on the hook to duplicate that logic in all of the backend implementations. That's a lot of extra effort, so keep it simple. Scene management should be part of a higher-level portion of the API that uses the wrapper rather than being part of the wrapper itself.
Choose the targets you will support and understand them. It's hard to write decent wrappers for "everything," and you probably don't need to (arguably you don't need to write a single wrapper, either, as noted in Philipp's answer). It's almost impossible to write a decent wrapper if you don't know the APIs you're going to wrap already.
Evaluate the state of your API regularly. It should in general have a smaller surface area than the underlying wrapped APIs; if you find yourself creating one-to-one wrapper types for every D3D structure or every OpenGL function call you are probably veering off course.
Look at what work has gone before. Sokol and BGFX are APIs that provide levels of agnosticism that may be useful to you, and are relatively easy to understand (the former especially).