If you would use vertex indices, you could just skip those indices, but you would have to almost completely replace the indices list, which I guess would defeat the purpose of this question.
If you only want to have the visual effect of removing a polygon you can flip the order of vertices and use face culling. Or change the color of the polygon to clear(few ways to do this, but generally hard to do with UV mapping).
Because the data is linearly mapped to memory you can't really just remove data without rewriting your code in a more complicated manner. If you really desire to do this you can.
What I suggest you do is to use GLMultiDrawArrays or GLMultiDrawElements which would allow you to have all the objects in your scene in a single VAO, but you get to specify which you want to draw. The downside is that they all have to use the same state for drawing (rarely an actual problem). The upside is that you use code with minimal driver overhead. Now if you want to add and remove data from this VAO you have to apply your own memory management code so you wouldn't override data that is still in use. I wouldn't worry about deleting the data because your VAO is still linearly mapped and the memory is inaccessible for the GPU anyways and vertex data is rarely a significant part of used memory.
Now if you want to go hardcore on the optimizations you can use GLMultiDrawArraysIndirect or GLMultidrawElementsIndirect, but this requires OpenGL 4.3 or greater. The added benefit would be even less communication with the GPU, but you have to store some data in GPU memory and also manage that.