Should I delete the textures I use in OpenGL ES 2 (Android)?

Say I have a skybox and the user can choose out of 2 sets of 6 textures (one for each face of the cube) to use on that skybox. For example if user selects set A, I load the textures (GLES20.glGenTextures etc) and put them on the cube.

However, if the user then selects set B, should I delete the textures I previously created? Or what's the standard way of dealing with textures that are (at least temporarily) no longer needed? Also, is texture deletion an expensive operation?

My app will include other scenarios similar to the above, so I might end up having 10-15 (or more) textures in memory with only a few of them displayed, therefore I expect deleting them is the way to go.