You have to consider that GPUs when fetching the data for feeding into the vertex shaders requires a lot of small calls because they can only cache a few triangles at the same time, and fetching data from the VRAM is a very slow operation. So I suppose that if you use half the bytes for the indices, the GPU will be able to fetch twice the indices for the cache to render triangles.
4MB Can be a small amount of memory for storage space, but if you have to make a complex call to only process a few bytes from them it may affect performance in a noticeable way. Nowadays video cards are powerful enough to make it irrelevant in games which don't have to process a lot of geometry, you... could do some benchmarks rendering a model with both index sizes if you really want to know.
Loading times are also faster with 16 bit indices, and actually every write or read, unless you can effectively control memory alignment and proper paging. So it's not just about the memory it requires but how much it takes to handle it. If you are working with dynamic buffers that need to be moved over the bus to the GPU at some point between Frames, then you should use 16 bit indices when possible. If you are rendering even smaller geometries, you can batch many indices in a single 16 bit index buffer for them all to get copied to the GPU in a single call.
Here you may find some information about how the newer ATI GPUs work:
ATI Stream Developer Training Resources
(sorry I couldn't find the exact document)
I didn't read much about the NVidia ones, but I think they work more or less with a similar approach.