First of all, I think you meant to ask whether a 2-bit texture will be 16 times faster than a 32-bit texture, not slower.
The answer is: sometimes. Performance of graphics hardware is a complex topic. It is a concurrent, pipelined system that contains many different processes operating simultaneously and passing data to one another.
In general, the speed of the whole system is controlled by the speed of the slowest component. At different times and under different circumstances, different components may be the bottleneck. For instance, we often render triangles by running a vertex shader followed by a pixel shader. If the vertex shader is slower than the pixel shader, the speed of the whole rendering will be determined by how fast the vertices can be done, and we say the system is "vertex-bound" or "vertex-limited". Or if the pixel shader is slower, we say it's "pixel-bound", etc.
Within shaders we can also look at computation versus memory access. Sometimes a shader may be compute-bound, meaning it has enough math operations to hide the latency of memory reads (texture sampling) and writes (output to a render target). In other cases the shader may be bandwidth-bound, meaning that it needs to wait for memory access to complete, so memory bandwidth (how fast you can read and write memory) is the bottleneck. Within bandwidth-bound cases we can distinguish being bound by input bandwidth (texture bound), or by output bandwidth.
The texture-bound case is the one you're asking about. If a shader is texture-bound, then switching to a texture format twice as wide will make it render approximately half as fast. However, switching to a thinner texture format might not make it go much faster, if it moves the bottleneck somewhere else.
Conversely, if the shader is not texture-bound, switching to a thinner format will have no effect on the overall speed, since the bottleneck is somewhere else. But switching to a wider format might cause the shader to become texture-bound and slow it down.
In conclusion, there's no fixed relationship between texture format and rendering speed. Thinner texture formats will be no slower than wider ones, but they might not be faster either - it depends on details of the shader and the scene.