I've read this document about HLSL registers, but it's more of a syntax description and not a purpose explanation.
What are registers in HLSL, actually? How and when do I use them? What happens if I don't (what's the default)?
A given shader model exposes a particular set of registers to HLSL; these registers are underlying hardware registers on the GPU, like CPU registers, but have more refined scopes (for example, there are registers dedicated to holding samplers).
Registers are where all your data is stored during the execution of your shader (with the exception of data, like texture or vertex buffer data, that is held in the GPU's RAM). On a GPU, you don't have the same flexibility in reading and manipulating memory as you do on a CPU. There's no "store this value to RAM for a moment," so everything has to be in registers. This is particular obvious in old shader assembly (from the shader model 1 era).
Data transfer can occur via these registers as well, which is why the keywords still exist in more modern shader models. If you're not using the effects framework, for example, you can manage the allocation of your sampler or constant registers yourself (for example via methods like SetVertexShaderConstant). Similarly, data from buffers can be mapped to registers; the vertex input registers typically get your vertex buffer data mapped to them prior to execution of your shader.
You can find a list of registers and their purpose on the MSDN. Here, for example, is the set for VS 5.0.
Often, you don't need to use them directly, much like CPU registers. They will be used automatically by the higher-level language constructs that you use in your shader code or in the API (such as the aforementioned effects framework).
The specific registers you deal with syntactically will be some form of I/O register. Josh explained excellently what registers are and why they're used on the GPU. The syntax you use to declare variables in specific registers is for transferring data. On the CPU you create an Input Layout (Vertex Array Object in GL) or set uniforms or samplers/textures. This maps values in your vertex buffers to specific registers (via semantics for vertex data). The C++ code and HLSL code must agree on which data will be in which register for input as well output for MRT and the like.
If you don't specify registers explicitly, the HLSL picks them for you. You should not rely on how it picks them; even if it seems stable it may change in a future version. You can use the shader introspection APIs to determine which registers the compiler picked so you can setup your input layouts appropriately. This is less efficient and more difficult than just declaring the registers in your shader code.
Essentially, the C++ says something like "the camera data is in constant buffer register 3." The shader then must read that buffer from register 3 or it won't work correctly. You can do it by name like old GL forced you to, but you shouldn't.
This is all the same for recent GL/GLSL too, though obviously with differing syntax and APIs.
Registers are a type of storage that is located on a processor, and accessible very rapidly. They are typically substantially smaller, and are used as input and output locations for processor instructions. HLSL provides a method for mapping up data to these registers from the CPU side, to be used within your shader on the GPU. They should be used whenever you have data that needs to be referenced within a GPU, but is only available at run time on the CPU(such as world view projection matrices), and thus must be copied to a GPU register. Here is an article describing registers in a non-hlsl context http://en.wikipedia.org/wiki/Processor_register