Non sequitur, your question does not entirely make sense. You cannot store infinite chunks. You can only ever store the subset that is currently of interest, i.e. those around the player(s).
So how does the infinite part work? You generate chunks according to some global function. For example, Minecraft's use of Perlin noise (a global function) means that any given
[x,y,z] location can be generated deterministically; changes which were applied at that voxel location are then applied following each deterministic regeneration of that location (more typically, of its parent chunk).
Now we get to the meat of your question -- the Area of Interest (AoI) around the player. Storing these can be done in many different ways, some of which you touch on. Let's look at those.
What is the difference between shifting a 2D array (approach 3) and using a dictionary? A 2D array is used when each of a bunch of closely-packed
[x,z] locations are expected to be filled with valid chunks at all times, representing the current AoI; this is a dense collection. Think about that; it may well be all you need (I believe this is how MC works). OTOH, a dictionary is used when you need a sparse collection, i.e. data in memory is contiguous, but your chunks in world space are not, that is they are at non-adjacent chunk coordinates.
As for using chunk values (structs) vs references (pointers) to chunks that are stored elsewhere, this is typically a matter of flexibility. Using an array of values is faster for reading than using an array of pointers, because in the second case you have to not only index into the array, but you (or rather the Mono runtime) then still have to JMP to another location in order to get the chunk data; cost thereof may or may not be negligible depending on how well or poorly you have packed your data. Using an array / dictionary of pointers can be better for writing, because all you have to do then is change the pointer to another location, bearing in mind that you then pay a read-performance penalty. The other aspect to using references (for C#, nullable types) is that you can have null chunk references if you require them.
Your approach 1 is probably not very efficient because besides having to do lookups in two dictionaries (why?) you also have to do multiple jumps to get to the chunk data.