Is my only option to calculate the tiles from each chunk and determine where they need to be drawn on screen? It seems like this will be a little nuisance of a problem to deal with.
If that seems hard, then you should change how you do it so that it's easy. It's not clear from your question what makes the case problematic, but I suspect that it's that you're trying to load chunks based on how the player/camera is moving. This is a bad idea, as it's much simpler to do something which works independently of movement.
First, if you don't have it already, you should have a data structure which stores loaded chunks keyed on their chunk coordinates. Example of chunk coordinates: The chunk containing the "middle" of your world is at coordinates (0, 0). Then the chunk immediately to the right of that one is (1, 0), and the one below that one is (1, 1), and so on. Then you can use those coordinates as keys in a hash-table or similar to keep references to the chunks you've loaded.
Every frame, do the following:
Figure out the coordinates of the chunk which should be visible in the upper left corner of the screen. For example, if the upper left corner has screen coordinates (0, 0), and your camera position is expressed in pixels (likely in a sprite-based game), then divide the camera position by the tile size, and the chunk size, to obtain chunk coordinates. On the other hand, if you're using 3D-style graphics and the tile size is just 1, then you just divide by the chunk size.
Do the same for the bottom-right corner of the screen (same computation but offset by the screen size). You now have four chunk coordinates which can be interpreted as the corners of a rectangle of chunks which will be always at least the size of the screen.
Loop over the coordinates of each chunk in the rectangle (just two nested loops). Look it up in the table of loaded chunks; if not found, load/generate it. Draw the chunk, positioning it according to its coordinates and the camera position.
Each chunk which is loaded but was not drawn in the previous step (you can use a "was drawn" flag on each chunk to figure this out) may be unloaded (but you probably want to keep some around as a cache in case the player is moving back and forth near a boundary).
This is a general algorithm for 2D rectangular chunks and a 2D rectangular camera view. It sounds like in your case your chunks are larger than the camera, so you always have at at most four chunks visible; this could simplify things, but I recommend using this algorithm which avoids having any special cases until you're more confident in how to tweak things.
Note that this algorithm does not care how the camera is moving, or when it crosses a boundary. It only makes sure that the current frame is drawn correctly, regardless of what happened before. (I suspect trying to make use of the previous position/movement is part of your original problem.) This also means that it can handle arbitrary movement like teleporting or respawning.
