This can be done with masking. There are multiple ways to do it.
One way would be to have a way to render a shadow version of foreground elements. Draw the background. Then draw the foreground elements, offset slightly depending on how you want lighting to move, in such a way that you only output black colors. Then draw the foreground normally.
Another way would be to render the foreground into a buffer, a little bit more than you'd actually see. Then do some "to black" pass and maybe a blur pass. Now render the output of this set of passes and then the visible section of the foreground buffer.
To support multiple lights at different angles, you can just do the previous step multiple times, or accumulate them all in a single buffer. The math will vary based on what you're trying to achieve, but a first guess at what's going on in that first video is that they just take the position of the light against the position of the camera (center of the visible area) and then use that vector (scaled) to determine how to offset the shadows. Something like:
draw background
for each light intersecting viewing bounds:
offset = light.position - camera.position
offset *= SCALE FACTOR (0.25 maybe)
draw shadow foreground at offset
draw foreground
Lights that are "closer" to the scene will have a smaller scale factor. If you want multiple depths, there's some math to work out there, simple geometric stuff.