You mention that the user feels "safe" if the spiders are all offscreen. This is probably the most important thing to address. Fortunately, you want the right kind of thing: a gut feeling that the user has that nothing is ever safe.
The "gut feeling" people get is an amalgamation of many small cues that the brain processes together to try to guestimate an unknown (in this case, the position of the spiders). Your goal should be to provide many small clues in many different ways, visual and auditory, rather than trying to come up with one big clue which tells you where the spider is.
Many have mentioned music. Sound is always good for gut feeling effects because we humans use it for wide awareness of our surroundings. It's the thing which tells us to stop focusing on trying to fix our radio because there's a zombie behind us. However, visual effects can also be used. You just need to make the spiders have a presence. A few ideas:
- Have lots of small animals in the background which start to flee as the spiders get closer.
- Play games with color saturation when the spiders are near. Maybe when they are far away, everything is fully saturated, but when they are getting near, the non-essential parts of the background start to lose saturation, like the color draining from someone's cheeks.
- Cues from the squirrel. The player controls their movement, but they don't control any animated twitches the squirrel has. Perhaps the squirrel could show panic as the spiders get close.
- Music cues, obviously
- Sounds associated with the spider, getting louder.
How you present these effects as the spiders close in is more important than anything else. The gut feeling parts of the brain work almost entirely on logarithmic scales. Thus we get the same amount of relief from the spider moving from 1 to 2 units away as we get from the spider moving from 2 to 4 or from 10 to 20. Cues which work to this scale will be more well received than anything else.
Take the spider sounds, for instance. Instead of just having one spider sound which gets louder as they get closer, consider having multiple, which all come in at different rates. Perhaps have the music start to change when the spiders are 8 feet away. Around 4 feet away, you start fading in the sound of their scratching legs against the trees. When you get to 2 feet away (which may be right off screen), you could start hearing the noises from them mashing their mouth parts hungrily (why not... they're evil spiders!).
You don't always have to use powers of 2, though they are obviously natural for game developers. Maybe the small animals flee when the spider is at 30 feet away, and the squirrel starts looking visibly nervous when they get 10 feet away. Then, at 3.3 feet, color drains from the background.
Of course, for all of these, its best to fade the effects in gradually. That way, for any spider distance, the player's gut has many smoothly varying signals with which to develop its gut feeling for where the spiders are. The more small signals the player is paying attention to, the more they will be drawn into the moment. This is the real key. A small number of obvious cues will have less effect than a large multitude of small cues which all operate on different scales, and ideally you want to let the user construct a reasonably logarithmic worldview using those cues.