"The particles 'pop up' in front of the player when they're moving."
Your calculations aren't generating points outside the view frustum, and it seems they're going to be re-generated until they happen to be placed either inside the frustum (getting you the 'pop up' issue), or right outside the frustum, in the direction of travel.
I'm assuming throughout this that you're allowing the player to move arbitrarily in all 3 dimensions.
To fix this, you need two changes:
Generate your particles farther ahead. Without changing that you're generating in a uniform cube, a hack solution would mean changing the final (s * direction.x) into something like (s * 5 * direction.x), where 5 is some number big enough to place things outside the frustum. Ideally you would calculate it based on the frustum.
Points generated outside the frustum need to be left alone until they appear on-screen, or they're behind the frustum, too old, or what-have-you. Lots of ways to do this. In your case, perhaps change the initial check to also verify the particle has appeared and isn't too old, and set those variables right after resetting the point. (Below, I use a max age of 5 seconds, which may be too much or too little, depending on player velocity.)
"If the player backs up and moves backwards, the particles won't reset, leaving the cloud visible popping the illusion."
I'm assuming here that you're describing the player facing one way, but moving the opposite direction. (If this is wrong, let me know. :P)
Without changing the meaning of "direction" anywhere else, you could fix this by passing the function a fourth parameter which is essentially a "facing" multiplier that's 1 when moving forwards, -1 when moving backwards. Then (s * 5 * direction.x) becomes (s * 5 * direction.x * facing).
This should change particles so they generate behind the player instead.
"Particles appear to be flying through the player."
I'll take this literally and assume you're saying you don't want particles passing through your player's model. To fix this, you need to prevent particles from being placed in column your player's model will sweep as it moves forward.
You could maybe change the initial point generation to enforce this from the get-go, but I think in this case I'd recommend just re-generating any points that don't satisfy that constraint.
Given your direction vector and player's position, you can calculate two points (position, and position+direction) and build off of the javascript code in this answer.
Removing the two lines if (t < 0) ... and if (t > 1) ... - should yield the distance from some point to some arbitrary infinite line defined by those two points.
Armed with this, you can check each generated point to ensure its result is far enough away from center to not hit the player's model. If it's too close, just generate it again. Unless your cube cross-section is similar in size to the sweep cross-section, this shouldn't usually take more than a couple of attempts.
I'm assuming the model is centered on the player's position and roughly symmetric. So then we want to calculate half the longest dimension, perpendicular to the model's "forward" direction. So for a humanoid model, this would be height. For a typical airplane it would be half the wingspan, etc. In the code below I'm calling this modelWidth and just hard-coding it, but you should get this value in whatever way fits your game.
Lastly, here's the code. Deliciously untested!
ParticleField.prototype.updateCloud = function(frustum, position, direction, facing) {
var s = this.size, s2 = s * 2;
var modelWidth = 11; // arbitrary magic number representing width of your model
var d = new Date();
for (var i = 0; i < this.vertices.length; i++) {
if (!frustum.containsPoint(this.vertices[i])) {
// you'll need to initialize the variables elsewhere of course
if(this.vertices[i].dieAfter <= d.getTime() || this.vertices[i].hasAppeared)
{
do {
this.vertices[i].set(
(position.x + s - Math.floor(Math.random() * s2)) + (s * 5 * facing * direction.x),
(position.y + s - Math.floor(Math.random() * s2)) + (s * 5 * facing * direction.y),
(position.z + s - Math.floor(Math.random() * s2)) +
((s + Math.floor(Math.random() * s2)) * 5 * facing * direction.z)
// I'm unsure why z has this extra factor in it. :P
);
} while( distToSegment(this.vertices[i], position, position+direction) <= modelWidth);
// may want to cap how many times this can run
this.vertices[i].dieAfter = d.getTime() + 5000; // 5s in milliseconds
this.vertices[i].hasAppeared = false;
}
}
else
{
this.vertices[i].hasAppeared = true;
}
}
};
...
// swiped from https://stackoverflow.com/questions/849211/shortest-distance-between-a-point-and-a-line-segment
function sqr(x) { return x * x }
function dist2(v, w) { return sqr(v.x - w.x) + sqr(v.y - w.y) }
function distToSegmentSquared(p, v, w) {
var l2 = dist2(v, w);
if (l2 == 0) return dist2(p, v);
var t = ((p.x - v.x) * (w.x - v.x) + (p.y - v.y) * (w.y - v.y)) / l2;
return dist2(p, { x: v.x + t * (w.x - v.x),
y: v.y + t * (w.y - v.y) });
}
function distToSegment(p, v, w) { return Math.sqrt(distToSegmentSquared(p, v, w)); }
