The raycasting is a preprocess. It has nothing to do with GLSL.
You raycast to find ambient occlusion, really. Where a ray does not strike any geometry, it is "open to ambient lighting" from that direction.
What Green is trying to do on page 26 is estimate the ambient occlusion function in spherical harmonics. To do so he first generates a "sphere of rays" in every which direction (page 7). He evaluates the SH function on each (l,m) band and saves that in a data structure coupled with the ray direction.
Want: To estimate the ambient occlusion function at VERTEX in spherical harmonics
To estimate the ambient occlusion function at VERTEX in SH, you cast all the rays in the "sphere of rays collection". Call an example ray RAY
. Where RAY
DOESN'T strike anything, it should be used to estimate the ambient occlusion function at VERTEX. So, you add a scaled copy of the SH coefficients of RAY to the SH approx to of the ambient occlusion function at VERTEX. The scaled of the SH coefficients of RAY is the dot product of RAY with the VERTEX's normal.