Checking if I am looking through a window using angles

I am developing a mod for a game with a Lua api.

I am trying to detect if I am looking through from behind a window when I shoot an enemy. For context, when I shoot through this window I apply a special buff effect.

Characters lie on a map which is an xy coordinate system with 4 quadrants. 0, 0 lies at the center of the map.

I am doing a top-down view. I can get the xy position of my character, the xy position of the enemy character, and the 2 xy positions of the boundaries of the window.

I got the angle of the character relative to the angle of the window edges, and I got the angle of the character relative to the angle of the enemy.

here is an example:

player position -149.55355834961 -54.274295806885
target position -147.99998474121 -67.999992370605
left position -151.8394317627 -56.843517303467
right position -147.86442565918 -56.397045135498
angle target 96.457664489746
angle left 48.340045928955
angle right 128.51026916504


this is how I am calculating the angles (the game uses xz as the horizontal plane):

function getAngleBetween(object1, object2)
-- y is height here
local x1, y1, z1 = GetWorldPosition(object1)
local x2, y2, z2 = GetWorldPosition(object2)
local zDiff = z1 - z2
local xDiff = x1 - x2
local phi = math.atan2(zDiff, xDiff)
if phi < 0 then
phi = phi + (2 * math.pi)
end
local degrees = phi * (180/math.pi)
return degrees
end


I used this code to check if the angle with the enemy was between the two sides of the window, but it gives inconsistent results. I think it has to do with how I am calculating the angles, and the coordinate system. For example, If I am facing north, it works. But if I am facing south, it doesn't.

    function isAngleBetweenDeg(leftAngle, rightAngle, checkAngle)

leftAngle = math.mod(leftAngle,360)
rightAngle =  math.mod(rightAngle,360)
checkAngle =  math.mod(checkAngle,360)

if leftAngle > rightAngle then
return checkAngle >= leftAngle or checkAngle <= rightAngle
end
return checkAngle > leftAngle and checkAngle < rightAngle
end


How do I check if I am behind the window? as far as the game's API, you only need to know GetWorldPosition give me the coordinates.

• Do I understand correctly that your window is an axis-aligned rectangle, and the player can see/shoot through it from all sides (even in through one short side and out the opposite short side)? Or should the window be treated as a line segment, say, the line joining the bottom corners? Jan 23 at 13:15
• the window is not necessarily axis aligned (other than always standing vertically). it can have any orientation. It should be a line segment with the bottom corners. The player can walk anywhere around it. Jan 23 at 15:30