For arbitrary shapes
The basic algorithm goes like this:
// Gets the distance from a point to a shape made of line segments.
float GetDistance(Shape shape, Vector2 point):
min_dist = float.max;
foreach LineSegment line in shape:
dist = GetDistance(line, point);
min_dist = min(min_dist, dist);
return min_dist;
// Gets the distance from a line segment to a point
float GetDistance(LineSegment, Vector2 point):
...
The distance from a line segment to a point is also pretty easy to find.
For axis aligned rectangles
Axis aligned rectanges (ie, non-rotated) have the nice property that they're already rotated into the same coordinate frame as the point. This makes the problem a lot easier. Suppose the rectangle has the parameterization:
[min_x, min_y, max_x, max_y]
You can clamp to the nearest point in the rectangle, and find the distance to that point. The nearest point on the rectangle is given by:
nearest = [point.x, point.y];
if (point.x < min_x)
nearest.x = min_x;
else if(point.x > max_x)
nearest.x = max_x;
if(point.y < min_y)
nearest.y = min_y;
else if(point.y > max_x)
nearest.y = max_y;
And the distance is given by:
(nearest - point).Length();