# Recognize hand drawn shape

I want to ask player to replicate a simple shape (triangle, square, star, anything that can be done in single stroke) with mouse or on a touchscreen.

Orientation matters, size does not. Example:

Can anyone give me directions to what techniques can be used?

• Are you going to scan a piece of paper with drawings on them or is it direct input with a tablet or something? Because with direct input, you could create vertices where the player changes direction, and use those Commented Mar 25, 2016 at 20:27
• It will be a mouse(possibly touch) driven input. Commented Mar 26, 2016 at 11:14

You will need to think of the shapes in vector form. So for your triangle, the shape might be represented as the series of vectors: (-0.5, -0.5), (1, 0), (-0.5, 0.5). because you know the shape and orientation, you are basically matching the user's finger movement to the expected shape.

The way you determine the scale of the image is by logging the user's initial touch point and then, every update, the point where their finger is, up until they reach the end of the first line (more on this later). The length of the vector from where the touch started to where the first corner is, is the scale of the image you are expecting them to draw. you multiply your vector representation of the expected shape by this scale to get the correct shape which they should draw.

To determine the first corner, you should probably keep track of the angle of the vector from the initial tap point to the current tap location. Once this angle differs from the expected angle by some threshold, you assume the user changed drawing direction.

Since the point where we detect the change in direction (lower blue cross on the image above) is not the actual corner is, instead it's where the threshold angle was crossed, we need to still determine the actual point to determine the correct scale. The way this is done is by looking at the first two vectors of our shape - in our case (-0.5, -0.5) and (1, 0). We adjust them so that the first goes through the tap start position and so that the second goes through the point where the threshold angle is crossed. We are left with (-0.5 + x1, -0.5 + y1) and (1 + x2, 0 + y2), where x1, y1, x2 and y2 are the offsets needed to ensure the lines go through the specified points. Then we find the intersection of these two lines. This is the point where the X value of both lines will be the same and the Y value of both lines will be the same.

To calculate the drawing quality, you will need to sample the user's path in equal distance intervals.

At each interval, you will find the closest point on the correct shape. The distance is the error.

At the end you will also need to normalize this error based on the drawn distance versus the correct distance. This is so that drawing a longer shape does not increase your error and so drawing a shorter one does not decrease it.

adjusted_error = measured_error * (correct_shape_length / drawn_shape_length)


Many of the things I mentioned here have alternative approaches. I could also go a lot more in depth regarding the linear algebra needed to do the calculations or the specific classes and methods used in Unity itself. I'll however need more information as to what additional help you need before I can do that.