What intuitively meaningful 2D transform can three-finger multitouch represent? [closed]

Question

With two fingers on a touch sensing plane, we can "grab" (control) two points within a section of 2D plane.

It means it is possible, using two fingers, to translate, rotate, and scale a 2D object with full precision.

A good way to see that this is optimal is by seeing that the 4 degrees of freedom of our two points (one x and one y on each point) can be used to compute the 4 degrees of freedom of a transform which defines translation (2), rotation (1), and scale (1).

With this we can create magical virtual 2D items that we can grow and shrink and orient in the plane.

Looking at the 1 point version is quite a bit simpler: We can control translation directly. It's possible to wire up rotation or scale also by specifying a second fixed point.

My question is how can I extend this concept to three points? I am having difficulty determining whether 3 points (6 DOFs) can define a 2D affine transformation. Even if it is the case that a unique affine transformation can be computed, there's not really a lot of value to being able to skew/shear shapes.

How else might these 6 DOFs be used to map out a transformation that has intuitive value? It seems difficult even if I expand the scope to 3D transformations.

For instance, the 3D rigid transformation space is also 6 DOF (3 translation, 3 rotation), but it's definitely not clear to me how a mapping that makes sense can be done. Consider expanding and contracting your 3 fingers. It must modify translation or rotation or both but intuitively the only right thing seems to be scaling. I could say that pinching 3 fingers together can take you further away (translate) based on the current rotation value, but that's not actually any more than a single scalar control and is not much different from controlling a 1 DOF scale transform either. I guess another way that hints at this being impossible is the fact that the 6 DOFs coming out of 3 points cannot be split evenly into two sets of 3 DOFs that define a 3D rigid transform. What am I gonna do, set a translation using the x-coordinate of each point? That would be meaningless at best.

I can see some value in a super-virtual-trackball by using two fingers to manipulate a trackball (turning 4 DOFs into a somewhat fully controlled 3D rotation(except it's not full control because you'll be limited to a hemisphere due to the trackball) plus a scale) which I don't believe I've seen any demo of this sort of thing yet, I guess that would be cool to try to implement. Maybe yet another virtual rubik's cube game. It's clear that 3 fingers are not necessary if you're doing just rotation. I can't really come up with a good way to use all 3 fingers because scale can be accomplished with two. Seems like the issue with 3D translation is the axis normal to the plane. You can't really map anything to that intuitively at all.

Here's an article that is pretty awesome that has nice diagrams and talks about a single-point-controlled virtual trackball.

I guess that's it. Has anybody ever come up with a cool way to manipulate something that makes use of (i.e. requires) 3 control points in the plane?

Maybe a Rubik's Cube where you use two fingers to fixate the 3D orientation (rotation) of it while a third finger manipulates 9 of the cuboids. That is 3 point control that requires all of them.

Update: I just thought of something. Maybe considering a hypothetical device, the 3D touch sphere, (which of course we can map a portion of it to a 2d planar touch screen) can help in thinking about this.

It may just collapse to similar dynamics as the virtual trackball however.

Answer 1

First of all, a set of 3 points can indeed define a general affine transformation in 2D. Affine transformations can map a triangle to any other triangle, so using the initial and final positions of a triangle's 3 points, one could derive the affine transformation to map between those triangles. However, as you say, it doesn't seem likely users will want to nonuniformly scale and shear things.

In 3D, the first thing I can think of is to use the third finger as a control for rotation not in the plane of the screen. That is, a two-finger gesture would translate, scale/zoom, and rotate in the plane of the screen as before; in a three-finger gesture, manipulating one finger while keeping the other two fixed would rotate around the other two axes, rather like rolling a ball under your finger. However, it's not clear to me how you would decompose all the motions in the general case where all three fingers are in motion.