Joint state representation

For computing joint angles we have to answer the question how to represent the state of a joint.

Representing the joint state by a rotation matrix

If we place a local coordinate system (LCS) into each landmark of our body model, we can define the joint state by the rotation matrix2 that rotates the LCS in the parent landmark into the LCS in the child landmark.

2_{A rotation matrix 𝑅 ∈ R}𝑁 ×𝑁 _{is an orthogonal matrix with det(𝑅) = 1. A matrix 𝑅} is called orthogonal, if it is a square matrix with real entries and its columns and rows are orthogonal unit vectors. An equivalent description is that the transpose of 𝑅 is equal to its inverse, i.e., 𝑅𝑇 _{= 𝑅}−1_{, which means that 𝑅}𝑇_{𝑅 = 𝑅𝑅}𝑇_{= 𝐼,} where 𝐼 is the identity matrix. Rotation matrices preserve lengths and angles of vectors, i.e. ‖𝑅𝑣‖ = ‖𝑣‖ and 𝑅𝑣 ∘ 𝑅𝑤 = 𝑣 ∘ 𝑤 (𝑣, 𝑤 ∈ R𝑁_{), where ∘ is the inner} product.

4.3. Joint state representation 61

Fig. 4.5 shows the LCS in the hip and the right elbow for some example frames.

Since a LCS is nothing more than three orthonormal vectors, we can represent the LCS by a rotation matrix as well. The relative rotation matrix 𝑅𝑖𝑗 that rotates the LCS 𝑅𝑖 in parent landmark 𝑖 into the LCS 𝑅𝑗 in child landmark 𝑗 – i.e., 𝑅𝑗 = 𝑅𝑖 𝑅𝑖𝑗 – can then be computed by (see [Craig, 2005], p. 22): 𝑅𝑖𝑗 = ⎡ ⎣ x𝑏∘ x𝑎 y𝑏∘ x𝑎 z𝑏∘ x𝑎 x𝑏∘ y𝑎 y𝑏∘ y𝑎 z𝑏∘ y𝑎 x𝑏∘ z𝑎 y𝑏∘ z𝑎 z𝑏∘ z𝑎 ⎤ ⎦ (4.1)

where the 3x3 rotation matrices 𝑅𝑖 and 𝑅𝑗 are denoted here with the help of 3x1 column vectors x𝑎, y𝑎, z𝑎 (x𝑏, y𝑏, z𝑏 respectively):

𝑅𝑖 = [︀x𝑎 y𝑎 z𝑎 ]︀ (4.2) 𝑅𝑗 = [︀x𝑏 y𝑏 z𝑏 ]︀ (4.3) This relative rotation matrix 𝑅𝑖𝑗 ∈ R3𝑥3 is one possible representation for the joint state. Though since it consists of 9 scalar values and a 3D orientation has only 3 independent degrees of freedom, it is a non-minimal representation for the joint state.

Representing the joint state by Euler angles

A minimal representation for a 3D rotation is an Euler angle triplet (𝛼, 𝛽, 𝛾) which encodes three successive elementary rotations about the axes of a coordinate system. While Euler angles describe successive rotations of a moving coordinate system (intrinsic rotations), we can in principle also describe rotations about fixed axes, i.e., within a fixed coordinate system (extrinsic rotations). A rotation about the x-,y-, or z-axis by 𝛼 degrees

can be represented by a corresponding 3x3 rotation matrix:

𝑅𝑥(𝛼) = ⎡ ⎣ 1 0 0 0 cos(𝛼) − sin(𝛼) 0 sin(𝛼) cos(𝛼) ⎤ ⎦ (4.4)

Figure 4.5.: Local Coordinate System Examples. Top row: LCS in hip

landmark. Bottom row: LCS in right elbow landmark.

𝑅𝑦(𝛼) = ⎡ ⎣ cos(𝛼) 0 − sin(𝛼) 0 1 0 sin(𝛼) 0 cos(𝛼) ⎤ ⎦ (4.5) 𝑅𝑧(𝛼) = ⎡ ⎣ cos(𝛼) − sin(𝛼) 0 sin(𝛼) cos(𝛼) 0 0 0 1 ⎤ ⎦ (4.6)

Since rotations are not commutative, the order is important and must be specified if we talk about Euler angles. In our case, we use the Z-Y-X

4.3. Joint state representation 63

order: i.e., we first rotate around the z-axis of the initial coordinate system by 𝛼 degrees, then about the resulting new y-axis by 𝛽 degrees, and finally about the new resulting x-axis by 𝛾 degrees.

Figure 4.6.: Example joint angles for a walking sequence. Selected joint

angles for the person with the T-shirt are plotted. The sequence starts with both arms stretched for some seconds (corresponding to the start interval in the plots where the joint angles are constant), followed by a hand clapping pose sequence, and then the actual walking in a circle sequence.

The overall rotation matrix can be computed by a multiplicative concate- nation of the elementary rotation matrices3

3_{Note the interesting result mentioned in [Craig, 2005], p.45: “three rotations taken} about fixed axes yield the same final orientation as the same three rotations taken

𝑅𝑧𝑦𝑥= 𝑅𝑧(𝛼)𝑅𝑦(𝛽)𝑅𝑥(𝛾) (4.7) Given a LCS in a parent landmark and a child landmark, we can compute the relative rotation matrix𝑅 according to 4.1 and then retrieve the Euler angles as a minimal representation of the joint state. For a rotation matrix 𝑅 𝑅 = ⎡ ⎣ 𝑟11 𝑟12 𝑟13 𝑟21 𝑟22 𝑟23 𝑟31 𝑟32 𝑟33 ⎤ ⎦ (4.8)

we can compute the Euler angles for the Z-Y-X order by (see [Craig, 2005], p. 43):

𝛽 = atan2(−𝑟31, √︁

𝑟2

11+ 𝑟221) (4.9)

𝛼 = atan2(𝑟21/ cos(𝛽), 𝑟11/ cos(𝛽)) (4.10) 𝛾 = atan2(𝑟32/ cos(𝛽), 𝑟33/ cos(𝛽)) (4.11)

Note that the Euler angles are only defined if 𝛽 ̸= ±𝜋/2. If 𝛽 = ±𝜋/2, then cos(𝛽) = 0, i.e., in this case 𝛼 and 𝛾 are not defined uniquely. One possible convention is then to define (see [Craig, 2005], p. 43) the Euler angles for 𝛽 = 𝜋/2:

𝛽 = 𝜋/2 (4.12)

𝛼 = 0.0 (4.13)

𝛾 = atan2(𝑟12, 𝑟22) (4.14)

and for 𝛽 = −𝜋/2:

in opposite order about the axes of the moving frame”, i.e., rotating in the order ZYX with a moving coordinate system results in the same rotation as rotating in the order XYZ within a fixed coordinate system.

4.3. Joint state representation 65

𝛽 = −𝜋/2 (4.15)

𝛼 = 0.0 (4.16)

𝛾 = −atan2(𝑟12, 𝑟22) (4.17)

While they are a minimal representation, Euler angles have two disadvan- tages: if one of the rotation axes aligns during the successive rotations with another rotation axis, one degree of freedom for rotating the coor- dinate system is lost. This situation is called “singularity” or “gimbal lock”. Another drawback are possible discontinuities, i.e., an angle near to −𝜋/2 can “jump” to +𝜋/2 and reverse, which has to be regarded when comparing joint angles4_.

The 1, 2, and 3 DoF joints are modeled by allowing for 1, 2, or 3 Euler angles to be adjustable. Fig. 4.6 shows examples of joint angles computed for a sequence in which a person walks in a circle. The plots show that the joint angle representation is independent of the orientation of the person to the camera and clearly reflects the periodic nature of the walking movement pattern.

4_{Quaternions are an alternative representation for rotations which does not suffer} from these problems at the cost of a non-minimal representation for 3D rotations: quaternions are 4D vectors. We also experimented with quaternions as joint state representation. Nevertheless, the usage of quaternions did not result in better 3D pose estimation results, which seems to indicate that the gimbal lock and discontinuity problem of Euler angles seems not to play a major role concerning the overall results in our context.