In this chapter, we presented a nonlinear (elastic) deformation based covariance analysis which is a variant of Principal Components Analysis to operate in shell space, providing a hybrid between physical and statistical modelling of shape variation. Principal components are obtained via an eigendecomposition of a covariance matrix arising from an inner product based on the Hessian of an elastic energy. We have shown that such a model is better able to capture the nonlinear variations present in articulated body pose data and facial expression data with complex deformations.
Figure 3.9: Compactness of Shell PCA and Euclidean PCA.
(a) Error in Euclidean distance (b) Error in deformation energy
Figure 3.10: Generalisation error of Shell PCA and Euclidean PCA.
(a) Error in Euclidean distance (b) Error in deformation energy
Our method is capable of capturing non-linear deformations without using hand-crafted skeleton (Challenge 2) and is physically-inspired due to the usage of thin shell deformation energy (Challenge 3). However, the model is still based on vertex displacement and hence is alignment-dependent. As a result, we chose to minimise rigid alignment error, i.e. Procrustes analysis, before computing the displacement between input shapes and the average. This choice is arbitrary and affects effective shape variations extraction. Furthermore, thin shell deformation energy only behaves like a Riemannian distance within a small deformation. To this end, the shell PCA model allows for not-very-large deformations. In our experiments, facial expression is shown suitable to be modelled with Shell PCA, while human poses not. Taking uplifting arms and laying them down for an example, the displacements are not good representation for the pose changes and so it is not considered in our evaluation. In the next chapter, we would overcome these drawbacks by proposing a PGA variant that avoids performing any operations in the tangent space and works directly with objects lying on Riemannian manifold.
CHAPTER
4
P
RINCIPAL
G
EODESIC
A
NALYSIS IN
S
HELL
S
PACE
4.1
Introduction
The analysis of principal variations on shape spaces has attracted a lot of attention. Classical tools from PCA on linear vector spaces has been transferred to shape spaces considered as Riemannian manifolds and applied, for instance, to tasks such as classification, reconstruction and clustering [119–124].
Such “statistical” shape models seek to learn principal modes that compactly capture shape variability within or between classes. The classical approach to statistical shape modelling is due to Kendall [125] and deals with objects represented by a configuration of landmark points. A point in Kendall’s shape space corresponds to a configuration of landmarks in which the effect of rotation, translation and (optionally) scaling have been “factored out”. Usually, linear PCA is performed in the tangent space to this Riemannian manifold. Great advances have been made in the decades since, including dense modelling of discretely sampled surfaces [24, 126], modelling of continuous curves on infinite dimensional Riemannian manifolds [88], hybrid articulated and statistical models [3, 26] and state-of-the-art human body models that capture dependencies between body shape, pose and dynamic deformations [11]. However, there remain important challenges in learning statistical models of nonlinear shape variation.
Shape space must be viewed in a Riemannian setting and statistical notions have to be adapted from the Euclidean setup. The method in the previous chapter did not do this. There was no underlying manifold representation and the model was built using vertex displacement that are not meaningful for large, nonlinear deformations. We introduce a nonlinear Riemannian principal component analysis and propose an effective time-discretisation thereof.
We take into account a Riemannian structure on the space of triangular surfaces, which reflects the physics of viscous shells with a metric measuring the energy dissipation caused by
Figure 4.1: Visualisation of physically sound bending (col. 3) and membrane (col. 5) energy dissipation between resting pose (col. 1) to sample poses (col. 2 and col. 4).
membrane distortion and normal bending (see Fig. 4.1 - note the high bending energy in the armpits and membrane energy associated with stretching at the knee).
The shell space in which we work is a space of equivalence classes of shapes that differ only by rigid body motions (see Fig. 4.3) and we take special care to transfer this invariance to our time-discrete statistical analysis. Therefore, our whole framework is invariant to rigid body motion and does not require a preprocessing alignment step.
The use of the words “learning” and “manifold” in this chapter should not cause confusion with manifold learning. Unlike manifold learning, we work with manifolds that arise as a natural property of the discrete shell model. We endow our input meshes with a physical model (the discrete shell model) so that we have a notion of the energy required to deform each shape. Then, we treat each input shape (now a shell) as a point on a high dimensional, nonlinear, Riemannian manifold - this is shell space. In this space, we perform statistical learning in a way that respects its known Riemannian geometry. In practice, our treatment of the manifold uses a time- discretisation for reasons of computational tractability. So, another potential source of confusion here is the use of the word “discrete”. In the context of the discrete shell model, “discrete” refers to spatial discretisation of a continuous surface using a triangular mesh representation. In the context of our proposed time-discrete statistical model, “discrete” refers to discretisation of geodesic paths.