Pattern-Based Invariants

In this section, it will be shown how invariants can be derived that only depend on the system axis and some properties of the pattern on the calibration object. The calibration object does not need to be measured completely. The only requirement is that the features of the pattern are arranged in straight lines. A checker pattern provides the necessary properties for both proposed invariants, while the single invariants do not depend on all of them. This makes it interchangeable. The restrictions on the used calibration object decrease between the first and the second invariant. The first invariant depends on a known arrangement between multiple straight lines on the pattern and the second depends only on feature points arranged in straight lines.

Elementary Components. Figure 5.12shows the elements that will be needed for the derivation of the invariants. Note that only one of the flat refractive interfaces is illustrated to improve clarity. This is sufficient, since the second interface does not influence the basics of the invariants. Let us consider a set of real object points O_ij

5 Calibration of Shared Flat Refractive Systems

Figure 5.12: Left: Ray paths from the calibration object to the cameras. Right: Exemplary

construction of a feature plane Ψi.

at the inner corners of the squares on a checker pattern arranged in i ∈ {1, .., m} rows and j ∈ {1, .., n} columns. On the left side of Figure5.12, the ray paths for a subset of these points are shown. The emitted rays w_ij and w_ij0 are refracted at the interface Φ. After passing through the respective centers of projection C or C0, they end up at the left and right image points I_ij and I_ij0 . The depicted points O_ij on the right side of Figure 5.12form a line l_i. According to the utilized VOP model, the VOPs are situated on the object axes s_ij. All these object axes have the same normalized direction vector ¯s,

which is known from the system axis s₀. Hence, a line on the calibration object and the respective subset of the object axes s_ij form a plane. Such a plane will be called feature plane in the following. This is shown exemplary for the feature plane Ψ_i with i = 1, which is formed by any point O_ij on the line l_i, the normalized direction vector ¯l_i of this line and the normalized direction vector ¯s of the system axis s0. The normalization of

both direction vectors is not mandatory.

As proposed in Sections4.2 and5.3.1, the VOPs ˇO_ij and ˇO0_ij, seen by the left and the right camera, are located on the object axes s_ij, at the intersection points with the respective left and right rays a_ij and a0_ij. The basic constraint is that a feature plane, such as Ψ_i, contains all the VOPs ˇOij and ˇO

0

ij that are related to the respective real

object points O_ij on the line l_i. This is true for every feature plane in horizontal and vertical direction. In the following, it will be presented that this basic constraint implies invariants that can be utilized later on to define novel cost functions. All that is necessary is the assignment of VOPs to straight lines on the pattern of the calibration object.

1. Invariant Property

The first invariant is constrained the most, since it additionally makes use of an ar- rangement of the straight lines on the pattern. On a checker pattern, these lines run horizontally and vertically, distinguished in their direction by a right angle. On the left side of Figure 5.13, all the object axes s_ij are shown, while the cameras and the rays are neglected for better visibility. All the feature planes Ψ_i and Ψ_j are formed by these

5.4 Definition and Computation of Cost Functions

Figure 5.13: Left: Intersection of the object axes sij and of the feature planes Ψi and Ψj with

the refractive interface Φ. Right: Orthogonal view of the refractive interface Φ.

object axes and the horizontal lines l_i or the vertical lines l_j on the checker pattern. This is shown exemplary for the feature plane Ψ_i with i = 1.

Let us consider the intersections of the feature planes with the refractive interface Φ. If the refractive interface is considered as a virtual image plane, the mapping of the real object points O_ij to the points A_ij has the same properties as an orthographic projection. The orthographic projection is an affine transformation that preserves parallel lines [HZ04]. This makes it evident that the feature planes originating from a checker pattern have to be parallel in horizontal as well as in vertical direction. The right side of Figure 5.13shows an orthogonal view of the refractive interface Φ. Since the viewing direction is exactly the same as the direction of the object axes s_ij, these are seen as the points A_ij and the feature planes Ψ_i and Ψ_j are seen as parallel lines in two different directions. The shear of the rectangles is a direct consequence of a checker pattern that is not parallel to the refractive interface. As should be clear by now, an additional parallel interface does not change these properties.

The first pattern-based invariant property is based on the constraint that the feature planes are parallel in horizontal as well as in vertical direction. The orientation of a feature plane can be represented by its normalized normal vector.

• The first part of the invariant property can be defined as that the angles between the normal vectors of parallel feature planes have to be zero.

Besides the parallel feature planes, the ones that intersect can be considered as well. • The second part of the invariant can be defined as that the angles between the

normal vectors of all the intersecting feature planes have to be equal.

Note that the second part of the invariant is in this form only valid for feature planes that run parallel in only two different directions, as in the case of a checker pattern. If feature planes in more than two directions are considered, it needs to be modified

5 Calibration of Shared Flat Refractive Systems

accordingly. The angle between two feature planes α( ¯n₁, ¯n₂), with ¯n₁ and ¯n₂ being the normalized normal vectors of the feature planes, can be computed with the aid of Equation 2.1. The arithmetic mean of the all the angles between the parallel feature planes and likewise between the intersecting feature planes can be computed by:

µ_p = _m1 2
m−1 X i=1 m−i X k=1 α( ¯n_i, ¯n_i+k) ! + 1_n 2
  n−1 X j=1 n−j X k=1 α( ¯n_j, ¯n_j+k)  = 0, (5.11) µc = 1 m · n   m X i=1 n X j=1 α( ¯ni, ¯nj)  . (5.12)

In the case of parallel feature planes, the mean µ_p is computed in horizontal and vertical direction. Since it is supposed to be equal to zero, it is used directly in the expression for the invariant. In the case of intersecting feature planes, the mean µ_c has to be subtracted from each angle. Overall, the invariant has to result in zero and can be expressed by:

µp+   m X i=1 n X j=1 α( ¯ni, ¯nj) − µc  = 0. (5.13)

It is worth mentioning that this invariant could be extended to all possible straight lines on the pattern. This includes the diagonals in both directions. Furthermore, any calibration object with features arranged in straight lines could be used instead, if comparable relations between the feature planes can be defined.

2. Invariant Property

The second invariant property does not depend on a known arrangement of the straight lines on the calibration object. The only prerequisite is the arrangement of the points

O_ij on the calibration object in straight lines. Let us take the feature plane Ψ_i on the right side of Figure5.12 as an example. The invariant is based on the fact that all the object axes s_ij that originate from line l_i lie in the same feature plane Ψ_i. The direct consequence is that the VOPs ˇOij and ˇO

0

ij that are related to the real object points Oij

on the line l_i also have to lie in this plane.

The second pattern-based invariant property is based on the constraint that the feature planes contain VOPs.

• The invariant property can be defined as that the distance between a feature plane and all of its associated VOPs has to be zero.

If the Hesse normal form of the equation of the feature plane is available: Ψ_i : ¯ni· x = di,

then Equation 2.5 can be utilized. It directly yields the Euclidean distance d(A, Ψ_i) between an arbitrary point A and the feature plane Ψ_i. Note that the point A needs

5.4 Definition and Computation of Cost Functions to be substituted by the VOPs ˇO_ij or ˇO_ij0 . Since this distance is supposed to be zero between all the VOPs and their associated feature plane, the invariant can be expressed exemplary for the line l_i with i ∈ {1, .., m} on the calibration object by:

n X j=1 d( ˇOij, Ψi) + n X j=1 d( ˇOij0 , Ψi) = 0. (5.14)

Every single line on the pattern of the calibration object is independent of the remaining straight lines. Therefore, the utilized checker pattern can be replaced accordingly.