5.2 Virtual Object Point Model and Monocular Vision
5.2.3 Virtual Camera Model
In the context of the computation of the reprojection error, the utilization of a virtual camera represents an alternative to refractive forward projection. This means that a known 3D point needs to be projected to 2D image space, while its measured 2D image coordinates are known. By comparison of the projected and the measured image point, the accuracy of the recovered 3D point can be evaluated. The proposed virtual camera model is based on Dolereit and Kuijper [DK14].
The starting point of the computation of the virtual camera is a real image point I, which belongs to an arbitrary real object point O on the ray w. As can be seen on the left side of Figure5.6, the real object point O, the center of projection ˇC of the virtual
camera and the virtual image point ˇI are collinear. Hence, refraction is avoided and
conventional forward projection based on the pinhole camera model becomes an option. The virtual image point ˇI can be determined by perspective projection of the real object
point O with the aid of a set of virtual camera parameters. These have to be computed per image point I, since they differ as the angle of incidence α changes. Note that a set of parameters of a virtual camera, such as the one on the left side of Figure 5.6, is valid for all the possible real object points on the ray w in water.
The proposed approach differs from the approaches proposed in the literature. The first difference is that every resulting virtual camera has an image sensor that has the same size the as real camera. Furthermore, for every image point I, the virtual image point ˇI
is located at the exact same image coordinates. Hence, every pair of these two images is identical for this one image point.
Requirements. The required quantities are the refractive indices na, nw, ng, the glass layer thickness tg = d(S1, S2), the air layer thickness ta= d(C, S1) and the axis s0 with
a normalized direction vector ¯s.
Preparatory Computations. This part of the computation comprises the steps of the previously proposed refractive back-projection. The normalized direction vector ¯a of
the ray in air a, the angle of incidence α and the ray in water w, with direction vector w and position vector c, are computed as described in Section5.2.2. The position vector
c belongs to the VOP ˇC, which will be the center of projection of the virtual camera.
Additionally, the computation of the angle γ between the direction vector ¯a and the
direction vector p1 of the principal axis p and the known focal length f = d(C, P ) of the real camera are necessary for the following derivations.
Computation of the Intrinsic Parameters of the Virtual Camera. This part of the computations comprises the determination of the virtual focal length ˇf = d( ˇC, ˇP )
and the angle ˇγ between the direction vector p2 of the virtual principal axis ˇp and the
direction vector w of the ray w. The necessary steps are:
• Since the real object point O can be located arbitrarily on the ray w, choose the related VOP ˇO to be the point of intersection between the refractive interface Φ2
5.2 Virtual Object Point Model and Monocular Vision
Figure 5.6: Virtual camera model: Left: Exemplary illustration of the virtual camera that is
associated to the image point I. Right: Superimposition of the principal axis of the real and the virtual camera.
• Compute the distances d(C, ˇO) and d( ˇO, S2) in the right-angled triangle formed by the points C ˇOS2 by simple trigonometric rules utilizing the known angle α and
distance d(C, S2) = ta+ tg.
• Compute the angle of refraction β according to Snell’s law from the angle of incidence α by Equation 2.20.
• Compute the distance d( ˇC, O) in the right-angled triangle formed by the points
ˇ
CA3O by simple trigonometric rules utilizing angle β and distance d(O, A3) =
d( ˇO, S2).
Let us extract some of the quantities from the left side of Figure 5.6and illustrate them as shown on the right side. One can easily derive that if d(P , I) = d( ˇP , ˇI), then
d(A2, ˇO) = d(A1, O). (5.3)
With the previously computed distances d( ˇC, O) and d(C, ˇO) the virtual parameters ˇγ
and ˇf can be computed as follows:
• Compute the distance d(A2, ˇO) in the right-angled triangle formed by the points
A2C ˇO by: d(A2, ˇO) = sin γ · d( ˇO, C).
• Compute the angle ˇγ in the right-angled triangle formed by the points A1CO withˇ
the aid of Equation 5.3by: ˇ γ = arcsin d(A1, O) d(O, ˇC) ! = arcsin sin γ · d( ˇO, C) d(O, ˇC) ! .
5 Calibration of Shared Flat Refractive Systems
• Compute the distance d(I, P ) in the right-angled triangle formed by the points
CIP by simple trigonometric rules utilizing the known angle γ and focal length f .
• Compute the virtual focal length ˇf = d( ˇC, ˇP ) in the right-angled triangle formed
by the points ˇCˇI ˇP by simple trigonometric rules utilizing angle ˇγ and distance d(I, P ).
With the virtual focal length ˇf , it is possible to compose a virtual camera calibration
matrix. As already described in Section2.2.1, it has the form:
K = fu 0 u0 0 fv v0 0 0 1 .
The entries fu = muf and fˇ v = mvˇf are the virtual focal length of the virtual camera
scaled to pixel units. The scaling factors mu and mv, as well as the coordinates of the principal point P = (u0, v0) in the image coordinate system are adopted from the real
camera.
Computation of the Orientation of the Virtual Camera. To complete the pa- rameters of the virtual camera, this part of the computation comprises the determination of the orientation of the virtual camera, which can be represented by the axes of its coordinate system. The virtual principal axis ˇp is to a certain degree arbitrary as it is up
to now only parametrized by the angle ˇγ. In 3D space, as can be seen on the left side of
Figure5.7, this leads to a number of possible locations arranged in a circle Ω around w. The obviously choice is one of the two possibilities ˇp1 and ˇp2 in the plane of refraction Π. Both form an angle equal to ˇγ with the ray w. The following steps are necessary:
• Compute the direction vector p21 of the virtual principal axis ˇp1 or the direction vector p22 of the virtual principal axis ˇp2 with the aid of angle ˇγ.
• Choose p21 or p22 to simultaneously be the direction vector of the virtual principal axis ˇp and of the ˇz-axis of the coordinate system in question.
• Compute the ˇx-axis utilizing the superimposition of the image plane of the virtual
camera and the image plane of the real camera illustrated in the middle of Figure5.7: – The direction vector of a line connecting image point I and principal point P
forms an angle ε to the x-axis of the real camera. Consequently, the direction vector of a line connecting virtual image point ˇI and virtual principal point ˇP
has to form the same angle to the ˇx-axis in question.
– Compute the ˇx-axis in question from this angle ε and the fact that the ˇx-axis
needs to be orthogonal to the ˇz-axis.
• Compute the ˇy-axis with the cross product of direction vectors of the ˇz- and the
ˇ
5.2 Virtual Object Point Model and Monocular Vision
Figure 5.7: Left: Ambiguity of the virtual principal axis. Middle: Superimposition of the real
and virtual image plane. Right: Resulting coordinate system of the virtual camera.
The vectors ¯x, ¯y and ¯z are the normalized direction vectors of the axes of the coordinate
system of the virtual camera in the coordinate system of their computation (e.g. coordinate system of real camera), as illustrated on the right side of Figure 5.7. The transformation from the coordinate system of the real camera into the coordinate system of the virtual camera can be expressed by the rotation matrix:
R = (¯x, ¯y, ¯z)−1
and by the translation vector:
t = R · (−c),
with c being the position vector of the virtual center of projection ˇC.
Projection Matrix of the Virtual Camera. The final projection matrix of the virtual camera for one image point I and hence for one ray w is composed of:
P = K [R|t] .
With this projection matrix, any given real object point O on the ray w in the 3D coordinate space of the real camera can be projected to the point ˇI in the 2D image
space of the virtual camera. Since the coordinates of the virtual image point ˇI and its
corresponding real image point I are equal by definition of the proposed virtual camera model, this direct projection is equivalent to refractive forward projection, which usually needs to be computed iteratively. However, although a direct projection is possible, a disadvantageous overhead arises out of the need for the computation of the virtual camera parameters per image point. The proposed virtual camera model shows how underwater imaging can be converted to imaging in air. In other words, an underwater image can be equivalently expressed by image formation based on the pinhole camera model. Its usefulness will be shown in context of the computation of the reprojection error in Section 5.4.4.
5 Calibration of Shared Flat Refractive Systems