Partial Models

A partial model of the environment should not only record the surfaces that have been acquired, but also give clues to how the unknown surfaces can be acquired. To achieve the sampling quality requirement, it should also keep track of the sampling quality of the surfaces. It should also provide some means for surface registration to be performed between it and the new scan, and allow information in the new scan to be merged. Since the computed view can only be positioned at a location which has been determined to be empty, the partial model must keep track of known empty space in the environment. All the above are necessary to support the generation of feasible views (views that satisfy the positioning constraints), and to evaluate them using the view metric.

In this section, I first describe how a partial model may be constructed from a single scan, and later describe how partial models can be merged to produce a more complete partial model.

3.3.1.1 Partial Model from a Single Scan

When a surface region has been acquired by the scanner, the view planning system can be sure that the volume of space between the scanner and the surface region is empty. However, occlusions, drop-outs and the limited field of view of the scanner result in unknown volumes of space whose status is yet to be determined. A simulated example is shown in Figure 3.8. In the example, a light-blue box is placed in a room (the room’s ceiling is not shown), and the scanner is set up pointing towards the box (in the direction of the red axis). On the box, there is a dark circular area right in front of the scanner. Figure 3.8(b) shows the surfaces successfully acquired by the scanner. The dark circular area is too absorbent of the light from the scanner, so it appears as a region of drop-outs in the range image. The occlusion by the front face of the light-blue box not only causes part of the wall to be occluded, but the status of the volume of space between the box face and the occluded region of the wall remains

52

unknown. As for the circular area of drop-outs, the conical volume of space between it and the scanner’s viewpoint has unknown status too. Finally, the limited field of view of the scanner results in all 3D space outside the field of view to be unknown in status. In the example, the simulated scanner’s field of view is limited both vertically and horizontally, which is a more common scenario than the 360° horizontal field of view used in the specific view planning problem.

To differentiate the volumes that are known to be empty from those that are of unknown status, false surfaces are added to the surface model in Figure 3.8(b) to bound the known empty space, so as to keep the model “water-tight”. Three types of false surfaces can be added: (1) occlusion surfaces for occlusions, (2) hole-boundary surfaces for holes caused by drop-outs, and (3) image-boundary surfaces for range image boundaries caused by limited field of view of the scanner. These false surfaces are shown in Figure 3.9.

Figure 3.8: (a) The scanner is pointing towards the light-blue box (in the direction of the red axis). On the box, there is a dark circular area right in front of the scanner. (b) The surfaces acquired by the scanner. The dark circular area is too absorbent of the light from the scanner, so it appears as a region of drop-outs in the range image.

scanner position

Figure 3.9: Three different types of false surfaces. (a) The occlusion surfaces are shown in red. (b) The hole-boundary surfaces are shown in blue. (c) The image-boundary surfaces are shown in green.

Occlusion surfaces are false surfaces that connect surfaces across depth boundaries. When a range image is converted to a triangle mesh by connecting neighboring range sample points to form triangles, the occlusion surfaces will be those very elongated triangles that extend across large depth differences. There are many heuristics to detect occlusion surfaces in a triangle mesh created from a range image. For example, if a triangle’s longest edge is beyond a threshold length, it is considered a triangle of the occlusion surfaces. Another method, which is scale-independent, is to use the angle between the triangle’s normal vector and the sensor’s line of sight to the center of the triangle. If the angle is greater than a threshold angle, the triangle is considered a part of the occlusion surfaces. The volume bounded by the occlusion surfaces and behind the occluder is considered on the outside of the known empty space, and it is unknown in status, thus the next scanner view should not be located in that volume of space.

Drop-outs in the range image result in holes in the triangle mesh. Hole-boundary surface triangles are added to connect the boundaries of the holes to the scanner’s viewpoint, as shown in Figure 3.9(b). In the example, the space bounded inside the cone is considered on the outside of the known empty space. Similarly, image-boundary surface triangles are added to connect samples on the range image boundaries to the scanner’s viewpoint. The space outside the scanner’s field of view is considered on the outside of the known empty space.

With the true surfaces acquired by the scanner and the added false surfaces, the known empty space is properly bounded. What is created is a solid model [Hoffmann1989] of the known empty space.

(b) (c)

54

The false surfaces provide clues to how the unknown volumes can be resolved by subsequent scans. One strategy is to place the next view in the known empty space where it can see as much area of the false surfaces as possible. This allows the next scan to “penetrate” as much of the false surfaces as possible, in the hope of resolving the unknown volumes behind them. This strategy is incorporated in the view metric described in Section 3.3.2.

3.3.1.2 Merging Partial Models

Two partial models are merged by computing the union of their known empty space and the union of their surfaces, with the additional rules that (1) the empty space and the true surfaces have precedence over all false surfaces, and (2) the precedence of one false surface over another is arbitrary. The first rule means that if a false surface point in one partial model coincides with a point in the empty space of the other model, the result is empty space; or, if a false surface point in one partial model coincides with a true surface point in the other model, the result is the true surface point. Before the two partial models can be merged, they must be registered or aligned to each other.

The result of the merging is another partial model that can later be merged with other partial models of the same environment. This is how information from many scans can be accumulated to get a more complete partial model of the environment.

A 2D analogy of the merging process is shown in Figure 3.10. The 2D partial model in Figure 3.10(a) is a 2D version of the one shown in Figure 3.9. Figure 3.10(b) is a partial model created from a scan made from another viewpoint. The result of the merging is a partial model that has more known empty space and more true surfaces, and the volumes of unknown status have become smaller. It is always true that the unknown volumes will never become larger.

Figure 3.10: Merging of two partial models. The magenta regions are known empty space. The thick black lines are true surfaces, the red lines are occlusions surfaces, the blue lines are hole-boundary surfaces, and the green lines are image-boundary surfaces.

To support the evaluation of the view metric function, some attributes, for example the sampling quality, are associated with each surface point in the partial model. These attributes have to be properly combined during merging, and the details are described in Section 3.4. The data structure used to represent the partial models, and the implementation of the construction and merging of the partial models are also described in that section.