When to use marker-based tracking

Visual tracking does not require extra devices, as the camera is usually already part of the augmented reality system. Sensor tracking would require extra devices, which makes the system more complex and expensive. Thus, a developer may decide to use visual tracking because of the low costs or to keep the application and setup as simple and lightweight as possible. Model-based tracking would require a priori information, which limits its use. In practice, after the decision to use a visual tracking method is made, the choice is often between feature tracking and marker tracking methods (or a combination of these two).

Marker-based tracking often outperforms feature-based tracking in certain oc-casions and there are other reasons to prefer marker-based tracking. In the follow-ing, we list situations where a marker-based solution is a good choice.

1. Tracking in environments that are challenging for feature tracking

Environments with large uniform areas (e.g. large white walls) have almost no features and therefore feature tracking is impossible or at least very unreliable.

However, if the user adds markers in such environment, tracking becomes possible.

Environments with repetitive textures (e.g. tiled walls) are extremely challenging for feature tracking due to the large number of similar features. In this kind of envi-ronment marker-based tracking is often more robust. In addition, envienvi-ronments with dynamic textures (e.g. trees moving in the wind) may be even more difficult than solely repetitive features as the locations of the features vary. If the user can attach markers in this kind of environment, a marker-based system is often a bet-ter choice. A tracking system can avoid some of these problems using prior 3D reconstruction of the static part of the scene with feature-based tracking, but it then requires an initialisation process.

An environment with reflective surfaces is also challenging for feature-based tracking systems, as the features seen in a physical location change due to reflec-tions when the viewpoint changes. This kind of situation confuses a feature track-er. Again, a marker tracker may perform much better in this kind of environment, as a mirror image of a non-symmetric marker is distinguishable from the marker itself, whereas a feature tracker may collect confusing reflected features in its feature map.

There are also other challenging situations for feature tracking, where applica-tion developers might decide to use markers to increase the robustness of the system. For example, if the camera rotates wide angles between frames, features detected in one frame are invisible in the next, and the system is unable to deduce the relationship between the two views. Another difficult situation is a case where the camera stays in one location, because then the system is unable to get paral-lax between observations required to calculate features’ distances. An adequate multi-marker configuration solves both problems.

2. Acquiring the correct scale and a convenient coordinate frame

A feature-based tracking system cannot deduce the scale from the images it sees.

The environment could be a tiny scale model or a huge space; only the relative proportions can be derived from images (see Figure 58). The scale is fixed if the physical distance between two points is known. A marker tracking system knows the physical dimensions of markers and thus it knows the correct scale.

Figure 58. An optical tracking system can only derive relative proportions from camera images, not the scale. The camera image can be a projection of small object near camera or a projection of a big object far away. Based on the camera image, the system is unable to tell which of the three objects produced it.

A visual tracking system cannot deduce earth coordinates (i.e. which direction is

“up”, “down” or “horizontal”) from what it sees without any additional clues. There-fore, the origin and coordinate directions of a feature tracking system are random.

Markers lying on a floor, a table, a wall or on other known planar surface, as is

often the case, define a reference frame for the world coordinates. Typically, the marker tracking system’s origin is also relative to the marker origin.

An understandable coordinate origin and alignment of coordinate axes are im-portant usability issues. A good example is an interior design application aimed at common users, where user-friendliness is one of the main issues. The user puts a marker on the floor and virtual furniture pop up on the floor plane in the applica-tion. Thereafter the user can move the virtual furniture on the floor plane. With a pure feature tracker, the user should do extra work to get the right scale and de-fine the orientation floor plane (i.e. to align virtual and natural coordinate axis ori-entation). This might be all right in an application operated by experts, but it would probably prevent occasional end users from using this kind of the application.

In environments that are challenging for feature tracking, e.g. because of lack of features, the user could add posters (with suitable images) on the environment to enable feature-based tracking. However, as markers also provide the correct scale and convenient coordinate axis orientation, a better solution might be to use the posters as image markers and combine marker and feature tracking (see Point 5 Hybrid methods).

3. Environments with lots of moving objects and occlusions

Feature tracking may also fail in environments where a number of moving objects frequently occlude the background and objects themselves contain features. This happens for example when moving people or cars cover a major part of the scene.

A feature tracking system often loses track in such situations and therefore a mul-ti-marker system is often more robust. In such environments, a feature tracking system may also use markers for fast recovery (cf. initialisation in Point 5).

4. Need for extra information

Markers can maintain additional information, e.g. an ID, URL, text, etc.

This enables the system to associate data with markers and retrieve information.

This is something that a feature-based tracking method is unable to do. Therefore, if an application needs the extra information that a marker can provide, a marker-based system is the natural choice, especially if the system needs to be able to read previously unknown information that cannot be stored in a database. Decod-ing a marker (see Section 4.2.1) is easier and faster than text recognition. There-fore, in a real-time application a marker-based system is more convenient than an OCR system, for example.

5. Hybrid methods

Hybrid methods aim to combine the advantages of different tracking methods. For instance, in an environment where the initialisation of a feature-based system is difficult for one reason or another, developers might find the use of markers for

initialisation a good solution. In a completely new environment, the use of a mark-er is a good way to get the right scale, an undmark-erstandable coordinate origin and alignment of the coordinate axis as explained in Point 2. This is especially im-portant if end users are non-experts and cannot carry out an initialisation phase.

A common problem in feature-tracking methods is that they tend to drift over time. A hybrid tracking system is able to reset/adjust tracking each time a marker is visible and keep the system running correctly.

6. Efficiency

A marker-based system is typically computationally cheaper to implement. Marker-based tracking might be good for a proof-of-concept type of application where the emphasis is not yet on the tracking implementation but on easily demonstrating the application concept. Later, the real application can then use any tracking method (e.g. an off-the-shelf sensor tracking system).

7. Environment with existing markers

Should the environment already contain markers, the system could take ad-vantage of them, e.g. an augmented reality application for a catalogue or journal with images could use those images as natural image markers (see Section 4.3.1 Image markers). In addition, if a system operates in an environment where some marker-like signs exist, the application developer could train the marker tracker to detect them and use them to achieve a more robust tracking system. This kind of environment could be a storehouse where each shelf is marked with an ID sign, for example, and the ID signs could function as markers.

8. Devices with limited computational capacity and memory

Marker-based systems need less processing power and memory compared to feature tracking. This is an important aspect in mobile augmented reality, for ex-ample, with lightweight mobile devices.

9. Interaction with the user

User interaction in certain types of applications is easy to implement with markers.

For example, the user might move augmented objects by moving markers. Mark-ers are tangible and even for an inexperienced user it is easy to undMark-erstand how to move the objects. Users can also pick up and drop virtual objects with a marker, using it as a paddle [132], or the developer may attach markers to physical han-dles to create a tangible user interface [133].

10. Indication of the existence of virtual data

Markers also indicate the existence of virtual data for the user. Let us consider a magazine that has additional augmented content that the user can see with a camera phone. The system needs to indicate to the user somehow which pages do have virtual content. In practice, the magazine needs to use some sort of sym-bology (icons, markers) to catch the user’s attraction. The system could utilise the same markers for tracking as well.

In VTT’s self-funded TULARMAN project in 2010, we interviewed different players in printed media and advertising (printing houses, publishers, media houses, digi-tal printing houses, advertising agencies, brand owners, etc.)

Figure 59. Concept of mobile AR advertising on printed media (image: VTT Aug-mented Reality team).

In this project, we wanted to clarify how to make it easy to add AR to printed me-dia as an additional component. One thing that came up in many of the interviews was that although it is possible to use natural images for tracking, there is a need for an icon, tag or marker that indicates the existence of the digital content to the user. Otherwise, the user would be unsure which pages are linked to AR content and which are not. Should there already be a tag, there is no reason why it should not be used for initialising tracking as well.