Dynamic Binary Frame Illumination (DBI)

Our illumination method enables the display controller to illuminate each binary frame with any color and intensity of light within the illuminator’s dynamic range. We call this method dynamic binary frame illumination (DBI). The word “dynamic” refers to the fact that the luminosity and color of illumination is dynamically-controlled on a per-binary-frame basis. In contrast, traditional BSLM-based displays use constant-intensity (static, as opposed to dynamic) illumination and PTM.

Traditional DMD-based AR Display using PTM

3.3.1.1

A figurative traditional DMD-based AR display is shown in Figure 8; some optical components have been omitted for clarity. This 4-bit monochrome display uses a constant-intensity light source and generates grayscale using PTM. An optical combiner (bottom left) combines incoming light from the real world with virtual imagery generated by the DMD-based display. The architecture depicted is equivalent to that of the low-latency, 6-bit monochrome display reported by Lincoln, Blate, et al. (2016).

To display the 4-bit monochrome teapot image (“Integrated Frame”, bottom right), the DMD displays a sequence of 16 binary frames (center right). This sequence of binary frames is a pulse train modulation (PTM) of the input image. Due to persistence of vision and the DMD’s high binary frame rate (16 kHz in this example), the sequence of black-and-white frames is perceived as the grayscale integrated frame. Each pixel’s perceived intensity is proportional to its duty cycle over the integrated frame period.

PTM requires O(2n) binary frames to generate an n-bit-per-pixel monochrome image (integrated frame). Consequently, PTM requires O(2n) binary frame periods to output a single n-bit-per-pixel image. DBI was conceived to make more efficient use of the BSLM’s bandwidth. The efficiencies of PTM and DBI are discussed quantitatively in section 3.3.1.3.

DBI-based AR Display

3.3.1.2

A figurative DBI-based AR display is shown in Figure 9. The architecture depicted is equivalent to that of the HDR, scene-adaptive, low-latency display reported by Lincoln, Blate, et al. (2017), the embodiment of which is shown in Figure 7. In Figure 9, the constant-intensity light source of the

traditional display has been replaced by a (variable-intensity) DBI illuminator. Note that both the DBI illuminator and the DMD itself are controlled by the display controller.

To display the 4-bit monochrome teapot image (“Integrated Frame”, bottom right), the DMD displays four binary frames, each illuminated at a binary-weighted intensity (center right). The resulting illuminated binary frames are shown below the green arrow. Due to persistence of vision, the user perceives this sequence of four binary frames as the 4-bit monochrome integrated frame (bottom right).

DBI requires O(n) binary frames to generate an n-bit-per-pixel monochrome image (integrated frame). Consequently, DBI requires only O(n) binary frame periods to output a single n-bit-per-pixel image. DBI is O(2n/n) times more efficient than traditional PTM in terms of binary frames per integrated frame.

Efficiency: DBI versus PTM

3.3.1.3

Binary Frames per Integrated Frame

As n is increased, the efficiency of DBI versus PTM is more pronounced. Table 1 lists numerical examples for several values of n (and the corresponding luminous dynamic ranges). Recall that we estimated the luminous dynamic range of typical scene to be 100-120 dB. At 16 bpp (96 dB), PTM requires 65,536 binary frames per integrated frame; recalling that the DMD’s binary frame rate is 16 kHz, a 16 bpp PTM-based display would have an integrated frame rate of about 0.25 Hz or four seconds per integrated frame. In contrast, DBI requires only 16 binary frames – an integrated frame rate of about 1 kHz (1 ms per integrated frame). Even at 20 bpp (120 dB), DBI is over 12 (256/20) times more efficient than 8 bpp PTM.

Overall Intensity

Traditional PTM-based displays use a constant-intensity light source, i.e., the light source is always run at its maximum intensity. DBI, however, uses different intensities for each binary frame – most of which are less than the DBI light source’s maximum intensity. All things being equal, then, we

would expect a display’s maximum luminance to be lower under DBI than under PTM. Let us consider this tradeoff quantitatively.

Assume that the display is displaying an image where every pixel is white (i.e., the brightest image that could be displayed). Let the light source’s maximum luminous flux (lumens) be denoted L and, for simplicity, that, when all of the DMD’s mirrors are in the “on” position, 100% of L arrives at the optical combiner (i.e., the DMD and optics have zero insertion loss). Let LPTM(n) be the mean luminous

flux of the display under PTM when displaying an n-bit all-white image let and LDBI(n) be the mean

luminous flux of the display under DBI when displaying an n-bit all-white image. Under PTM, all mirrors will be in the “on” state for each of the 2n

binary frames per integrated frame and the light source’s luminous flux will be L for every binary frame; thus, LPTM(n)=L for all n.

Under DBI, all mirrors will be in the “on” state for each of the n binary frames per integrated frame but, the light source’s luminous flux will be { L, L/2, L/4, … , L/2n-1

} or, equivalently, 𝐿_𝐷𝐵𝐼(𝑛) =1_𝑛[𝐿 ∙ (1 + ∑ 1 2𝑖 𝑛−1 𝑖=1 )] = (_𝑛𝐿) ∙ (1 +2𝑛−1_2𝑛−1− 1)

For n≥7, LDBI(n) = 2 ∙ 𝐿 𝑛⁄ within less than 1%

34_{. We thus can state that, for n≥7,}

LPTM(n)/LDBI(n)=n/2. Equivalently, we might say that DBI is n/2 times less efficient than PTM in terms of

the display’s maximum luminance.

In the context of a projector, this loss of intensity might be problematic because the projector’s output will be typically viewed on a large screen a significant distance away from the projector (see, e.g., our discussion of Chang, Kumar, and Sankaranarayanan (2016) in section 3.2.2). In a near-eye display, however, we require far less luminous power because we are projecting into a much smaller area (the eye box, whose size is O(mm2), versus a projection screen, whose size is O(m2)) at a much closer distance (a

34

vertex distance35 on the order of 25-50 mm versus several meters to the projection screen). Methods for increasing a DBI illuminator’s overall (or perceived) intensity are discussed section 3.5.1.