Future Work

3.5.1Dynamic Range

Recall that dynamic range is proportional to the ratio of full-scale intensity to minimum (1 LSB) intensity. Thus, a DBI illuminator’s dynamic range can be extended by reducing its minimum intensity and/or by increasing its maximum intensity.

Dynamic range can be extended by altering pulse width. With our DBI illuminator, varying the pulse width between 2 and 10 μs adds about 14 dB (~2.5 bits) of dynamic range below the LSB. (This feature was implemented but not evaluated in the display of (Lincoln, Blate, et al. 2017).) The same technique – adjusting pulse width – can be used to add resolution anywhere in an illuminator’s full scale.

In some conditions, it may be possible to omit some binary frames. For very bright scenes, where vision is entirely cone-mediated (see Figure 3), the lower-order bits of intensity-adjusted virtual objects will be zero; if the corresponding binary frames are omitted, the mean intensity will increase

proportionally. This also has the pleasant side-effect of increasing the integrated frame rate because of the reduction in the number of binary frames per integrated frame. The same principle applies for very dark scenes, where the user’s eyes are adapted into the lower end of the mesopic region (see Figure 3) – the high-order binary frames can be omitted.

Other techniques can be used by the display controller when scene adaptation requires intensities above the nominal “brightest white.” If we are willing to sacrifice some color depth for additional luminance, the display controller could illuminate some binary frames with white light (i.e., activate all three color channels); this is tantamount to the clear section of the color wheel in a conventional DMD projector (see Figure 5) – except that we have control over the intensity.

If we are willing to trade correct color balance for additional luminance, the DAC codes used for red and blue can be scaled up such that MSB frames are illuminated at full intensity (full current); this will increase the perceived brightness significantly, though at the cost of incorrect color balance.

One may also start out with a brighter light source. For example, our illuminator’s

emitter could be substituted; for example, the emitter used in (Chang, Kumar, and Sankaranarayanan 2016) is about ten times brighter than the ASMT-MT00 LED used in our DBI illuminator.

3.5.2Self-calibration, thermal compensation

DMD optics packages commonly include a light monitoring port – an aperture through which a small portion of the light output can be sensed. Feedback from such sensing could be used, for example, to enable the display controller to self-calibrate from time-to-time, e.g., during device start-up. Online sensing of output intensity would also allow for real-time compensation thermal effects and real-time color calibration71. While thermal variations will be small in laboratory and indoor use cases, thermal compensation may be desirable or necessary for devices that require certification for a wider range of operating conditions72 and/or where proper color calibration is required.

3.5.3Application: Video Projectors

DMD-based video projectors typically use DMDs with binary frame rates in the 4 to 8 kHz range. Standard PTM of 24 bpp video at 30 fps73 would require 3∙28∙30 = 23,040 binary frames per second (assuming a single-DMD projector); this far exceeds 4-8 kHz frame rates of the DMDs used in these projectors. So, from first principles, we can conclude that manufacturers must, for lack of a better word, “cheat” – i.e., simulate 24-bit color with fewer binary frames. Projectors using PTM also face the same trade-offs between color depth and integrated frame rate faced by our OST AR display. That is, adding color depth requires increasing the number of binary frames per integrated frame, thereby reducing the integrated frame rate.

DBI could be integrated into DMD-based projectors that use LED (or likely laser) illumination. Using the same example of a 24 bpp video at 30 fps, DBI requires only 3∙8∙30=720 binary frames per

71

LED luminous output per unit current varies with junction temperature; see, e.g., (Avago 2014). Temperature also causes small variations in output wavelength [idem].

72

Examples include automotive and military certifications. The typical mil-spec temperature range for devices is -55 to 125°C (Najafizadeh, et al. 2009).

73

second. With DBI, DMDs in the 4 to 8 kHz range could display 24 bpp video at 166 to 333 fps and 48 bpp HDR video at 83 to 166 fps, respectively. So, manufacturers could provide higher frame rates, wider dynamic range, or both by changing how they implement color and intensity modulation. 3D or multi-viewer video would also be possible with the addition of polarizers or the use of shutter glasses.

One concern with DBI in applications using lower-frequency DMDs is that of perceptible flicker from the periodic repetition of the high-order bits at frequencies close to or below the flicker fusion thresholds of some users. This can be easily remedied using a technique similar to the pseudo-random pulse density modulation (PR-PDM) algorithm we presented in (Lincoln, Blate, et al. 2016). Specifically, if the sequence of intensities is randomized across integrated frames, the frequency content of the output signal74 looks like noise – i.e., power is spread essentially uniformly across the spectrum. We have verified this via frequency-domain analysis of simulation data; we have not conducted perceptual testing of this technique.

3.5.4Power Efficiency

The illuminator’s present implementation uses bipolar junction transistor-based (BJT) power op amps as the power stage of the current sources. This design arises, in significant part, from its simplicity and the off-the-shelf availability of the LM675 amplifiers. The power consumption and heat dissipation of this implementation are significant. For example, each LM675 has a quiescent power consumption of about 1.8 W (Texas Instruments 2013); the illuminator thus consumes at least 5.4 W at all times (even when outputting no light) and produces a commensurate amount of heat. This is not problematic in a laboratory or tethered application but would be of consequence in, for example, a mobile HMD application, where battery life is an important figure-of-merit. Interestingly, most of the power

74 _{The “signal” is the light output from the illuminator. In the time domain, this would be seen as a periodic}

sequence of pulses of varying amplitude. Without randomization, this will appear in the frequency domain as a series of tones at, among other things, multiples of the integrated frame rate, with the strongest tone at the integrated frame rate itself. If we randomize the sequence of amplitudes, these tones disappear.

consumption of the illuminator is due to quiescent current75 because the LEDs are off most of the time (16% duty cycle) and consume less than one Watt for all but 3-4 out of every 16 binary frames.

Power consumption can be reduced in a number of ways; we will briefly discuss several options. The most straightforward approach, which would bring quiescent power into the milliwatt range, would be to change the power stage so as to separate the current regulation and power stages. Specifically, each power op amp would be replaced by low-power op amp that controls a pass transistor (e.g., a power MOSFET or a BJT); see, e.g., Figures 4.11 and 4.12 in (Horowitz and Hill 1989). Our simulations indicate that such a solution is feasible but avoiding oscillations is trickier than in our simpler implementation76. Manufacturers could also, in principle, build higher-efficiency power op amps or specialized current sources using modern techniques similar to those used in low-dropout (LDO) linear regulators. Minimized power consumption and heat dissipation will likely be found in a high-frequency switch-mode power stage or integrated current source.

3.5.5Lasers

In principle, our present current-mode design could be used to drive direct-emission laser diodes, which, like LEDs, are current-mode devices. Note that we have not experimented with using our current sources to drive lasers.

75 _{In this context, quiescent current is the current consumption of the amplifier when its output current is set to zero.}

This current is consumed by the internals of the amplifier; this is intrinsic to the amplifier itself and not a side-effect of our circuit.

76

Without entering into an extended technical discussion, the observed instability in the op amp-pass transistor circuit can be understood as follows. First, the LED itself is a non-linear component; specifically, a small change in forward bias causes a large change in forward current. From the amplifier’s perspective, this appears as a gain stage in the feedback loop. A transistor is an amplifier in its own right; so the addition of a pass transistor introduces yet another gain stage – as well as additional parasitic capacitance and inductance. Together, these factors conspire to make it easy to build a very nice high-current oscillator but difficult to build the constant-current source we require.

Lasers are also capable of being modulated at much higher frequencies than LEDs. Thus, a more likely implementation of a laser DBI illuminator would be high-frequency-PTM-modulated lasers77. In this case, PTM is used to generate a range of laser intensities (at the scale of one binary frame).

3.5.6Position-Sensitive Light Sensors

The prototype wide dynamic range, position-sensitive light sensor discussed in section 3.4.3.2 covers four ~9° vertical strips of the user’s field of view and is optimized for scenes at a range of about 2 m. A practical implementation of scene-adaptive illumination would require at least two-dimensional measurements, a wider field of view, and a wider range of scene distances. Some applications may benefit from light measurements outside the user’s field of view – above or behind the user, for example; among other things, this would aid in more sophisticated relighting of virtual objects. For example, consider a scene illuminated, in part, by a bright spotlight located behind and to the side of the user; the shadows in such a scene would be very different from isotropic overhead lighting. This information could be used by, e.g., upstream rendering stages, to render more realistic shadows on virtual objects.

Position-sensitive color sensing may also be useful to perform real-virtual color temperature matching and/or color-calibrated displays. We engage in a more speculative discussion on future work on position-sensitive light and color sensing in section 5.3.