Lecture 15 Tone Mapping

(1)

Lecture 15

Tone Mapping

Oct. 8, 2015

Slide Credits:

Frédo Durand

Alexei Efros

(2)

The World is a High Dynamic Range (HDR)

(3)

Multiple exposure photography

• Sequentially measure all segments of the range

10 ^-6 10 ⁶

Real world

Picture

Low contrast

High dynamic range

Tuesday, March 6, 12

(4)

log Exposure

Assuming unit radiance for each pixel

After adjusting radiances to obtain a smooth response

curve

P ixe l va lue

3 1 2

log Exposure

P ixe l va lue

Slide stolen from Alyosha Efros who stole it from Paul Debevec

Response curve

• Radiance is unknown, fit to find a smooth curve

Thursday, March 8, 12

Response Curve

•

Radiance is unknown, fit to find a smooth curve

(5)

(6)

HDR File Formats

(7)

Overview of HDR Encoding

• RGBE/XYZE

• 24 bits/pixels as usual, plus 8 bit of common exponent

• Introduced by Greg Ward for Radiance (light simulation)

• Enormous dynamic range

• OpenEXR

• By Industrial Light + Magic, also standard in graphics hardware

• 16bit per channel (48 bits per pixel) 10 mantissa, sign, 5 exponent

• Fine quantization (because 10 bit mantissa), only 9.6 orders of magnitude

• Others: 48-/96-bit TIFF, 32-bit LogLuv TIFF, PPM Extension (32-bit/channel), JPEG200 Extension (can be lossy)

Summary (Greg Ward): http://www.anyhere.com/gward/hdrenc/hdr_encodings.html

(8)

Radiance RGBE and XYZE

•

Simple format with free source code

•

8 bits each for 3 mantissas and 1 exponent

•

76 orders of magnitude in 1% steps

•

Run-length encoding (20% avg. compression)

•

RGBE format does not cover visible gamut

•

Dynamic range at expense of accuracy

•

Color quantization not perceptually uniform

(9)

(145, 215, 87, 149) =

(145, 215, 87) * 2^(149-128) = (1190000, 1760000, 713000)

(145, 215, 87, 103) =

(145, 215, 87) * 2^(103-128) =

(0.00000432, 0.00000641, 0.00000259)

Ward, Greg. "Real Pixels," in Graphics Gems IV, edited by James Arvo, Academic Press, 1994

Radiance!format!(.pic,!.hdr,!.rad)!

Red Green Blue Exponent

32 bits/pixel

Radiance RGBE Format (.pic, .hdr)

(10)

ILM OpenEXR Format

• 16-bit/primary floating point (sign,1; exponent, 5; mantissa, 10)

• 9.6 orders of magnitude in 0.1% steps

• Additional order of magnitude near black

• Wavelet compression of about 40%

• Negative colors and full gamut RGB

• Alpha and multichannel support

• Open Source I/O library released Fall 2002

• Slower to read and write

(11)

ILM’s OpenEXR (.exr) ILM’s"OpenEXR"(.exr)"

•  6"bytes"per"pixel,"2"for"each"channel,"compressed"

sign exponent mantissa

•  "Several"lossless"compression"opAons,"2:1"typical"

• "CompaAble"with"the"“half”"datatype"in"NVidia's"Cg"

• "Supported"naAvely"on"GeForce"FX"and"Quadro"FX"

"

• "Available"at"hQp://www.openexr.net/"

(12)

Tone Mapping

Overview

(13)

The Radiance map

Thursday, March 8, 12

The Radiance

Map

(14)

Linearly scaled to display device

The Radiance map

Linear mapping to display is

insufficient!

The Radiance

Map

(15)

262 Chapter 4: Tonemapping

effects. Everything was simulated correctly ac- cording to the books, but it didn’t really look so convincing until game makers started to master tonemapping.

In this real-time context, the tonemapping operator can be considered the virtual camera the game is shot with. It has to dynamically meter the exposure as the player travels across varying lighting conditions and constantly

readjust exposure accordingly. But that is only half the story. A simple exposure adjustment doesn’t make it cinematic enough, and that is where most early games failed. Nowadays tonemapping operators also take on the role of virtual film stock. These algorithms use com- plex rule sets and custom tone curves to keep some indication of detail visible in shadows and highlights. The way the tones in between are distributed and how that virtual film stock responds to colors are nowadays crucial ele- ments to define the look of a game. That can be the gritty bleached look of a militaristic ego

shooter or the award-winning picturesque look of the Uncharted adventure series—in any case, the tonemapping style is meticulously crafted to express a certain mood and support the overall art direction. High-end games even contain a data track, where the game artists can animate exposure and other tonemapping parameters along with the camera motion, so it becomes a truly handcrafted experience.

Typically, games use global tonemappers for performance reasons. Local enhancements will surely come up in the future, but they are not urgently necessary because game makers have total control over lighting and textures anyway. They can simply tweak their world to have a lower contrast to begin with. However, these real-time tonemappers are still capable of adapting dynamically to changing lighting conditions and applying some very controlled looks. That makes them work well for batch processing, time lapse, and HDR video.

Figure 4-23: Tonemapping plays an essential role in real-time rendering. It defines the look and enhances the cinematic appearance of high-end games like Uncharted 3 by Naughty Dog.

Christian Bloch - The HDRI Handbook 2.0

(16)

Problem 2: Display the infromation

• Match limited contrast of the medium

• Preserve details

10 ^-6 10 ⁶

Real world

Picture

Low contrast

High dynamic range

Tone Mapping

• Problem: How should we map scene radiances (up to 1:100,000) to display radiances (only around 1:100) to produce a satisfactory image?

• Goal: match limited contrast of the display medium while preserving details

(17)

Visual Matching

•

We do not need to reproduce the true radiance as

long as it gives us a visual match.

(18)

Recall: Eyes and Dynamic Range

• We're sensitive to contrast (multiplicative)

• A ratio of 1:2 is perceived as the same contrast as a ratio of 100 to 200

• Use the log domain as much as possible

• Eyes are not photometers

• Dynamic adaptation (very local in retina)

• Different sensitivity to spatial frequencies

Headlights are ON in

both photos

(19)

Weber’s Law

• We can detect ~0.5% changes in intensity

• The perceived minimum increment is the just noticeable difference

• True for luminance as well as other things (length, weight, sound, etc.)

• Takeaway: The amount of difference that we can detect is relative to the context between the two things, not an absolute quantity

• If you change the context, you change the increment

Weber’s Law

• We can detect 0.5% change in intensity (!)

• True for intensity, length, weight, sounds, etc.

I

I = K _w

Just Noticeable Difference (JND) Base intensity

Weber fraction (constant!)

(20)

Haze removal

Landscape shots often have a lot of atmospheric haze, especially when a vista stretches far into the distance. Sometimes that may be a desired effect, but more often you would rather see

clearly all the way to the horizon. It’s not exactly what they’re designed for, but local detail enhancement algorithms just happen to be well suited for enhancing clarity and punching

through this haze. And that works even though the overall scene contrast typically doesn’t jus- tify a real HDR shoot—a single RAW is normally all you need for this application.

Just like natural toning, this is actually a reconstruction job. It’s very tempting to over- shoot, especially when you start to discover more things than you could actually see with your naked eye. But ultimately, you want this to be a natural image that looks like it was shot on

a clearer day. It helps to mentally set this goal up front.

Since you’re boosting contrast in the horizon area quite a bit, this process needs very clean and noise-free image data to begin with. Other- wise the image will break up and look manipu- lated. Shooting RAW is essential, at the highest possible bit depth and the lowest possible ISO.

Browse back to section 3.2.9 for some hints on pseudo-HDR conversion. Even if the hazy horizon will fit in a single RAW, it doesn’t hurt to combine multiple exposures just for the sake of averaging out the last bit of noise.

Texture extraction

When people think about using HDRI for computer graphics and VFX, they often overlook the applications of tonemapping for preparing texture images. The importance of good reference

Before After

Figure 4-21: Atmospheric haze is easily wiped away with tonemapping techniques. Recovering all these details from the murky shadows is exactly where local TMOs excel.

(21)

Common Artifacts to Avoid

•

Tone Reversal: large image areas have swapped brightness

•

Noise boost: noise emphasized in regions which were smooth

•

Oversaturation: increasing color beyond what is natural

•

Halos: bright glows around dark edges

•

Flattening: over compressing the dynamic range into something less than the display can do

•

Webbing: Division of the image into cells based on natural

lines

(22)

Best Practice Tips 273

4.3.3

Global tonemappers are immune to halos, and most local tonemappers nowadays have a dedi- cated halo reduction built in. In some rare cases, halos can also benefit an image by adding a

strong separation between foreground and background.

Tone reversal

This term means that large image areas have swapped overall brightness levels; for example, the sky may suddenly appear much darker than the foreground. Some people call it contrast

reversal, and it can also happen on a smaller scale. An obnoxious example is a dark spot in the middle of a lightbulb. Tone reversal is often accompanied by halos as the result of excessive noodling with the image. Global tonemappers are immune, and when using local TMOs, thinking in zones is the best way to keep a thumb on tone reversal. Used with caution in the context of artistic toning, tone reversal can lead to very

dramatic results and may even be a desired effect.

Noise boost

Local tonemappers are like watchdogs, trained to snoop out details everywhere. Especially when applied without smoothing or with a small radius value, they sometimes confuse residual noise with valuable details worth ex- aggerating. Smooth gradients like clouds, clear skies, and flat walls are often affected and start looking grainy. One way to prevent this is to start off with a cleaner HDR image. Shoot at the lowest ISO and consider shooting more exposures with smaller EV steps so the noise will be averaged away during the merging process.

Another way is to treat the affected areas af- terward with a noise removal filter. Noiseware, Topaz DeNoise, Neat Image, and Noise Ninja are all great solutions for that.

Tone Reversal Noise Boost Oversaturation Halos Flattening Webbing

Figure 4-33:

Compilation of typical tonemapping artifacts, each one driven to the extreme.

(23)

Simple Approaches to

Tone Mapping

(24)

Can we just scale? Maybe!

•

For a color image, try to convert the input (world) luminance Lw to a target display luminance Ld

•

This type of scaling works

(sometimes). In particular, it works best in the log and/or

exponential domains (because

of Weber’s law) ! ! ! ! ! ! !

"

#

$ $

$

%

&

=

! !

!

"

#

$ $

$

%

&

w d w

d d d

L L B

L L G

L L R

B

G

R

(25)

Recall: Gain, Bias, and Gamma

•

𝛂 is known as gain (exposure)

•

𝛃 is known as bias (offset)

•

ɣ maps us to a non-linear curve (gamma

correction)

y Controlling Photoshop’s Highlight Compression method

1 In the 32-bit Preview Options dialog box, set Method to Highlight Compression.

2 Choose ^IMAGE > ADJUSTMENTS > ^EXPOSURE and play with the sliders in the Exposure dialog.

These sliders behave in a slightly different way than they normally do because highlight details always have the right of way. Nevertheless, they are extremely interactive and easy to control.

3 When you’re happy with the tonal distribution, just bake it all down by performing the bit depth conversion in the ^IMAGE > ^MODE menu with the Highlight Compression method.

This mode works great when your major

concern is to protect the highlight details. As long as Highlight Compression is activated as the display method, the histogram is always anchored on the brightest end. Exposure and Gamma sliders now control how steep the loga- rithmic intensity decay across the midtones

and shadows is. You can push exposure adjustments much further than you normally could because highlights will only get increasingly compressed instead of blowing out to white.

Even though this mode is listed as having no options, it is actually one of the most versatile and intuitive ones. You have the entire toolkit Figure 4-5: The trick for controlling the Highlight

Compression method is to enable it in the View menu first.

Figure 4-6: Now Photoshop’s simple Exposure adjustment can be used to interactively tweak the look.

Figure 4-7: The actual tonemapping dialog has no options, but that’s okay. You don’t need them anymore.

C

out

= (𝛂C

in

+ 𝛃)ˠ

Photoshop has this functionality

(26)

Naïve technique

• Scene has 1:10,000 contrast, display has 1:100

• Simplest contrast reduction?

Normalization

• Typical scene has 1:10,000 contrast, display has 1:100

• Simplest reduction destroys midtones

(27)

Naïve: Gamma compression

• X −> X γ (where γ=0.5 in our case)

• But… colors are washed-out. Why?

Input Gamma

applied

independently on R, G & B

Gamma Compression

• C_out = C_inˠ, where 0 < ɣ < 1 applied to each R,G,B channel

• Colors are washed out, why?

(28)

Gamma compression on intensity

• Colors are OK,

but details (intensity high-frequency) are muddy

Gamma on intensity Intensity

Color

Gamma Compression on Intensity

•

Colors ok, but details in intensity are blurry

(29)

Histogram Equalization

•

Questions to answer: how many bins?

•

Better to equalize the log of intensity

Banterle - Advanced High Dynamic Range Imaging

(30)

Using Local Operators

(31)

Photographic Tone Reproduction for Digital Images

Erik Reinhard University of Utah

Michael Stark University of Utah

Peter Shirley University of Utah

James Ferwerda Cornell University

Abstract

A classic photographic task is the mapping of the potentially high dynamic range of real world luminances to the low dynamic range of the photographic print. This tone reproduction problem is also faced by computer graphics practitioners who map digital images to a low dynamic range print or screen. The work presented in this paper leverages the time-tested techniques of photographic practice to develop a new tone reproduction operator. In particular, we use and extend the techniques developed by Ansel Adams to deal with digital images. The resulting algorithm is simple and produces good results for a wide variety of images.

CR Categories: I.4.10 [Computing Methodologies]: Image Pro- cessing and Computer Vision—Image Representation

Keywords: Tone reproduction, dynamic range, Zone System.

1 Introduction

The range of light we experience in the real world is vast, spanning approximately ten orders of absolute range from star-lit scenes to sun-lit snow, and over four orders of dynamic range from shadows to highlights in a single scene. However, the range of light we can reproduce on our print and screen display devices spans at best about two orders of absolute dynamic range. This discrep- ancy leads to the tone reproduction problem: how should we map measured/simulated scene luminances to display luminances and produce a satisfactory image?

A great deal of work has been done on the tone reproduction problem [Matkovic et al. 1997; McNamara et al. 2000; McNamara 2001]. Most of this work has used an explicit perceptual model to control the operator [Upstill 1985; Tumblin and Rushmeier 1993;

Ward 1994; Ferwerda et al. 1996; Ward et al. 1997; Tumblin et al.

1999]. Such methods have been extended to dynamic and interactive settings [Ferwerda et al. 1996; Durand and Dorsey 2000; Pat- tanaik et al. 2000; Scheel et al. 2000; Cohen et al. 2001]. Other work has focused on the dynamic range compression problem by spatially varying the mapping from scene luminances to display luminances while preserving local contrast [Oppenheim et al. 1968;

Stockham 1972; Chiu et al. 1993; Schlick 1994; Tumblin and Turk 1999]. Finally, computational models of the human visual system can also guide such spatially-varying maps [Rahman et al. 1996;

Rahman et al. 1997; Pattanaik et al. 1998].

Radiance map courtesy of Cornell Program of Computer Graphics

Linear map

New operator

Figure 1: A high dynamic range image cannot be displayed directly without losing visible detail using linear scaling (top). Our new algorithm (bottom) is designed to overcome these problems.

Using perceptual models is a sound approach to the tone reproduction problem, and could lead to effective hands-off algorithms, but there are two problems with current models. First, current models often introduce artifacts such as ringing or visible clamping (see Section 4). Second, visual appearance depends on more than simply matching contrast and/or brightness; scene content, image medium, and viewing conditions must often be considered [Fairchild 1998].

To avoid these problems, we turn to photographic practices for in- spiration. This has led us to develop a tone reproduction technique designed for a wide variety of images, including those having a very high dynamic range (e.g., Figure 1).

2 Background

The tone reproduction problem was first defined by photographers.

Often their goal is to produce realistic “renderings” of captured scenes, and they have to produce such renderings while facing the

(32)

Ansel Adams

(33)

The Zone System

•

Formalism to talk about exposure, density

•

Zone = intensity

range, in powers of two

•

In the scene, on the negative, on the print

Ansel Adams

(34)

The Zones

(35)

Zone System

(36)

The Zone System

• You decide to put part of the system in a given zone

• Decision: exposure, development, print

(37)

Zones Example

they appear merely darkish. Because then they would blend in with the dark ground, and the clear separation of the shadow zone would disappear. Thinking in zones provides a much- needed sanity check and becomes utterly useful when faced with a myriad of “what if…” deci- sions. Pick a zone name and stick to it.

Sometimes the zones are harder to see than you would think. It helps to squint your eyes and scale the image down to thumbnail size.

In Photoshop you can also use a stepped gradient layer to get a visual representation in the form of a zone chart. In fact, there’s a gradient preset for you on the DVD, and I will show you in section 4.4.1 exactly how to use it. But don’t feel obliged to slavishly stick to that preset; the exact definition of each zone’s brightness range may well depend on the scene and your own rendering intent. We’re not after image metrics anyway, but rather the visual impression—so it’s completely fine to trust your instincts when calling out the zones. Also, nobody says you have to populate every zone. If you’re specifically going for a more painterly look, you may choose to settle everything in the Midtones with only a few Highlights sticking out. But at least this zone-esque system gives you a lan- guage to formulate your goal, which will lead to a very straightforward toning workflow.

If all that sounds too sloppy to you, there’s nothing holding you back from applying the

same thought process to Adams’s original zone system. Then you would assign 11 zone num- bers instead of names and stick to those assignments throughout the tonemapping process.

Just do a Google search for “Ansel Adams Zone System” and prepare for an afternoon of heavy reading. Personally, I will use this easy zone- esque system and scatter a few more of these handsome zone charts throughout the book.

4.3.3 B eware of Common Artifacts

No matter what your artistic intentions are, you should always be aware of some particular artifacts that may appear when processing HDR images. For natural looks you want to avoid

these artifacts, for impressionist pieces you

want to keep them at least under control, and if you feel artsy today, you may want to embrace them. But in any case, it always starts with your ability to spot them.

Halos

Bright glows around dark edges are called halos. They often appear on high-contrast build- ing edges against a bright sky or between the branches of a tree. Halos are a dead giveaway that an image was doctored; in appearance they are very similar to over-sharpening artifacts or a heavy-handed Unsharp Mask filter. There can also be dark halos (dark glows on bright edges), but these are less common and less obtrusive.

Figure 4-32:

Zone chart of the assignments for this image.

Highlights Brights

Midtones

Darks Shadows

Midtones Shadows

(38)

Dodging and Burning

(39)

Limited dynamic range can be good!

• W. Eugene Smith photo of Albert Schweitzer. 5 days to print!

• Things can be related because the intensity is more similar

• Balance, composition

Limited dynamic range can be good!

• W. Eugene Smith photo of Albert Schweitzer

• 5 days to print!

• Things can be related because the intensity is more similar

• Balance, composition

(40)

Dodging and burning

• During the print process, selectively hide parts of the print during exposure

• Dodging (left) block light from regions to make darker

• Burning (right) overexpose regions to make brighter

Dodging and burning

• During the print

• Hide part of the print during exposure –Makes it brighter

From The Master Printing Course, Rudman

(41)

Dodging and burning

From Photography by London et al.

Dodging and Burning

(42)

Dodging and burning

• Downside: Must be done for every single print

Dodging and burning

• Must be done for every single print!

Straight print After dodging and burning

(43)

Dodging & Burning is Difficult

Source: Rudman

(44)

Digital Zoning

(45)

Global&operator&

! !

"

#

$ $

%

&

+

= ∑

y x

w

L x y

L N

,

)) ,

( 1 log(

exp δ

) ,

( )

,

( L x y

L y a

x

L

_w

w

m

=

Approxima/on&of&scene’s&&

key&(how&light&or&dark&it&is).&

Map&to&18%&of&display&range&

for&averageCkey&scene&

UserCspeciﬁed;&high&key&or&low&key&

) ,

( 1

) ,

) ( ,

( L x y

y x

y L x

L

m d m

= +

transfer&func/on&to&compress&&

high&luminances&

(46)

Global&operator&

) ,

( 1

) ,

(

) ,

1 ( ) ,

( )

, (

2

y x L

y x y L

x L

y x L

m

white m m

d

+

!! "

#

$$ %

&

+

=

It&seldom&reaches&1&since&the&input&image&does&not&

have&inﬁnitely&large&luminance&values.&

L

_white_&

is&the&smallest&luminance&

to&be&mapped&to&1&

(47)

low$key$(0.18)$ high$key$(0.5)$

(48)

Dodging'and'burning'(local'operators)'

•  Area'receiving'a'diﬀerent'exposure'is'o8en'bounded' by'sharp'contrast'

•  Find'largest'surrounding'area'without'any'sharp' contrast'

) ,

( )

, ( )

,

( x y L x y G x y

L

^blur_s

=

_m

⊗

_s

blur s blur

s blur

s s

L s

a

y x

L y

x y L

x

V +

= −

₂ ⁺¹

2 ) ,

( )

, ) (

,

(

_φ

ε

y) <

(x, :

max

V

s

(49)

Dodging'and'burning'(local'operators)'

•  A'darker'pixel'(smaller'than'the'blurred'average'of' its'surrounding'area)'is'divided'by'a'larger'number' and'become'darker'(dodging)'

•  A'brighter'pixel'(larger'than'the'blurred'average'of' its'surrounding'area)'is'divided'by'a'smaller'number' and'become'brighter'(burning)'

•  Both'increase'the'contrast'

) ,

( 1

) ,

) ( ,

(

max

x y

L

y x

y L x

L

_blur

s d m

= +

(50)

Scale&Selec'on&

Radiance map courtesy of Paul Debevec

Scale too small (s )

Right scale (s )

Scale too large (s ) Center

Surround Center

Right scale (s ) Surround

Center

Surround

Scale too small (s ) Scale too large (s )

2

1 2

1

3 3

Figure 9: An example of scale selection. The top image shows cen- ter and surround at different sizes. The lower images show the re- sults of particular choices of scale selection. If scales are chosen too small, detail is lost. On the other hand, if scales are chosen too large, dark rings around luminance steps will form.

tion 7 makes thresholding V independent of absolute luminance level, while the 2^φa/s² term prevents V from becoming too large when V approaches zero.

Given a judiciously chosen scale for a given pixel, we observe that V1(x, y, sm) may serve as a local average for that pixel. Hence, the global tone reproduction operator of Equation 3 can be con- verted into a local operator by replacing L with V1 in the denomi- nator:

Ld(x, y) = L(x, y)

1 + V1(x, y, sm(x, y)) (9) This function constitutes our local dodging-and-burning operator.

The luminance of a dark pixel in a relatively bright region will sat- isfy L < V1, so this operator will decrease the display luminance Ld, thereby increasing the contrast at that pixel. This is akin to photographic “dodging”. Similarly, a pixel in a relatively dark region will be compressed less, and is thus “burned”. In either case the pixel’s contrast relative to the surrounding area is increased. For this reason, the above scale selection method is of crucial importance, as illustrated in the example of Figure 9. If sm is too small, then V1 is close to the luminance L and the local operator reduces to our global operator (s1 in Figure 9). On the other hand, choosing

sm too large causes dark rings to form around bright areas (s3 in the same figure), while choosing the scale as outlined above causes the right amount of detail and contrast enhancement without intro- ducing unwanted artifacts (s2 in Figure 9).

Using a larger scale sm tends to increase contrast and enhance edges. The value of the threshold ϵ in Equation 8, as well as the choice of φ in Equation 7, serve as edge enhancement parameters and work by manipulating the scale that would be chosen for each pixel. Decreasing ϵ forces the appropriate scale sm to be larger.

Increasing φ also tends to select a slightly larger scale sm, but only at small scales due to the division of φ by s². An example of the effect of varying φ is given in Figure 10.

A further observation is that because V1 tends to be smaller than L for very bright pixels, our local operator is not guaranteed to keep the display luminance Ld below 1. Thus, for extremely bright areas some burn-out may occur and this is the reason we clip the display luminance to 1 afterwards. As noted in section 2, a small amount of burn-out may be desirable to make light sources such as the sun look very bright.

In summary, by automatically selecting an appropriate neigh- borhood for each pixel we effectively implement a pixel-by-pixel dodging and burning technique as applied in photography [Adams 1983]. These techniques locally change the exposure of a film, and so darken or brighten certain areas in the final print.

4 Results

We implemented our algorithm in C++ and obtained the luminance values from the input R, G and B triplets with L = 0.27R + 0.67G + 0.06B. The convolutions of Equation 5 were computed using a Fast Fourier Transform (FFT). Because Gaussians are sepa- rable, these convolutions can also be efficiently computed in image space. This is easier to implement than an FFT, but it is somewhat slower for large images. Because of the normalization by V1, our method is insensitive to edge artifacts normally associated with the computation of an FFT.

The key value setting is determined on a per image basis, while unless noted otherwise, the parameter φ is set to 8.0 for all the images in this paper. Our new local operator uses Gaussian profiles s at 8 discrete scales increasing with a factor of 1.6 from 1 pixel wide to 43 pixels wide. For practical purposes we would like the Gaussian profile at the smallest scale to have 2 standard deviations overlap with 1 pixel. This is achieved by setting the scaling parameter α1 to 1/2√

2 ≈ 0.35. The parameter α² is 1.6 times as large.

The threshold ϵ used for scale selection was set to 0.05.

We use images with a variety of dynamic ranges as indicated throughout this section. Note that we are using the photographic definition of dynamic range as presented in Section 2. This results in somewhat lower ranges than would be obtained if a conventional computer graphics measure of dynamic range were used. However, we believe the photographic definition is more predictive of how challenging the tone reproduction of a given image is.

In the absence of well-tested quantitative methods to compare tone mapping operators, we compare our results to a representative set of tone reproduction techniques for digital images. In this section we briefly introduce each of the operators and show images of them in the next section. Specifically, we compare our new operator of Equation 9 with the following.

Stockham’s homomorphic filtering Using the observation that lighting variation occurs mainly in low frequencies and hu- mans are more aware of albedo variations, this method operates by downplaying low frequencies and enhancing high frequencies [Oppenheim et al. 1968; Stockham 1972].

Tumblin-Rushmeier’s brightness matching operator . A model of brightness perception is used to drive this global operator.

Right scale (s )

Surround Center

Center Surround

2

1 2

1

3 3

Ld(x, y) = L(x, y)

4 Results

Right scale (s )

Surround Center

Center Surround

2

1 2

1

3 3

Ld(x, y) = L(x, y)

4 Results

(51)

Dodging'and'burning'

(52)

Bilateral Filtering

(slides from Frédo Durand)

(53)

Fast Bilateral Filtering

for the Display of High-Dynamic-Range Images

Fr´edo Durand and Julie Dorsey

Laboratory for Computer Science, Massachusetts Institute of Technology

Abstract

We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer. Only the base layer has its contrast reduced, thereby preserving detail. The base layer is obtained using an edge-preserving filter called the bilateral filter. This is a non-linear filter, where the weight of each pixel is computed using a Gaussian in the spatial domain multiplied by an influence function in the intensity domain that decreases the weight of pixels with large intensity differences. We express bilateral filtering in the framework of robust statistics and show how it relates to anisotropic diffusion. We then accelerate bilateral filtering by using a piecewise-linear approximation in the intensity domain and appropriate subsampling. This results in a speed-up of two orders of magnitude. The method is fast and requires no parameter setting.

CR Categories: I.3.3 [Computer Graphics]: Picture/image generation—Display algorithms; I.4.1 [Image Processing and Com- puter Vision]: Enhancement—Digitization and image capture Keywords: image processing, tone mapping, contrast reduction, edge-preserving filtering,weird maths

1 Introduction

As the availability of high-dynamic-range images grows due to ad- vances in lighting simulation, e.g. [Ward 1994], multiple-exposure photography [Debevec and Malik 1997; Madden 1993] and new sensor technologies [Mitsunaga and Nayar 2000; Schechner and Nayar 2001; Yang et al. 1999], there is a growing demand to be able to display these images on low-dynamic-range media. Our visual system can cope with such high-contrast scenes because most of the adaptation mechanisms are local on the retina.

There is a tremendous need for contrast reduction in applications such as image-processing, medical imaging, realistic rendering, and digital photography. Consider photography for example.

A major aspect of the art and craft concerns the management of contrast via e.g. exposure, lighting, printing, or local dodging and burning [Adams 1995; Rudman 2001]. In fact, poor management of light – under- or over-exposed areas, light behind the main char- acter, etc. – is the single most-commonly-cited reason for rejecting

Figure 1: High-dynamic-range photography. No single global exposure can preserve both the colors of the sky and the details of the landscape, as shown on the rightmost images. In contrast, our spatially-varying display operator (large image) can bring out all details of the scene. Total clock time for this 700x480 image is 1.4 seconds on a 700Mhz PentiumIII. Radiance map courtesy of Paul Debevec, USC. [Debevec and Malik 1997]

Base Detail Color

Figure 2: Principle of our two-scale decomposition of the input intensity. Color is treated separately using simple ratios. Only the base scale has its contrast reduced.

photographs. This is why camera manufacturers have developed sophisticated exposure-metering systems. Unfortunately, exposure only operates via global contrast management – that is, it recenters the intensity window on the most relevant range. If the range of intensity is too large, the photo will contain under- and over-exposed areas (Fig. 1, rightmost part).

Our work is motivated by the idea that the use of high-dynamic- range cameras and relevant display operators can address these is- sues. Digital photography has inherited many of the strengths of film photography. However it also has the potential to overcome its limitations. Ideally, the photography process should be de- composed into a measurement phase (with a high-dynamic-range output), and a post-process phase that, among other things, man- ages the contrast. This post-process could be automatic or user- controlled, as part of the camera or on a computer, but it should take advantage of the wide range of available intensity to perform appropriate contrast reduction.

In this paper, we introduce a fast and robust operator that takes a high-dynamic-range image as input, and compresses the contrast while preserving the details of the original image, as introduced by Tumblin [1999]. Our operator is based on a two-scale decomposi- tion of the image into a base layer (large-scale features) and a detail

Lecture 15 Tone Mapping

Lecture 15

Tone Mapping

Oct. 8, 2015

Slide Credits:

Frédo Durand

Alexei Efros

The World is a High Dynamic Range (HDR)

Multiple exposure photography

• Sequentially measure all segments of the range

10 -6 10 6

10 -6 10 6

Real world

Picture

Low contrast

High dynamic range

log Exposure

Assuming unit radiance for each pixel

After adjusting radiances to obtain a smooth response

curve

P ixe l va lue

3

1 2

log Exposure

P ixe l va lue

Response curve

• Radiance is unknown, fit to find a smooth curve

Response Curve

Radiance is unknown, fit to find a smooth curve

HDR File Formats

Overview of HDR Encoding

Radiance RGBE and XYZE

Simple format with free source code

8 bits each for 3 mantissas and 1 exponent

76 orders of magnitude in 1% steps

Run-length encoding (20% avg. compression)

RGBE format does not cover visible gamut

Dynamic range at expense of accuracy

Color quantization not perceptually uniform

(145, 215, 87, 149) =

(145, 215, 87) * 2^(149-128) = (1190000, 1760000, 713000)

(145, 215, 87, 103) =

(145, 215, 87) * 2^(103-128) =

(0.00000432, 0.00000641, 0.00000259)

Radiance!format!(.pic,!.hdr,!.rad)!

Red Green Blue Exponent

32 bits/pixel

Radiance RGBE Format (.pic, .hdr)

ILM OpenEXR Format

ILM’s OpenEXR (.exr) ILM’s"OpenEXR"(.exr)"

• 6"bytes"per"pixel,"2"for"each"channel,"compressed"

sign exponent mantissa

• "Several"lossless"compression"opAons,"2:1"typical"

• "CompaAble"with"the"“half”"datatype"in"NVidia's"Cg"

• "Supported"naAvely"on"GeForce"FX"and"Quadro"FX"

"

• "Available"at"hQp://www.openexr.net/"

Tone Mapping

Overview

The Radiance map

The Radiance

Map

Linearly scaled to display device

The Radiance map

Linear mapping to display is

insufficient!

The Radiance

Map

Problem 2: Display the infromation

• Match limited contrast of the medium

• Preserve details

10 -6 10 6

10 -6 10 6

Real world

Picture

Low contrast

High dynamic range

Tone Mapping

Visual Matching

We do not need to reproduce the true radiance as

10 ^-6 10 ⁶

10 ^-6 10 ⁶

•  6"bytes"per"pixel,"2"for"each"channel,"compressed"

•  "Several"lossless"compression"opAons,"2:1"typical"

• "CompaAble"with"the"“half”"datatype"in"NVidia's"Cg"

• "Supported"naAvely"on"GeForce"FX"and"Quadro"FX"

• "Available"at"hQp://www.openexr.net/"

10 ^-6 10 ⁶

10 ^-6 10 ⁶

I = K _w