GPU Shading and Rendering: Introduction & Graphics Hardware

(1)

Introduction & Graphics Hardware

Marc Olano

Computer Science and Electrical Engineering University of Maryland, Baltimore County

SIGGRAPH 2005

(2)

Schedule

Shading Technolgy

8:30 Intro/Hardware(Olano)

9:25 Compilers(Bleiweiss)

Shading Languages 10:30 GLSL(Olano)

10:55 Cg^(Kilgard) 11:20 HLSL(Sander)

11:45 Sh(McCool)

GPU Rendering 1:45 Rendering

Algorithms^(Hart) 2:35 GPU Production

Rendering^(Gritz) Hardware Systems 3:45 ATI(Sander)

4:25 NVIDIA^(Kilgard) 5:05 Panel Q&A(all)

(3)

Introdution

(4)

What is a GPU?

• Graphics Processing Unit

• Graphics accelerator

• Parallel processing unit

• We’re doing graphics, what is it good for?

• Better real-time graphics

• Faster non-real-time graphics

(5)

(6)

What is a GPU?

(7)

(8)

What is Shading?

• What color are the pixels

• Programmable

• Flexible Appearance

• Arbitrary computation

• Procedural

• Simple procedures

• High-level language

(9)

• Programmable

• Procedural

(10)

What is Shading?

• Programmable

• Procedural

(11)

• Programmable

• Procedural

(12)

What is Shading?

• Programmable

• Procedural

(13)

• Programmable

• Procedural

(14)

What is Shading?

• Programmable

• Procedural

(15)

• More realistic appearance

• Bump mapping, Anisotropic, Precomputed radiance transfer, ...

• Non-realistic appearance

• Cartoon, Sketch, Illustration, ...

• Animated appearance

• Skinning, Water, Clouds, ...

• Visualization

• Data on surfaces, Volume rendering, ...

(16)

Some examples

• Visualization

(17)

• Visualization

(18)

Some examples

• Visualization

(19)

• The rest of the problem!

• In our case, using GPU for other than polygon rendering

• Curved surfaces

• Ray tracing

• Point based rendering

• ...

(20)

What is Rendering?

• Curved surfaces

• Ray tracing

• ...

(21)

• Curved surfaces

• Ray tracing

• ...

(22)

Non-Real Time vs. Real-Time

Real-Time Non-Real-Time

Tens of frames per second Seconds to hours per frame Thousand instruction shaders Thousands line shaders Limited computation, texture,

memory, ...

“Unlimited” computation, texture, memory, ...

(23)

memory, ...

(24)

Non-Real Time vs. Real-Time

memory, ...

(25)

memory, ...

(26)

How is this possible?

• GPUs are programmable!

• Per-vertex programs

• Per-fragment programs

(27)

• Pixel-Planes 5 [Rhoades et al., 1992]

• PixelFlow/pfman [Olano and Lastra, 1998]

• RTSL [Proudfoot et al., 2001]

• Sh [McCool and Toit, 2004]

• (BrookGPU) [Buck et al., 2004]

(28)

Research Languages

(29)

(30)

Research Languages

(31)

(32)

Commercial Languages

• GL or DX low-level

• OpenGL Shading Language

• DirectX HLSL

• NVIDIA Cg

(33)

• DirectX HLSL

• NVIDIA Cg

(34)

Commercial Languages

• DirectX HLSL

• NVIDIA Cg

(35)

• DirectX HLSL

• NVIDIA Cg

(36)

Ignoring Hardware Differences

Part II

Graphics Hardware

(37)

Outline

Ignoring Hardware Differences Simplified Models

RenderMan Hardware

(38)

Machine Complexity

• Graphics machines are complex

• User does not want to know

• How machine does what it does

• Tons of machine-specific differences

• Answer:

• Simple model of machine

• High-level language for procedures

• Well-defined procedure input & output

• System connects procedures

(39)

Machine Complexity

• Answer:

(40)

Machine Complexity

• Answer:

(41)

Machine Complexity

• Answer:

(42)

Machine Complexity

• Answer:

(43)

Machine Complexity

• Answer:

(44)

Machine Complexity

• Answer:

(45)

Simplified Machine

• User’s mental model

• Hide details

• Device independent

• Procedural stages

(46)

Simplified Machine

• Hide details

(47)

Simplified Machine

• Hide details

(48)

Simplified Machine

• Hide details

(49)

RenderMan Model

Model

Transform Displacement

Shade Light

Volume Imager

• “Abstract” interface

• Blocks = procedures

• Block interfaces well defined

• Connections

• Inputs & outputs don’t have to match

• System handles conversion

(50)

RenderMan Model

Model

Shade Light

Volume Imager

• “Abstract” interface

• Blocks = procedures

• Block interfaces well defined

• Connections

• Inputs & outputs don’t have to match

• System handles conversion

(51)

RenderMan Shader types

Model

Shade Light

Volume Imager

(52)

RenderMan Shader types

Model

Transform

Displacement

Shade Light

Volume Imager

(53)

RenderMan Shader types

Model

Shade Light

Volume Imager

(54)

RenderMan Shader types

Model

Shade Light

Volume Imager

(55)

RenderMan Shader types

Model

Shade Light

Volume Imager

(56)

RenderMan Shader types

Model

Shade Light

Volume Imager

(57)

RenderMan Shader types

Model

Shade Light

Volume Imager

Katsuaki Hiramitsu www.edit.ne.jp/˜katsu/

(58)

RenderMan Model

Model

Shade Light

Volume Imager

• What it says:

• Input and output of each block

• What each block should do

• What it doesn’t say:

• Order or grouping of processing

(59)

RenderMan Model

Model

Shade Light

Volume Imager

• What it says:

• Order or grouping of processing

(60)

RenderMan REYES

Model

Shade Light

Volume Imager

Object

Object / Sample

Image / Pixel

(61)

RenderMan Ray Tracing

Model

Shade Light

Volume Imager

Ray

Image / Pixel

(62)

RenderMan

SGI Multi-pass RenderMan

Model

Shade Light

Volume Imager

Object

Object / Pass / Pixel

Image / Pixel

(63)

Hardware Model

Vertex

Fragment Texture

• Vertex shading

• Transform

• Procedural transformation

• Skinning

• Shade

• Per-vertex shading

• Computed texture coordinates

(64)

Hardware Model

Vertex

Fragment Texture

• Fragment shading

• Per-fragment shading

• Computed and dependent texture

(65)

Hardware Model

Vertex

Fragment Texture

• Render to texture

• Rendered shadow &

environment maps

• Multi-pass fragment shading [Proudfoot et al., 2001]

(66)

Hardware Model

Vertex

Fragment Buffers

• Render to vertex array / buffer objects

• Geometry images [Gu et al., 2002]

• Multi-pass vertex shading

• Merge vertex & fragment capabilities

(67)

Hardware Model

Vertex

Fragment Buffers

• Vertex texture

• Texture-based vertex displacement

• Tabulated functions

(68)

Hardware Model

Vertex

Fragment Buffers

• It’s all about the memory

• What it says:

• Vertex processing order

• Fragment processing order

• Interleaving of vertex and fragment

(69)

Hardware Model

Vertex

Fragment Buffers

• What it says:

(70)

Hardware Model

Vertex

Fragment Buffers

• What it says:

(71)

No, but really, what’s in there?

Vertex

Fragment Buffers

• Some other stuff,

• Parallelism,

• And more parallelism

(72)

No, but really, what’s in there?

Vertex

Fragment Buffers

Triangle

Zbuf / Blend

• Parallelism,

(73)

No, but really, what’s in there?

Vertex

Fragment Buffers

Triangle

Zbuf / Blend

Ver Ver

Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragment Fragment ent ent ent ent

Ver Ver Vertex Vertex Vertex Vertex Vertex Vertex tex tex

• Parallelism,

(74)

No, but really, what’s in there?

Vertex

Fragment Buffers

Triangle

Zbuf / Blend

Ver Ver

Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragm Fragment Fragment ent ent ent ent

Ver Ver Vertex Vertex Vertex Vertex Vertex Vertex tex tex

• Parallelism,

(75)

Part III

Noise

(76)

What is this Noise? Perlin noise Modifications

Outline

What is this Noise?

Perlin noise Modifications

(77)

Why Noise?

• Introduced by [Perlin, 1985]

• Heavilyused in production animation

• Technical Achievement Oscar in 1997

• “Salt,” adds spice to shaders

(78)

Why Noise?

• Introduced by [Perlin, 1985]

• Heavilyused in production animation

• Technical Achievement Oscar in 1997

• “Salt,” adds spice to shaders

+ =

(79)

Noise Characteristics

• Random

• No correlation between distant values

• Repeatable/deterministic

• Same argument always produces same value

• Band-limited

• Most energy in one octave (e.g. between f & 2f)

1 2 3 4 5 6 7

(80)

Noise Characteristics

• Random

• Band-limited

1 2 3 4 5 6 7

(81)

Noise Characteristics

• Random

• Band-limited

1 2 3 4 5 6 7

(82)

Gradient Noise

• Original Perlin noise [Perlin, 1985]

• Perlin Improved noise [Perlin, 2002]

• Lattice based

• Value=0 at integer lattice points

• Gradientdefined at integer lattice

• Interpolate between

• 1/2 to 1 cycle each unit

1 2 3 4 5 6 7 1 2 3 4 5 6 7

Original Improved

(83)

Gradient Noise

• Lattice based

1 2 3 4 5 6 7 1 2 3 4 5 6 7

Original Improved

(84)

Gradient Noise

• Lattice based

1 2 3 4 5 6 7 1 2 3 4 5 6 7

Original Improved

(85)

Value Noise

• Lattice based

• Valuedefined at integer lattice points

• At most 1/2 cycle each unit

• Significant low-frequency content

• Easy hardware implementation with lower quality

1 2 3 4 5 6 7 1 2 3 4 5 6 7

Linear Interp Cubic Interp

(86)

Value Noise

• Lattice based

1 2 3 4 5 6 7 1 2 3 4 5 6 7

(87)

Value Noise

• Lattice based

1 2 3 4 5 6 7 1 2 3 4 5 6 7

(88)

Value Noise

• Lattice based

1 2 3 4 5 6 7 1 2 3 4 5 6 7

(89)

Hardware Noise

• Value noise

• PixelFlow [Lastra et al., 1995]

• Perlin Noise Pixel Shaders [Hart, 2001]

• Noise textures

• Gradient noise

• Hardware [Perlin, 2001]

• Complex composition [Perlin, 2004]

• Shader implementation [Green, 2005]

(90)

Outline

What is this Noise?

Perlin noise Modifications

(91)

Noise Details

• Subclass of gradient noise

• Original Perlin

• Perlin Improved

• All of our proposed modifications

(92)

Find the Lattice

• Lattice-based noise: must find nearest lattice points

• Point ~p = (~p^x, ~p^y, ~p^z)

• has integer lattice location

~pi = (⌊~p^x⌋, ⌊~p^y⌋, ⌊~p^z⌋) = (X , Y , Z )

• and fractional location in cell

~p_f = ~p − ~pi = (x, y , z)

(93)

Find the Lattice

• Point ~p = (~p^x, ~p^y, ~p^z)

~pi = (⌊~p^x⌋, ⌊~p^y⌋, ⌊~p^z⌋) = (X , Y , Z )

~p_f = ~p − ~pi = (x, y , z)

(94)

Find the Lattice

• Point ~p = (~p^x, ~p^y, ~p^z)

~pi = (⌊~p^x⌋, ⌊~p^y⌋, ⌊~p^z⌋) = (X , Y , Z )

~p_f = ~p − ~pi = (x, y , z)

X

Y

(95)

Find the Lattice

• Point ~p = (~p^x, ~p^y, ~p^z)

~pi = (⌊~p^x⌋, ⌊~p^y⌋, ⌊~p^z⌋) = (X , Y , Z )

~p_f = ~p − ~pi = (x, y , z)

x X y

Y

(96)

Gradient

• Random vector at each lattice point is a function of ~pi

g (~pi)

• A function with that gradient grad(~p) = g (~p_i) • ~p_f

= g^x(~pi) ∗ x + g^y(~pi) ∗ y + g^z(~pi) ∗ z

(97)

Gradient

g (~pi)

(98)

Gradient

g (~pi)

(99)

Interpolate

• Interpolate nearest 2ⁿ gradient functions

• 2D noise(~p) is influenced by

~p_i+ (0, 0); ~p_i + (0, 1); ~p_i + (1, 0) ; ~p_i + (1, 1)

• Linear interpolation

• lerp(t, a, b) = (1 − t) a + t b

• Smooth interpolation

• fade(t) =

3t²− 2t³ for original noise 10t³− 15t⁴+ 6t⁵ for improved noise

• flerp(t) = lerp(fade(t), a, b)

(100)

Interpolate

~p_i+ (0, 0); ~p_i + (0, 1); ~p_i + (1, 0); ~p_i + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(101)

Interpolate

~pi+ (0, 0); ~pi + (0, 1); ~pi + (1, 0); ~pi + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(102)

Interpolate

~pi+ (0, 0) ; ~pi + (0, 1); ~pi + (1, 0); ~pi + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(103)

Interpolate

~pi+ (0, 0) ; ~pi + (0, 1) ; ~pi + (1, 0); ~pi + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(104)

Interpolate

~pi+ (0, 0) ; ~pi + (0, 1) ; ~pi + (1, 0) ; ~pi + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(105)

Interpolate

~p_i+ (0, 0) ; ~p_i + (0, 1) ; ~p_i + (1, 0) ; ~p_i + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(106)

Interpolate

~p_i+ (0, 0) ; ~p_i + (0, 1) ; ~p_i + (1, 0) ; ~p_i + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(107)

Interpolate

~p_i+ (0, 0) ; ~p_i + (0, 1) ; ~p_i + (1, 0) ; ~p_i + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(108)

Interpolate

~p_i+ (0, 0) ; ~p_i + (0, 1) ; ~p_i + (1, 0) ; ~p_i + (1, 1)

• lerp(t, a, b) = (1 − t) a + t b

• fade(t) =

(109)

Hash

• n-D gradient function built from 1D components g(~pi)

• Both original and improved use a permutation table hash

• Original: g is a table of unit vectors

• Improved: g is derived from bits of final hash

(110)

Hash

• n-D gradient function built from 1D components g(hash(X , Y , Z ))

(111)

Hash

• n-D gradient function built from 1D components g(hash(Z + hash(X , Y )))

(112)

Hash

• n-D gradient function built from 1D components g(hash(Z + hash(Y + hash(X ))))

(113)

Hash

• n-D gradient function built from 1D components g(hash(Z +hash(Y +hash(X ))))

• Both original and improved use a permutation tablehash

(114)

Hash

• n-D gradient function built from 1D components g(hash(Z + hash(Y + hash(X ))))

(115)

Outline

What is this Noise?

Perlin noise

Modifications Corner Gradients Factorization Hash

(116)

Gradient Vectors of n-D Noise

• Original: on the surface of a n-sphere

• Found by hash of ~pi into gradient table

• Improved: at the edges of an n-cube

• Found by decoding bits of hash of ~pi

(117)

Gradient Vectors of n-D Noise

• Original: on the surface of a n-sphere

• Found by hash of ~pi into gradient table

• Improved: at the edges of an n-cube

• Found by decoding bits of hash of ~pi

(118)

Gradients of noise(x,y,0) or noise(x,0)

• Why?

• Cheaper low-D noise matches slice of higher-D

• Reuse textures (for full noise or partial computation)

• Original: new short gradient vectors

• Improved: gradients in new directions

• Possibly including 0 gradient vector!

(119)

Gradients of noise(x,y,0) or noise(x,0)

• Why?

(120)

Gradients of noise(x,y,0) or noise(x,0)

• Why?

(121)

Solution?

• Observe: use gradient function, not vector alone grad = g^x x + g^y y + g^z z

• In any integer plane, fractional z = 0 grad = g^x x + g^y y + 0

• Any choice keeping projection of vectors the same will work

• Improved noise uses cube edge centers

• Instead use cube corners!

(122)

Solution?

(123)

Solution?

(124)

Solution?

(125)

Solution?

(126)

Corner Gradients

• Simple binary selection from hash bits

±x, ±y , ±z

• Perlin mentions “clumping” for corner gradient selection

• Not very noticeable in practice

• Already happens in any integer plane of improved noise

(127)

Corner Gradients

• Simple binary selection from hash bits

±x, ±y , ±z

• Perlin mentions “clumping” for corner gradient selection

• Not very noticeable in practice

• Already happens in any integer plane of improved noise

Edge Centers Corner

(128)

Separable Computation

• Like to store computation in texture

• Texture sampling 3-4x highest frequency

• 1D & 2D OK size, 3D gets big, 4D impossible

• Factor into lower-D textures

• (e.g. write noise(~p^x, ~p^y, ~p^z) as several x/y terms) noise(~p^x, ~p^y, ~p^z) = flerp(z,+ ∗ z

+ ∗ (z − 1))

(129)

Separable Computation

+ ∗ (z − 1))

(130)

Separable Computation

+ ∗ (z − 1))

(131)

Separable Computation

+ ∗ (z − 1))

(132)

Separable Computation

• (e.g. write noise(~p^x, ~p^y, ~p^z) as several x/y terms) noise(~p^x, ~p^y, ~p^z) = flerp(z,xyz-term+xyz-term ∗ z

xyz-term+xyz-term ∗ (z − 1))

(133)

Separable Computation

• (e.g. write noise(~p^x, ~p^y, ~p^z) as several x/y terms)

noise(~p^x, ~p^y, ~p^z) = flerp(z,xy-term(Z₀)+xy-term(Z₀) ∗ z xy-term(Z₁)+xy-term(Z₁) ∗ (z − 1))

(134)

Factorization Details

noise(~p) = flerp(z,zconst(~p^x, ~p^y,Z₀)+zgrad(~p^x, ~p^y,Z₀) ∗ z, zconst(~p^x, ~p^y,Z₁)+zgrad(~p^x, ~p^y,Z₁) ∗ (z − 1))

• With nested hash,

zconst(~p^x, ~p^y,Z₀)= zconst(~p^x, ~p^y+ hash(Z₀)) zgrad (~p^x, ~p^y,Z₀)= zgrad (~p^x, ~p^y+ hash(Z₀))

• With corner gradients, zconst = noise!

(135)

Factorization Details

noise(~p) = flerp(z,zconst(~p^x, ~p^y,Z₀)+zgrad(~p^x, ~p^y,Z₀) ∗ z, zconst(~p^x, ~p^y,Z₁)+zgrad(~p^x, ~p^y,Z₁) ∗ (z − 1))

• With nested hash,

zconst(~p^x, ~p^y,Z₀)= zconst(~p^x, ~p^y+ hash(Z₀)) zgrad (~p^x, ~p^y,Z₀)= zgrad (~p^x, ~p^y+ hash(Z₀))

• With corner gradients, zconst = noise!