Visual perception basics. Image aquisition system. P. Strumiłło

(1)

  

 P. Strumiłło

Image aquisition system

Visual perception basics

(2)

Light perception by humans

Humans perceive approx. 90% of information about the environment by means of visual system.

Efficiency of the human visual system is characterised by a number of features:

• the ability to resolve image details (θ=1’=1°/60=pi/10800); • the ability to discriminate between brightness levels

(contrast sensitivity); • colour perception;

(3)

retina 125×106 _receptors

Human

visual system

pupil Optic nerve cornea _iris Optic chaism Visual cortex lens Lateral geniculate nucleus

(4)

Structure of the

human eye

lens retina 125××××106 receptors Nerve  PWN Fovea Blind spot Visual axis iris

(5)

Distribution of rods and cones in the retina

 PWN Cones Cones Rods Rods Blind spot Angle No of cones/rods per mm2 50x50 um50x50 um   _Webvision

(6)

Eye convergence angle

Eye convergence angle Disparity in binocular visionDisparity in binocular vision

~10 m

Binocular vision

Perception of depth (distance)

(7)

 Denis Meredith Role of colours Role of colours in depth perception in depth perception

A

B

(8)

Electromagnetic spectrum

1018

1024 ₁₀22 ₁₀20 ₁₀16 ₁₀14 ₁₀12 ₁₀10 ₁₀8 ₁₀6 ₁₀4 ₁₀2

frequency [Hz]

gamma rays radio waves

X rays microwaves Infrared waves ultraviolet Visible Visible spectrum spectrum 400 500 600 700 [nm]

(9)

infrared

Spectral sensitivity characteristic

of the human eye

ulraviolet 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 µm 0 0 0.2 0.4 0.6 0.8 1 rods cones

(10)

Visual perception

Subjective brightness sensation assumes a logarithmic characteristic.

Human eye can percieve brightness in the range of 1010_.

Log [mL] -6 -4 -2 0 2 4 Eye adaptation range Local adaptation range Glare limit Glare limit

(11)

Contrast sensitivity (Weber fraction)

The ratio is termed the Weber fraction.

It reflects contrast sensitivity characteristc of the human eye.

2% I I ∆

I

3⋅10-1 ₃⋅₁₀1 ₃⋅₁₀3

I

I+∆ I [cd/m2_] I I ∆

(12)

The eye achieves maximum sensitivity for: I+

∆

I

≈

I₀ I I ∆

I

3 ₃⋅₁₀1 ₃⋅₁₀3 0

I

[cd/m2_] I+∆ I

I

3⋅102 I_o=3 I_o=30 I_o=300 I_o=3000

(13)

Image coded using 16 gray levels

 MIT

(14)

Mach bands

Subjective brightness Brightness intensity

(15)

(16)

Image aquisition system

Visual perception basics

(17)

Visual path of the image processing system

Visual path – a set optical and electronic elements converting

radiant energy into an electrical signal and imaging it using display devices.

Image formation Opto-electrical _conversion Visualisation 3D

Imaging sensor

(18)

Pros and cons:

• small hole → little light goes in • large hole → image blurring

Pinhole camera (camera obscura)

(19)

Cameras today

CCD sensor

Pros and cons:

• sharp, high-contrast image • geometric distortions

(20)

Image formation model

x y

Image plane of the imaging sensor 3D Image formation model (x,y) f(x,y) (

α,β

) 2D

(21)

Image formation model

Image – a 2-D light intensity function

f(x,y)>=0 reflecting light energy

distribution

)

,

(

)

,

(

)

,

(

x

y

i

x

y

r

x

y

f

=

∞ < < ( , ) 0 i x y 1 ) , ( 0 < r x y < - illumination (x,y)

reflectance coefficient at (x,y)

y

x (x,y)

f(x,y)

illumination: sunny day ~ 5000 cd/m2_,

cloudy day ~ 1000 cd/m2_{, full moon ~ 0.001 cd/m}2_,

(22)

For a linear process of energy acummulation in the image sensor plane:

( )

x

,

y

f

(

α

,

β

) (

h

x

,

y

,

α

,

β

)

d

α

d

β

f

∫ ∫

∞ ∞ −

=

h(.)

– is the impulse response of the system; in optical systems it is termed the point spread function of the system

(23)

If the point spread function is shift invariant, then the

image formation model is given by a convolution integral:

( )

x, y f

( ) (

α

,

β

h x

α

, y

β

)

d

α

d

β

f =

∫∫

− − ∞ ∞ −         ₊ − = 2 ₂ 2 2 y x exp ) y , x ( h σ

(24)

Sampling of 1-D signals

x f(x) ω F(ω) -Ω Ω x s(x) ∆x ω S(ω) -1/∆x 1/∆x s(x)f(x) ω S(ω)*F(ω) -Ω Ω . . . . . .

(25)

Assume the source image (analog image) features a limited Fourier bandwidth

ω

_maxx

ω

_maxy

−ω

_maxy

−ω

_maxx

ω

_y

ω

_x

(

x

,

y

)

F

ω

Sampling of 2-D signals

(26)

( )

∑ ∑

−

(

)

= − =

−

∆

−

∆

=

1 0 1 0

,

M i N k

y

k

y

x

i

x

y

x

S

δ

Image sampling function:

and a sampled image:

( ) ( ) ( )

(

) (

)

∑ ∑

− = − =

∆

−

∆

−

∆

=

1 0 1 0

,

M i N k s

y

k

y

x

i

x

y

k

x

i

f

y

x

S

y

x

f

y

x

f

δ

∆

x

∆

y

(27)

(

)

∑ ∑

−

(

)

= − =

−

∆

−

∆

=

1 0 1 0

,

1

,

M i N k x x y y y x s

F

i

k

y

x

F

ω

Fourier spectrum of the sampled image:

where: y x y x = _∆ ∆ = _∆ ∆

ω

1 ,

ω

1

∆ω

_x

∆ω

_y

(28)

∆ω

_x

∆ω

_y

ω

_maxy

ω

_maxx _image bandwidth x x ∆ = ∆ < Ω 2 1 2 max

ω

(29)

Aliasing distortion - example

100 dpi

(dots per inch) 500 dpi

(30)

Image acquisition

Image acquisition is the process of converting light energy radiating from image scene points into an electrical signal (suitable for storing or transmission).

Image acquisition devices: • CCD camera

• Video camera • Scanner

(31)

There are two basic schems of converting optical images into electrical signals:

• without accumulation of photo-charges (eg. optical scanner),

• with accumulation of photo-charges (np. vidicon, CCD array)

(32)

Imaging sensor (no photo-charges)

+ – s(x,y) photodiode focusing screen R scanning direction

(33)

CCD array (accumulation of photo-charges)

Image formation is based on the internal photo-electric phenomenon

U_F<0

Φ

n

Capacitor cell

electrical potential well

~10µm

(34)

The Bayer matrix

Raw CCD Format

Calculate RGB image by interpolating colour components from the Bayer matrix

N

M

N

(35)

Pixim – Digital Pixel System (DPS)

A/D converter for each pixel (no charge couplings)

(36)

CMOS image sensors

Pros:

• cheap technology (used for fabricating memory and CPU modules), • low power consumption (100 times!)

• random access to pixel regions (block image processing) • no „charge leaking” typical for CCD technology

• on-chip analog-to-digital conversion and signal processing

OmniVision

Cons:

• more susceptibel to noise than CCD

• lower light sensitivity due to many transistors used

(37)

Monochrome TV standards

European CCIR standard: (625 (575) lines, line display

time 64us, 50 half-images per sec., 1Vpp, 75Ω, signal

American RS170 standard: (525 (484) lines, line display

time 63,5 us, 60 half-images per sec.,1.4 Vpp, 75Ω signal

American RS-343 standard”: (875 lines, 60 half-images,

(38)

TV CCIR standard

Composite video signal

peak white level

blanking level sync level odd lines even lines 625/575 lines video signal

horizontal sync pulse

4 64 µs 0 V (DC) 0.7 V -0.3 V 3 52 µs

(39)

Resolution:

RS-170A: 580 horizontal TVL, 350 vertical TVL; CCIR: 560 horizontal TVL, 450 vertical TVL Synchronization: Crystal/H&V/Asynchronous, standard Shutter: 1/60 to 1/10,000 AGC: 20 dB Integration: 2 - 16 Fields Sensitivity:

Full video, No AGC: 0.65 lux; 80% video, AGC on: 0.04 lux; 30% video, AGC on: 0.008 lux

S/N Ratio (Gamma 1, gain 0 dB): 55 dB

Specification Highlights Imager: 1/2" interline transfer CCD Picture Elements: RS-170A: 768 (H) x 494 (V); CCIR: 752 (H) x 582 (V)

Pixel Cell Size:

RS-170A: 8.4 µm (H) x 9.8 µm (V);

CCIR: 8.6 µm (H) x 8.3 µm (V)

(40)

CCD image sensors characteristics

small size,

robust to mechanical vibrations (70 G), no geometrical distortions,

low supply voltage (12 V, 1.4W), SNR ~70 dB,

linear (gamma coefficient),

no intra-frame photo-charge accumulation, high resolution,

reliable cheap

(41)

Image frame grabber

Matrox CronosPlus

Video capture board for PCI captures from NTSC,

PAL, RS-170 and CCIR video sources,

connect up to 4 CVBS or 1 Y/C trigger input, 7 TTL auxiliary I/Os,

32-bit/33MHz PCI-bus master _{Matrox ®}

Software is sold

separately, includes e.g., Matrox ®

Imaging Library for Microsoft® Windows®