Model networks!!!primary visual cortex (V1):!!!Orientation and spatial frequency!

(1)

Model networks

!

! !

Primary visual cortex (V1):

!

Orientation and spatial frequency

!

Suggested reading:

!

•  Chapter 2.3 in Dayan, P. & Abbott, L., Theoretical Neuroscience, MIT Press, 2001.!

Primary visual cortex (V1) 2!

Primary visual cortex (V1)

!

The first cortical synapses for neurons carrying visual information are in the medial portion of the occipital lobe – Area 17 in Brodmann’s map. !

(2)

Lernen von orientierungs-selektiven rezeptiven Feldern

Receptive field in V1

LGN V1

Early experiments of Hubel and Wiesel

!

V1 has a retinotopic map of the visual field. Neigboring neurons process nearby locations in the visual field.! V1 cells are selective for!

•  Spatial location!

•  Color!

•  Orientation of edges!

•  Depth!

•  Motion!

Each position in such a map consists of a number of neurons that are selective for different properties.!

Hypercolumns

!

(3)

V1 simple cell receptive field

Zwei verschiedene Typen: On-Zentrum und Off-Zentrum

V1 simple cell receptive field

!

Data:!

2

!

"

=

!

=

0

2

!

"

=

#

)

sin(

)

cos(

!

y

!

x

u

=

+

!

g

(

x

,

y

)

=

1

2

"#

x

exp(

$

(

x

$

x

0

)

2

"#

x2

)

1

2

"#

y

exp(

$

(

y

$

y

0

)

2

"#

y2

)cos(

ku

$

%

)

Gabor-Fit

2

!

"

=

!

=

0

2

!

"

=

#

Gabor-Fit

V1 simple cell receptive field

!

Gabor Function:

!

)

sin(

)

cos(

!

y

!

x

u

=

+

!

g

(

x

,

y

)

=

1

2

"#

x

exp(

$

(

x

$

x

0

)

2

"#

x2

)

1

2

"#

y

exp(

$

(

y

$

y

0

)

2

"#

y2

)cos(

ku

$

%

)

(4)

Orientierte rezeptive Felder von ‚simple‘-Zellen in V1

Gabor-Fit

V1 simple cell receptive field

!

Parameters of the Gabor Function:

!

)

sin(

)

cos(

!

y

!

x

u

=

+

!

g

(

x

,

y

)

=

1

2

"#

x

exp(

$

(

x

$

x

0

)

2

"#

x2

)

1

2

"#

y

exp(

$

(

y

$

y

0

)

2

"#

y2

)cos(

ku

$

%

)

!

x

₀

,

y

₀

: Center of the RF

!

"

x

,

"

y

: Size of the RF

!

k

: Spatial frequency

!

"

: Phase of the cosine - function

!

"

: Orientation

Reizantwort einer ‚simple‘-Zelle in V1 als (lineare) Filteroperation: Korrelation zwischen Gaborfunktion und Luminanzfunktion

V1 simple cell response

!

The response of a V1 cell is often described by a linear filter

operation.

!

(5)

Raumfrequenzselektivität

Gabor-functions are band-pass filters, i.e. they are selective for a particular spatial frequency.

This spatial frequency is too high:

This spatial frequency is too low:

V1 simple cell response

!

The receptive field (RF, Gaborfunction) defines at each location the weight that will be multiplied with the local input. The response of a single cell to the stimulus is the weighted sum of the local responses:!

!

Stimulus (e.g. image)

=

f

(

x

,

y

)

,

(

RF

=

g

x

y

!

Response

L

=

f

(

x

,

y

)

g

(

x

,

y

)

dxdy

"# #

$

"# #

$

If we consider multiple identical cells that cover the visual space, the spatial distribution of all responses is described by a convolution of the image with the RF-function:!

! !

" " # " " #

#

=

(

'

,

'

)

(

,'

'

)

'

)

,

(

x

y

g

x

y

f

x

y

dx

dy

L

Filter operation

!

(6)

more realisitic

Discrete sinus-Gaborfunction:

(this would be more realistic) g(x‘-x) Simple stimulus: f(x) Example:

!

" " #

#

=

(

'

)

(

'

)

'

)

(

x

g

x

f

x

dx

L

Filter operation

!

L(x) Example:

!

" " #

#

=

(

'

)

(

'

)

'

)

(

x

g

x

f

x

dx

L

Filter operation

!

(7)

Sinus-Gaborfunction

enhances edges and reflects the first derivative of f(x) L(x) Example:

!

" " #

#

=

(

'

)

(

'

)

'

)

(

x

g

x

f

x

dx

L

Filter operation

!

Similarly:

Cosinus-Gaborfunction reflects the second derivative of f(x).

Originalbild gefiltert

Orig. First _derivative Second _derivative

Zur Kantendetektion kann man die Maxima der ersten Ableitung oder die Nullstellen der zweiten Ableitung verwenden

Filter operation

!

Nullstellen der zweiten Ableitung

Zero-crossings

!

(8)

Filter operation

- positive rates

!

Antwortverhalten einer echten V1 ‚simple‘-Zelle:

Filter operation

- positive rates

!

(9)

Filter operation

- positive rates

!

Zellen arbeiten als

‚Halbwellen-Rektifizierer‘ (half wave rectifiers): Multiple periods:

!

R

(

x

)

=

Max[

L

(

x

),0]

=

[

L

(

x

)

]

+

=

"

L

(

x

)

#

How do we obtain the negative responses ?

Filter operation

- excitatory and inhibitory cells

!

The positive values can be encoded by excitatory and the negative ones by inhibitory neurons.!

Excitatory neuron: Inhibitory neuron:

!

[

L

₀

(

x

)]

+

!

[

L

_"

(

x

)]

+

The responses of the neurons have to be subtracted to obtain the linear (including negative) responses.

=

)

(

0

x

L

!

Das gleiche funktioniert natürlich auch für Sinus-Gaborfunktionen:

!

L

_"_{/ 2}

(

x

)

=

[

L

_"_{/ 2}

(

x

)]

+

(10)

V1 complex cell models

!

0 = ! " =! /2 " =! " =3!/2

!

R

=

"

L

₀

(

x

)

#

2

+

"

L

_$ _{/ 2}

(

x

)

#

2

+

"

L

_$

(

x

)

#

2

+

"

L

_3$ _{/ 2}

(

x

)

#

2

The summation of four half-squared linear responses, which are shifted in phase by "/2 leads to a so called energy-neuron (Adelson & Bergen, 1985). Its

properties resemble those of complex cells in V1.!

2 2 / 2 0

(

x

)

L

(

x

)

L

+

_!

=

This is mathematically equivalent to the sum of the squared outputs of a quadrature pair. Two linear operators with the same amplitude

response, but with phases that are shifted by !/2

are called a quadrature pair.

0

=

! " =! /2

V1 complex cell models

!

Example: Response to a line with preferred orientation at the location a

within the receptive field! ₀

= ! "=! /2 2 2 / 2 0

(

x

)

L

(

x

)

L

R

=

+

_!

!

Stimulus

=

f

(

x

)

(

RF

=

g

x

!

" " #

=

f

x

g

x

dx

L

(

)

(

)

!

"

(

x

#

a

)

=

1 if

{

x

=

a

; 0 else

}

)

cos(

)

exp(

2

1

a

2

a

!

"

)

sin(

)

(

exp

2

1

x

2

x

!

"

)

cos(

)

(

exp

2

1

x

2

x

!

"

)

sin(

)

(

exp

2

1

a

2

a

!

"

2 2 / 2 0

(

x

)

L

(

x

)

L

R

=

+

_!

exp(

)

[

cos

(

)

sin

(

)

]

2

1

2 2 2 2

a

!

+

"

#

$

%

&

'

(

(11)

V1 complex cell models

!

Example: Response to an edge with preferred orientation 0 = ! " =! /2 _"₌_! "=3!/2

!

R

=

"

L

₀

(

x

)

#

2

+

"

L

_$_{/ 2}

(

x

)

#

2

+

"

L

_$

(

x

)

#

2

+

"

L

₃_$_{/ 2}

(

x

)

#

2 2 2 / 2 0

(

x

)

L

(

x

)

L

+

_!

=

V1 complex cell models

!

0 = ! " =! /2 _"₌_! "=3!/2

!

R

=

"

L

₀

(

x

)

#

2

+

"

L

_$_{/ 2}

(

x

)

#

2

+

"

L

_$

(

x

)

#

2

+

"

L

₃_$_{/ 2}

(

x

)

#

2 2 2 / 2 0

(

x

)

L

(

x

)

L

+

_!

=

(12)

Example: spatial frequency

!

Example: orientation

!

Populationskodierung:

Feature space!

Response

(13)

Example: orientation

!

Populationskodierung:

Feature space!

Response

!

References:

!

•  Adelson EH, Bergen JR. (1985) Spatiotemporal energy models for the perception of motion.

(14)

Model networks

!

! ! !

!

Motion

!

Suggested reading:

!

•  Chapter 2.4 in Dayan, P. & Abbott, L., Theoretical Neuroscience, MIT Press, 2001.!

•  Adelson EH, Bergen JR. (1985) Spatiotemporal energy models for the perception of

motion. J Opt Soc Am A. 2:284-99.!

Motion is correlation in space and time

!

This location at time t

This location at time t+ "#

We perceive motion if two locations are stimulated slightly offset in time

(15)

Two nearby photoreceptors

The signal of the first is delayed by "#

A coincidence-detector detects when both signals arrive at the same time

Motion detection by delay and correlation

( - " )

Comparison

Motion detection by orientation in space and time

A moving two-dimensional line is equivalent to a three-dimensional oriented plane (x,y,t). A receptive field that is oriented in x, y, and t, responds selectively to a particular motion direction. Thus, we can use the same model principles. Stimulus

Receptive field

(16)

!

Stimulus

=

f

(

x

,

y

)

,

(

RF

=

g

x

y

!

Response

L

=

f

(

x

,

y

)

g

(

x

,

y

)

dxdy

"# #

$

"# #

$

Bewegung

!

Stimulus

=

f

(

x

,

y

,

t

)

,

(

RF

=

g

x

y

t

!

Response

L

(

t

)

=

f

(

x

,

y

,

"

)

g

(

x

,

y

,

"

)

dxdyd

"

y=#$ $

%

x=#$ $

%

"=#$ t

%

Motion detection by orientation in space and time

!

Lineare Raum-Zeit-Filter sind selektiv für die Orientierung und Bewegungsrichtung eines Drift-Gitters

The response is maximal if the direction of motion and the orientation of the moving stimulus match the RF profile.

R ect ifi ca tio n

Motion detection by orientation in space and time

!

Example: Stimulation with a drifting pattern!

Response!

Time!

(17)

Motion is like orientation in (x, t), and a spatiotemporally oriented receptive field can be used to detect it.

Motion detection by orientation in space and time

!

(a-e) (x, t) plots of bars moving to the left or to the right at various speeds.

Motion detection by orientation in space and time

!

One may think of a spatiotemporal impulse response as being fixed, while the spatiotemporal stimulus slides beneath it as if pulled along on a strip. A spatiotemporally separable impulse response. The spatial and temporal impulse responses are shown along the margins. Their product is shown schematically in the center. !

The spatiotemporal impulse response is a weighting function that sums inputs at various positions and times to determine the present output.

(18)

To approximate spatiotemporally oriented Gabor functions, a number of separable filters, which are shifted in phase and time, can be summed to form an oriented Gabor.

Motion detection by orientation in space and time

!

Constructing spatiotemporally oriented impulse responses from pairs of separable ones:!

Two spatial and two temporal impulse responses are shown in a and b. The four spatiotemporal impulse responses are the products of two spatial and two temporal impulse responses. !

The ones across the bottom are sums and differences of those above. The result is a pair of leftward- and a pair of rightward-selective filters. Members of a pair are approximately in quadrature.

(19)

The method of reverse correlation

allows to determine the spatiotemporal structure of receptive fields. Stimulus: flashed bar.

By calculating from each response on backwards the (x,t) position of the bar one can determine the excitatory and inhibitory regions of the RF for each position in time.

Measuring RF’s

!

Dependent on the (x,t) position of the bar a response is elicited.

Response!

Time!

Response!

Time!

Modell

The RF‘s show indeed a space-time structure.

XT-Plot

(20)

Summary: Motion perception

!

References:

!

•  DeAngelis, GC, Ohzawa, I, Freeman, RD. (1995) Receptive-field dynamics in the central

visual pathways Trends in Neuroscience 18:451-458. !

(21)

Additional reading:

!

•  Rust NC, Schwartz O, Movshon JA, Simoncelli EP. (2005) Spatiotemporal elements of macaque v1 receptive fields. Neuron 46:945-56.!

•  Rust NC, Mante V, Simoncelli EP, Movshon JA (2006) How MT cells analyze the motion of

visual patterns. Nat Neurosci., 9:1421-31.!

Model networks

!

! ! !

!

gain normalization

!

Suggested reading:

!

•  Heeger DJ. (1992) Normalization of cell responses in cat striate cortex. Vis Neurosci.

(22)

Brain areas are organized in multiple layers.!

•  Gain normalization!

•  Soft winner-takes all!

•  Dynamic gain control!

Divisive normalization

!

From: Heeger DJ, Simoncelli EP, Movshon JA. (1996 ) Computational models of cortical visual processing. Proc Natl Acad Sci U S A. 93:623-7.!

!

E

_i

=

E

i

"

2

+

E

_j j

#

Divisive normalization

!

E

=

1

4

"

L

0

(

x

)

#

2

+

"

L

_$_{/ 2}

(

x

)

#

2

+

"

L

_$

(

x

)

#

2

+

"

L

₃_$_{/ 2}

(

x

)

#

2

[

]

Energy response:!

(23)

Additional reading:

!

•  Heeger DJ., Simoncelli, E.P., Movshon, J.A. (1996) Computational models of cortical visual processing. PNAS 93:623-627.!

Model networks

!

! ! !

!

Disparity

!

Suggested reading:

!

•  De Angelis (2000) Seeing in three dimensions: the neurophysiology of stereopsis. Trends

(24)

Disparity coding in V1

!

Fr

om: De Angelis (2000) Seeing in thr

ee dimensions: the neur

ophysiology of ster

eopsis.

Tr

ends in Cognitive Sci, 4:80-90.

!

Horizontal disparity. A top-down view of an observer who is looking straight ahead and fixating on point P. The plane of the image corresponds to a horizontal cross-section through the observerʼs head and eyes. The images of a nearby point, N, will have a horizontal binocular disparity denoted by the angle d. The large (Vieth-Müller) circle is the locus of points in space that have zero horizontal disparity.#

Model networks

!

! ! !

!

Color

!

Suggested reading:

!

•  Gegenfurtner, KR (2003) Cortical mechanisms of colour vision. Nat Rev Neurosci.,

(25)

From: Solomon & Lennie (2007) The machinery of colour vision. Nature Rev Neurosci.!

P-cells receive inputs from only L- and M-cones, and these inputs generally have opposite signs, which indicates that P-cells are important for red–green colour vision.!

Early stages of color processing

!

Early stages of color processing

!

Color vision starts with the

absorption of light by three types of photoreceptors (L, M, S) in the eye. !

The electrical signals generated by these photoreceptors go

through complicated circuitry that transforms the signals into three channels: one luminance and the other being color opponent, red-green and blue-yellow.!

(26)

Early stages of color processing

!

S-(L+M)

bistratified

konio

+ ! + + + ! S M L L-M

midget

!

parvo

L+M

parasol

magno

Retina:

LGN:

From: Thorsten Hansen!

Image: Color Vision, De Gruyter!

Early stages of color processing

!

Midget cells:

!

(27)

Models of double-opponent cells

!

From: „Spatial and Temporal Properties of Cone Signals in Alert Macaque Primary Visual Cortex“; Bevil R. Conway and Margaret S. Livingstone!

From: „The Orientation Selectivity of Color-Responsive Neurons in Macaque V1“, Elizabeth N. Johnson, Michael J. Hawken, and Robert Shapley!

Left: Proposed sensitivity profile for an orientation-selective double-opponent simple cell. Within each subregion, the L- and M-cones send signals that are opposite in sign, but are not precisely balanced in strength. Also, the spatial symmetry is no longer the same as for a center-surround neuron, but resembles the asymmetric or odd-symmetric spatial receptive fields of non-opponent cells. !

(28)

Double-opponent cells in V1

!

Receptive field of a red-on-center/green-on-surround

double-opponent cell recorded in alert macaque!

From: Conway (2001) Spatial Structure of Cone Inputs to Color Cells in Alert Macaque Primary Visual Cortex (V-1). Journal of Neuroscience,

21:2768–2783!

Double-opponent cells in V1

!

From: „The Orientation Selectivity of Color-Responsive Neurons in Macaque V1“, Elizabeth N. Johnson, Michael J. Hawken, and Robert Shapley!

Double-opponent simple cell from layer 2/3. !

The pseudocolor maps depict excitation to increments in red and excitation to decrements in blue.!

(29)

Summary of color in V1

!

Classical version:!

Color is processed in blobs and later combined with contour

information.!

New emerging version:!

From early on color and edge

information are processed together.!