/SOLUTIONS/ where a, b, c and d are positive constants. Study the stability of the equilibria of this system based on linearization.

(1)

Technische Universiteit Eindhoven Faculteit Elektrotechniek NIET-LINEAIRE SYSTEMEN / NEURALE NETWERKEN

(5P060)

gehouden op donderdag 21 maart 2007, van 9:00 tot 12:00 uur. Dit examenonderdeel bestaat uit 8 opgaven.

/SOLUTIONS/

Problem 1 (1 point)

Consider the system:

2 2 1 2 2 1 1 1 dx x cx x x bx ax x − = − = & & (1) where a, b, c and d are positive constants.

Study the stability of the equilibria of this system based on linearization.

Solution

Equilibrium points: 0 ) ( 0 ) ( 0 0 2 1 2 1 2 2 1 2 1 1 = − = − ⇒ = − = − x d cx bx a x dx x cx x bx ax

Real solutions of this system are

{

T T

}

T _d _c _b _a x x * *] [0 0] , [ / / ] [ ₁ ₂ = Jacobian matrix: ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − − − = d cx cx bx bx a J 1 2 1 2

Stability of equilibrium points

• for [0 0]T: _⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − = d a J 0 0

(2)

for [d/c b/a]T_: ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ₋ = 0 0 b ac c bd

J , λ1,2 =±j ad ⇒ eigenvalues are on the imaginary axis jω and because of Hartman-Grobman theorem can’t study the stability of the equilibrium point [d/c b/a]T based on linearization.

Problem 2 (1.5 points)

Consider the following systems:

1 2 3 1 1 x x x x = − = & & _(2a) 1 2 2 3 1 1 x x x x x = − − = & & _(2b)

These systems have an equilibrium at the origin. The second system could be considered identical to the first one, but controlled by a signal u=-x2.

(a) Is the origin of system (2a) stable in sense of Lyapunov?

(b) Show that for system (2b) the origin is globally asymptotically stable. Hint (especially for the second system): try the Lyapunov function

(

2 2

)

1 2

1

2 x +x and apply the Lassale principle.

Solution

(a)

Jacobian matrix of 2a):

⎥ ⎦ ⎤ ⎢ ⎣ ⎡− = 0 1 0 3 2 1 x J

Stability of the origin of 2a) for [0 0]T: _⎥ ⎦ ⎤ ⎢ ⎣ ⎡ = 0 1 0 0

J , λ₁=0 ⇒ one eigenvalue is on the imaginary axis jω and because of Hartman-Grobman theorem can’t study the stability of the origin based on indirect Lyapunov method (linearization)

We can study the stability by direct inspection. It is easy to observe that the right hand side of the first equation of 2a) do not contains x2, and thus dx1/dt depends only on x1. If x1>0, dx1/dt<0, if

x1<0, dx1/dt>0 and if x1=0, dx1/dt=0. Therefore the x1 coordinate of system trajectory tends to 0.

(3)

change its sign and thus the second coordinate x2 of the system trajectory either increases or

decreases. Therefore system 2a) is not stable in the sense of Lyapunov, because the stability in sense of Lyapunov implies that if the initial point of the system trajectory is near enough to the origin, the system trajectory remains close to the origin.

(b) Let

(

2

)

2 2 1 2 1 2 1 ) , (x x x x V = + 0 ) , ( 4 1 2 1 2 1 4 1 2 2 1 1 2 1 x =x x +x x =−x −x x +x x =−x ≤ x

V& _& _& for every 2

2

1, )

(x x ∈R

Therefore the origin of system (2) is globally stable in the sense of Lyapunov, because V(x₁,x₂) is negative semi defined (V&(x₁,x₂) is 0 not only at the origin but for {(x1, x2)| x1=0 and ∀x2}).

At (0,0), V& =0 i.e. we can claim asymptotic stability if can apply the Lasalle’s theorem. Let {( , ) 2; 0} {( ₁, ₂)| ₁ 0, ₂}

2

1 x R V x x x x

x

S= ∈ &= = = ∀

But if x₁=0⇒x_&₁ =0, x_&₂ =0⇒x₂ =0

Letting M be the largest invariant set in S, we have M ={(0,0)}, i.e. only the origin and by the Lasalle’s theorem, the origin is globally asymptotically stable.

Problem 3 (1 point)

Draw phase portraits of the following one-dimensional system as the parameter μ changes:

2 3 5 4 5 x&= − μ x+ μx −x (3)

Solution

) ( ) 5 4 ( 2 _x2 _x4 _f _x x x&= − μ + μ − = _μ Equilibra * ₌{0, _± _μ, _±2 _μ} x 4 2 2 5 15 4 x x f Dx μ =− μ + μ − zero eigenvalue at (0,0) Stability: * 2 4 0⇒ μ =− μ = D f x x , stable ∀μ∈R * ₂ _μ ₂₄_μ2 μ =− ⇒ ± = D f x x , stable ∀μ>0 * _μ ₆_μ2 μ = ⇒ ± = D f x x , unstable ∀μ>0

(4)

Phase portrait

Problem 4 (1.5 points)

Consider the nonlinear system in the feedback form as given in the figure below

where the parameters of the nonlinear function NL are h=1, β=1 and the transfer function of the linear part is

2 1 ( ) ( 1) g s s s = +

Use describing function analysis to predict whether or not the system will oscillate. If yes, what is the predicted amplitude a and frequency ω0 of oscillation (amplitude and frequency of the

error signal e)?

Note: The describing function corresponding to the nonlinear function NL is N a( ) 4h a β π = +

y

NL

g(s)

e

v

+

_

r=0

μ (0, 0) x ) 5 . 0 1 ( * − = μ x ) 5 . 0 1 ( * − − = μ x 0 * = x ) 5 . 0 1 ( *₌ _μ ₊ x ) 5 . 0 1 ( * + − = μ x μ 2 = x_& μ = x& 0 = x_& μ = x_& μ 2 = x& μ μ 2 = x& μ = x& 0 = x_& μ − = x_& μ 2 − = x_&

e

v

h

NL

(

single-valued skew-symmetric function

)

-h

slope =

β

> 0

(5)

Solution

Oscillations appear if equation 1 ( ) ( ) g j N a ω = −

has real solutions with respect to amplitude a and frequency ω of the error signal e=a.sin(ωt)

2 3 2 1 1 ( ) ( 1) 2 g s s s s s s = = + + + 3 2 3 2 2 3 1 1 1 ( ) ( ) 2( ) 2 2 ( ) g j j j j j j j

ω

ω ω

= = = + + − − + − + − Because 1 ( ) N a − is real

(

)

3 0 0 0 Im ( ) 0 ( ) 0 1 / ; 0.159 2 g jω ω ω ω ω rad s f ω Hz π = ⇒ − = ⇒ = = = =

Thus at the intersection point of g j( ω) and 1 ( ) N a − , ω=ω0 and 0 2 0 1 1 ( ) 2 2 g jω ω − = = − , N a( ) 4 1 a π = + Hence from 0 1 ( ) ( ) g j N a ω = − follows 1 4 4 1,27 2 4 a a a a π _π π π − = − ⇒ = ⇒ = = +

Therefore the amplitude and frequency of the oscillating error signal e are a=1.27 and 0 1rad s/

(6)

Problem 5 (1.5 points)

Consider a two input perceptron without bias input. The output activation function of the perceptron is 2 2 1 2 1 1 1 if 0 ( ) 0 if 0 j j j j j j j j j w x y w x w x φ = = = ⎧ _≥ ⎪ ⎪ = _{= ⎨} ⎪ _< ⎪⎩

∑

Given the following input-output training pairs:

1 1 2 0 (1) , (1) 1 (2) , (2) 1 (3) , (3) 1 (4) , (4) 1 2 d 1 d 0 d 1 d − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ =_{⎢ ⎥} = =_{⎢ ⎥} = =_{⎢ ⎥} = − =_{⎢ ⎥} = − − ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ x x x x

Apply the perceptron learning rule to adjust the weights w1 and w2 starting with initial weights

vector 1 2 1 1 w w ⎡ ⎤ ⎡ ⎤ =_{⎢ ⎥ ⎢ ⎥}= − ⎣ ⎦ ⎣ ⎦ w .

Solution

Adjustment the weights

if d = 0 and φ(wTx) = 1 then w := w – x if d = 1 and φ(wT_{x) = 0 then w := w + x}

Apply the first training sample x(1)=[1 2]T to produce:

φ(wTx(1)) = φ(-1) = 0, d(1)=1

This is wrong so we need to move the weight vector towards the sample w = w + x

so

w = [1 -1]T + [1 2]T = [2 1]T

Apply the second training sample x(2)=[-1 1]T to produce:

φ(wTx(2)) = φ(-1) = 0, d(2)=1

This is wrong so we need to move the weight vector towards the sample w = w + x

so

w = [2 1]T + [-1 1]T = [1 2]T

Apply the third training sample x(3)=[2 0]T to produce:

φ(wTx(3)) = φ(2) = 1, d(3)=0

This is wrong so we need to move the weight vector away from the sample w = w - x

so

w = [1 2]T - [2 0]T = [-1 2]T

(7)

φ(wTx(4)) = φ(-2) = 0, d(4)=0

This is correct so we don’t adjust the weights.

Apply again the first training sample x(1)=[1 2]T to produce:

φ(wTx(1)) = φ(3) = 1, d(1)=1

Apply again the second training sample x(2)=[-1 1]T to produce:

φ(wTx(2)) = φ(3) = 1, d(2)=1

Apply again the third training sample x(3)=[2 0]T_{to produce:} φ(wTx(3)) = φ(-2) = -1, d(3)=0

Apply again the fourth training sample x(4)=[0 -1]T to produce:

φ(wTx(4)) = φ(-2) = 0, d(4)=0

The solution is w = [w1 w2]T = [-1 2]T because all training pairs are correctly classified.

Problem 6 (1 point)

Consider an autoassociative net with the bipolar step function as the activation function and weights set by the Hebb rule (outer products), with the main diagonal of the weight matrix set to zero.

a) Find the weight matrix to store the vector X(1)=[1 1 1 -1 1 1]T; b) Test the net using X(1) as input;

c) Test the net, using Y(1)=[1 1 -1 -1 1 1]T as input;

d) Find the weight matrix to store the vector X(2)=[1 1 -1 1 1 -1]T; e) Test the net using X(2) as input;

f) Test the net, using Y(2)=[1 1 -1 0 0 -1]T as input; g) Find the weight matrix to store the both X(1) and X(2); h) Test the new net on X(1), X(2), Y(1) and Y(2) as inputs.

Solution

a) The weight matrix to store X(1) = [1 1 1 -1 1 1]T is computed by Hebbian rule – outer product

(8)

1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 (1) (1) 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 T − − ⎡ ⎤ ⎡ ⎤ ⎢ ₋ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ − − ⎢ ⎥ ⎢ ⎥ =_⎢ _⎥ ⇒ =_⎢ _⎥ − − − − − − − − − − ⎢ ⎥ ⎢ ⎥ ⎢ − ⎥ ⎢ − ⎥ ⎢ ₋ ⎥ ⎢ ₋ ⎥ ⎣ ⎦ ⎣ ⎦ X X W b)

The bipolar step function is

⎪⎩ ⎪ ⎨ ⎧ < − ≥ = 0 1 0 1 _ _ i in i in i y if y if y 1 2 3 4 5 6 0 1 1 1 1 1 1 5 1 1 0 1 1 1 1 1 5 1 1 1 0 1 1 1 1 5 1 . (1) . (1) 1 1 1 0 1 1 1 5 1 1 1 1 1 0 1 1 5 1 1 1 1 1 1 0 1 5 1 y y y y y y − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − − − − − − _{⎢ ⎥} − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W X y X

correctly associated with X(1) c) 1 2 3 4 5 6 0 1 1 1 1 1 1 3 1 1 0 1 1 1 1 1 3 1 1 1 0 1 1 1 1 5 1 . (1) . (1) 1 1 1 0 1 1 1 3 1 1 1 1 1 0 1 1 3 1 1 1 1 1 1 0 1 3 1 y y y y y y − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − − − − − − _{⎢ ⎥} − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W Y y X

correctly associated with X(1)

d) The weight matrix to store X(2) = [1 1 -1 1 1 -1]T is computed by Hebbian rule – outer product 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 (2) (2) 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 T − − − − ⎡ ⎤ ⎡ ⎤ ⎢ ₋ ₋ ⎥ ⎢ ₋ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ − − − − − − − − ⎢ ⎥ ⎢ ⎥ =_⎢ _⎥ ⇒ =_⎢ _⎥ − − − − ⎢ ⎥ ⎢ ⎥ ⎢ − − ⎥ ⎢ − − ⎥ ⎢₋ ₋ ₋ ₋ ⎥ ⎢₋ ₋ ₋ ₋ ⎥ ⎣ ⎦ ⎣ ⎦ X X W e)

(9)

1 2 3 4 5 6 0 1 1 1 1 1 1 5 1 1 0 1 1 1 1 1 5 1 1 1 0 1 1 1 1 5 1 . (2) . (2) 1 1 1 0 1 1 1 5 1 1 1 1 1 0 1 1 5 1 1 1 1 1 1 0 1 5 1 T y y y y y y − − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ₋ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − − − − − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − _{⎢ ⎥} ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − − ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢₋ ₋ ₋ ₋ ⎥ ⎢ ⎥ ⎢ ⎥₋ ₋ ⎢ ⎥₋ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W X y X

correctly associated with X(2) f) 1 2 3 4 5 6 0 1 1 1 1 1 1 3 1 1 0 1 1 1 1 1 3 1 1 1 0 1 1 1 1 3 1 . (2) . (2) 1 1 1 0 1 1 0 4 1 1 1 1 1 0 1 0 4 1 1 1 1 1 1 0 1 3 1 y y y y y y − − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ₋ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − − − − − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − _{⎢ ⎥} ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ − − ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢₋ ₋ ₋ ₋ ⎥ ⎢ ⎥ ⎢ ⎥₋ ₋ ⎢ ⎥₋ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W Y y X

g) The weight matrix to store X(1) = [1 1 1 -1 1 1]T and X(2) = [1 1 -1 1 1 -1]T is computed by Hebbian rule – outer product

2 2 0 0 2 0 0 2 0 0 2 0 2 2 0 0 2 0 2 0 0 0 2 0 0 0 2 2 0 2 0 0 0 2 0 2 (1) (1) (2) (2) 0 0 2 2 0 2 0 0 2 0 0 2 2 2 0 0 2 0 2 2 0 0 0 0 0 0 2 2 0 2 0 0 2 2 0 0 T T ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − ⎢ ⎥ ⎢ ⎥ + =_⎢ _⎥ ⇒ =_⎢ _⎥ − − − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ₋ ⎥ ⎣ ⎦ ⎣ ⎦ X X X X W h) 1 2 3 4 5 6 0 2 0 0 2 0 1 4 1 2 0 0 0 2 0 1 4 1 0 0 0 2 0 2 1 4 1 . (1) . (1) 0 0 2 0 0 2 1 4 1 2 2 0 0 0 0 1 4 1 0 0 2 2 0 0 1 4 1 y y y y y y ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − − − _{⎢ ⎥} − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W X y X

(10)

1 2 3 4 5 6 0 2 0 0 2 0 1 4 1 2 0 0 0 2 0 1 4 1 0 0 0 2 0 2 1 4 1 . (2) . (2) 0 0 2 0 0 2 1 4 1 2 2 0 0 0 0 1 4 1 0 0 2 2 0 0 1 4 1 y y y y y y ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − _{⎢ ⎥} ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥₋ ₋ ⎢ ⎥₋ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W X y X

1 2 3 4 5 6 0 2 0 0 2 0 1 4 1 2 0 0 0 2 0 1 4 1 0 0 0 2 0 2 1 4 1 . (1) . (1) 0 0 2 0 0 2 1 0 1 2 2 0 0 0 0 1 4 1 0 0 2 2 0 0 1 0 1 y y y y y y ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}≠ − − − _{⎢ ⎥} ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W Y y X

no correctly associated with X(1)

1 2 3 4 5 6 0 2 0 0 2 0 1 2 1 2 0 0 0 2 0 1 2 1 0 0 0 2 0 2 1 2 1 . (2) . (2) 0 0 2 0 0 2 0 4 1 2 2 0 0 0 0 0 4 1 0 0 2 2 0 0 1 2 1 y y y y y y ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ − − − − ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ =_⎢ _{⎥ ⎢ ⎥ ⎢ ⎥}= ⇒ =_{⎢ ⎥}=_{⎢ ⎥}= − − _{⎢ ⎥} ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ₋ ⎥ ⎢ ⎥ ⎢ ⎥₋ ₋ ⎢ ⎥₋ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ W Y y X

Problem 7 (1.5 points)

Give a brief description of the following items related to Radial Basis Functions Networks (RBFN):

− Linear separability problem,

− Cover’s theorem on the separability of patterns,

− RBFN architecture,

− Radial basis functions,

− Free parameters in RBFN,

− Learning algorithms,

(11)

Solution

• Linear classifiers are easy to use. In reality, however, there are many cases that linear classifiers can’t handle. Therefore we are looking for a network that nonlinearly converts the input to a higher dimension after which it can be classified using only one layer of neurons with linear activation functions.

• Cover’s theorem on the separability of patterns:

A complex pattern classification problem that is nonlinearly separable in a low dimensional space, is more likely to be linearly separable in a high dimensional space.

• A simple three-layer structure – an input layer;

– a hidden layer with nonlinear activation function; – an output layer.

• Gaussian radial-basis function is used most: g(x) = exp[-(||x – m||/σ)2]

• Three different sets of variables affect the performance of a RBF network with Gaussian functions:

− The centre of each radial-basis activation function,

− The width of each radial-basis activation function,

− The weights of the output layer.

• Learning algorithms

– Modelling of radial basis functions: centres (random initialisation of centres or self-organised learning of centres or supervised) and widths (often unsupervised); – Supervised learning of weights.

• Features:

− Easier interpretation of the system's results;

− Universal approximation ability;

− Much faster training than back-propagation neural networks

Problem 8 (1 point)

Given a Maxnet with 6 neurons with self feedback weights wii=1.The output activation function

for each neuron is

_ if _ 0 ( _ ) 0 otherwise x in x in f x in _{= ⎨}⎧ > ⎩ .

Choose the inhibitory weights wij, i≠j and iterate the network while it stabilizes if the initial states

of the neurons are x1(0)=0.3, x2(0)=0.5, x3(0)=0.7, x4(0)=1.0, x5(0)=0.8, x6(0)=0.6.

Solution

wii = θ= 1, wij = -ε, i≠j and 0< ε <1/6; we choose ε=0.15

(12)

1 1 ( 1) . ( ) m . ( ) . ( ) m . ( ) i ii i ij j i j j j j i j i x t f w x t w x t f θ x t ε x t = = ≠ ≠ ⎛ ⎞ ⎛ ⎞ ⎜ ⎟ ⎜ ⎟ + = _⎜ + _⎟= _⎜ − _⎟ ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠

∑

x(0) = [0.3 0.5 0.7 1.0 0.8 0.6]T initial states x1(1)=f(1*0.3-0.15*(0.5+0.7+1+0.8+0.6))=f(-0.25)=0 x2(1)=f(1*0.5-0.15*(0.3+0.7+1+0.8+0.6))=f(-0.01)=0 x3(1)=f(1*0.7-0.15*(0.3+0.5+1+0.8+0.6))=f(0.22)=0.22 x4(1)=f(1*1.0-0.15*(0.3+0.5+0.7+0.8+0.6))=f(0.565)=0.565 x5(1)=f(1*0.8-0.15*(0.3+0.5+0.7+1+0.6))=f(0.335)=0.335 x5(1)=f(1*0.6-0.15*(0.3+0.5+0.7+1+0.8))=f(0.105)=0.105 Therefore: x(1) = [0 0 0.22 0.565 0.335 0.105]T

Similarly we calculate the neuron states for the next iterations until the Maxnet stabilized x(2) = [0 0 0.22 0.565 0.335 0.105]T x(3) = [0 0 0.0692 0.466 0.2015 0 ]T x(4) = [0 0 0 0.4254 0.1212 0 ]T x(5) = [0 0 0 0.4072 0.0574 0 ]T x(6) = [0 0 0 0.3986 0 0 ]T x(7) = [0 0 0 0.3986 0 0 ]T= x(6) stabilized