ECE 5314: Power System Operation & Control. Lecture 0: Mathematical Background

(1)

ECE 5314: Power System Operation & Control

Lecture 0: Mathematical Background

Vassilis Kekatos

R3 Boyd and Vandenberghe,Convex Optimization, Appendix A.

(2)

Vectors

• Notation for vectors: b=

      b1 . . . bN       ∈RN

• Alinear functionofxcan be expressed as theinner product:

f1(x) =

N

X

i=1

bixi=b>x

where>denotes transposition, i.e.,b>= [b1 · · · bN].

• Thegradientof a multivariate function is the vector of partial derivatives:

∇f(x) = _∂f ∂x1 · · · ∂f ∂xN > Q.0.1 Show that∇f1(x) =b.

Q.0.2 Writef(x) = 2x1+ 3x2−x4 as an inner product and find its gradient

Q.0.3 ExpressPN

(3)

Vector norms

A vector norm is a functionk.k:RN→Rsatisfying the following three properties for allx,y∈RN, anda∈R:

1. Positive definiteness: kxk ≥0, andkxk= 0if and only ifx=0

2. Scaling: kaxk=|a| · kxk

3. Triangle inequality: kx+yk ≤ kxk+kyk

(4)

`

p

norms

Forx∈RN andp≥1, kxkp= n X i=1 |xi|p !1/p . 1. p= 2: Euclidean norm 2. p= 1: sum-abs-valueskxk1=Pi|xi|

3. p=∞: max-abs-valuekxk∞= limp→∞kxkp= maxi|xi| Q.0.4 Findkxk1,kxk2, andkxk∞forx= [3 0 −4]>.

Q.0.5 Show thatx>x=kxk2 2.

(5)

Norm inequalities

Cauchy-Schwartz inequality

x>y≤ kxk2· kyk2

hold with equality iffxandyare linearly dependent.

H¨older’s inequality x>y≤ kxkpkykq for 1 p+ 1 q = 1 and p≥1. Comparing normskxk∞≤ kxk2≤ kxk1

Q.0.6 How does H¨older’s inequality apply forp= 1?

Q.0.7 Show all three norms are equal forx= [c0 · · · 0]>for anyc∈R.

(6)

Matrices

Notation for matrices: A=

        A11 A12 · · · A1M A21 A22 · · · A2M . . . . . . . .. . . . AN1 AN2 · · · AN M         ∈RN×M.

Q.0.8 What is the matrix transposeA>?

Matrix-vector product: Ifb=AxwithA∈RN×M, then

bi=

M

X

j=1 Aijxj

IfA:,j denotes thej-th column ofA, verify that

b=Ax=A:,1x1+. . .+A:,MxM = M X j=1 A:,jxj.

(7)

Quadratic functions

Every homogeneousquadratic functionofxcan be expressed as follows

f2(x) = N X i=1 N X j=1 Aijxixj = N X i=1 xi N X j=1 Aijxj ! = N X i=1

xi[Ax]i=x>Ax for some A∈RN×N

Q.0.9 Expressx21−2x22 asx>Ax. Q.0.10 Expressx21−2x1x2 asx>Ax.

Q.0.11 Expressx21−2x1x2+ 2x2 asx>Ax+b>x. Q.0.12 IfAis symmetric, show that∇f2(x) = 2Ax.

(8)

Square matrices

• Symmetric matrix: A=A>(Aij=Ajifor all(i, j)) Q13: Show thatx>Ax=x>AsxwhereAs= A+A

> 2

(Asis symmetric even ifAis not)

• Trace: Tr(A) =PN

i=1Aii (sum of diagonal elements)

• Inner product: Tr(AB>) =P

i,jAijBij

(9)

Hessian and Jacobian matrices

Hessian matrix: the matrix of second-order partial derivatives off:RN →R

∇2 f(x) =       ∂2f ∂x2₁ · · · ∂2f ∂x1∂xN . . . . .. ... ∂2f ∂xN∂x1 · · · ∂2f ∂x2 N      

Q.0.14 For symmetricA, show that∇2_f2_{(x) = 2A}_.

Jacobian matrix: its rows are the gradients off:RN→RM

J= df dx =      ∇f1(x)> . . . ∇fM(x)>      =      ∂f1 ∂x1 · · · ∂f1 ∂xN . . . . .. ... ∂fM ∂x1 · · · ∂fM ∂xN     

Q.0.15 What is the Jacobian matrix off(x) = 2Ax? Note f:RN→RN

(10)

Eigenvalue decomposition

• Eigenvalue/eigenvector pair(λ,v)ofA:

Av=λv for v6=0

• For every diagonalizableA(linearly independent eigenvectors)

A=VΛV−1

eigenvectors as columns ofV; eigenvalues as entries of diagonalΛ

• For a symmetric matrix:

A=UΛU>

whereUis orthonormal andΛis diagonal and real

Q.0.15 Use MATLAB’seigto compute the eigenvalue decomposition for the symmetric matricesAsobtained from Q.0.9-Q.0.11.

(11)

Singular value decomposition

• Defined even for non-square matrices:

A=UΣV>

whereUandVare orthonormal andΣdiagonal matrix • singular values: σi=

q

λi(AA>)

• left (right) singular vectorsare the eigenvectors ofAA>(A>A) • Rank of a matrix: number of non-zero singular values

Q.0.16 Show thatσi(A) =|λi(A)|for symmetricA.

(12)

Positive definite matrices

• If all eigenvalues are positive, symmetricAispositive definiteA0

• If all eigenvalues are non-negative, symmetricAispositive semi-definite

A0

• Square root: A1/2₌_U√_ΛU>

• MatrixAis positive (semi-)definite iffx>Ax>0 (≥0)for allx.

(13)

Schur complement

For invertibleAandsymmetricmatrixX=

  A B B> C  

DefineSchur complementasS=C−B>A−1B(Appendix A.5.5 of R3)

Q.0.18 Show thatX   y1 y2  =   b1 0   ⇐⇒ Sy1=b1 Properties

1. det(X) = det(A) det(S)

2. X0iffA0andS0

3. AssumeA0. ThenX0iffS0

(14)

Vector spaces of a matrix

Range space: range(A) ={x: x=Avforv∈RN} ⊆RM • vectors that are linear combinations of the columns ofA

• firstrank(A)columns ofUform a basis forrange(A)

Null space: null(A) ={x: Ax=0} ⊆RN • vectors perpendicular to all rows ofA

• a basis fornull(A)are the lastN−rank(A)columns ofV

Fundamental theorem of linear algebra:

range(A) = (null(A>))⊥

(15)

Taylor’s Series Expansion and Mean Value Theorem

• Univariate function (yields linear and quadratic approximations)

f(x) =f(x0)+f0(x0)(x−x0)+f 00 (x0) 2 (x−x0) 2 +. . .= ∞ X i=1 f(n)₍_x0₎ n! (x−x0) n • Multivariate function: f(x)≈f(x0) + (∇f(x0))>(x−x0) + 1 2(x−x0) > ∇2f(x0)(x−x0)

• Mean value theorem: There existyandz betweenxandx0 such that

f(x) =f(x0) +f0(y)(x−x0) f(x) =f(x0) +f0(x0)(x−x0) +f 00 (z) 2 (x−x0) 2

MVT generalizes to multivariate functions.

(16)

Open and closed sets

• Ballof radius >0aroundxis{y:kx−yk ≤}.

• A pointxis aninterior pointof a setS ifx∈ S and there exists a ball aroundxthat is contained entirely inS

• Open set: if every point inS is an interior point Examples: (0,1), interior of a circle

• Closed set: if its compliment set is open Examples: [0,1], circle