compsci 514: algorithms for data science Andrew McGregor Lecture 23

compsci

514:

algorithms for data science

Andrew McGregor

Lecture 23

summary

Last Class:

• Analysis of gradient descent for optimizing convex functions. • Introduction to convex sets and projection functions.

• (The same) analysis of projected gradient descent for optimizing under convex functions under (convex) constraints.

This Class:

• Online learning, regret, and online gradient descent. • Application to stochastic gradient descent.

constrained convex optimization

Often want to performconvex optimization with convex constraints. ~

θ∗= arg min

~ θ∈S

f (~θ), where S is aconvex set.

Definition – Convex Set: A set S ⊆ Rd is convex if and only if,

for any ~θ1, ~θ2∈ S and λ ∈ [0, 1]:

(1 − λ)~θ1+ λ · ~θ2∈ S

For any convex set let PS(·) denote the projection function onto S:

PS(~y ) = arg min ~ θ∈S

constrained convex optimization

Often want to performconvex optimization with convex constraints. ~

θ∗= arg min

~ θ∈S

f (~θ), where S is aconvex set.

Definition – Convex Set: A set S ⊆ Rd is convex if and only if,

for any ~θ1, ~θ2∈ S and λ ∈ [0, 1]:

(1 − λ)~θ1+ λ · ~θ2∈ S

For any convex set let PS(·) denote the projection function onto S:

PS(~y ) = arg min ~ θ∈S

constrained convex optimization

Often want to performconvex optimization with convex constraints. ~

θ∗= arg min

~ θ∈S

f (~θ), where S is aconvex set.

Definition – Convex Set: A set S ⊆ Rd is convex if and only if,

for any ~θ1, ~θ2∈ S and λ ∈ [0, 1]:

(1 − λ)~θ1+ λ · ~θ2∈ S

For any convex set let PS(·) denote the projection function onto S:

PS(~y ) = arg min ~ θ∈S

projected gradient descent

Projected Gradient Descent

• Choose some initialization ~θ

and set η =

.

• For i = 1, . . . , t − 1

• ~_θ

i +1= PS(~θ(out)i +1 ).

convex projections

Analysis of projected gradient descent is almost identifcal to gradient descent analysis!

Theorem – Projection to a convex set: For any convex set

S ⊆ Rd_{, ~}

y ∈ Rd_{, and ~θ ∈ S,}

convex projections

Analysis of projected gradient descent is almost identifcal to gradient descent analysis!

Theorem – Projection to a convex set: For any convex set

S ⊆ Rd_{, ~}

y ∈ Rd_{, and ~θ ∈ S,}

projected gradient descent analysis

Theorem – Projected GD: For convex G -Lipschitz function f ,

and convex set S, Projected GD run with t ≥ R2_G22 iterations,

η = _GR√

t, and starting point within radius R of ~θ∗= min~θ∈Sf (~θ),

outputs ˆθ satisfying:

f (ˆθ) ≤ f (~θ∗) + 

Recall: ~θ(out)_{i +1} = ~θi− η · ~∇f (~θi) and ~θi +1= PS(~θ(out)i +1 ).

Step 1: For all i , f (~θi) − f (~θ∗) ≤

k~θi−θ∗k2 2−k~θ (out) i +1 −~θ∗k 2 2 2η + ηG2 2 .

Step 1.a: For all i , f (~θi) − f (~θ∗) ≤ k~θi−~θ∗k 2 2−k~θi +1−~θ∗k 2 2 2η + ηG2 2 . Step 2: 1_tPt i =1f (~θi) − f (~θ∗) ≤ R 2 2η·t + ηG2 2 =⇒ Theorem.

projected gradient descent analysis

Theorem – Projected GD: For convex G -Lipschitz function f ,

and convex set S, Projected GD run with t ≥ R2_G22 iterations,

η = _GR√

t, and starting point within radius R of ~θ∗= min~θ∈Sf (~θ),

outputs ˆθ satisfying:

f (ˆθ) ≤ f (~θ∗) + 

Recall: ~θ(out)_{i +1} = ~θi− η · ~∇f (~θi) and ~θi +1= PS(~θ(out)i +1 ).

Step 1: For all i , f (~θi) − f (~θ∗) ≤

k~θi−θ∗k2 2−k~θ (out) i +1 −~θ∗k 2 2 2η + ηG2 2 .

Step 1.a: For all i , f (~θi) − f (~θ∗) ≤ k~θi−~θ∗k 2 2−k~θi +1−~θ∗k 2 2 2η + ηG2 2 . Step 2: 1_tPt i =1f (~θi) − f (~θ∗) ≤ R 2 2η·t + ηG2 2 =⇒ Theorem.

projected gradient descent analysis

Theorem – Projected GD: For convex G -Lipschitz function f ,

and convex set S, Projected GD run with t ≥ R2_G22 iterations,

η = _GR√

t, and starting point within radius R of ~θ∗= min~θ∈Sf (~θ),

outputs ˆθ satisfying:

f (ˆθ) ≤ f (~θ∗) + 

Recall: ~θ(out)_{i +1} = ~θi− η · ~∇f (~θi) and ~θi +1= PS(~θ(out)i +1 ).

Step 1: For all i , f (~θi) − f (~θ∗) ≤

k~θi−θ∗k2 2−k~θ (out) i +1 −~θ∗k 2 2 2η + ηG2 2 .

Step 1.a: For all i , f (~θi) − f (~θ∗) ≤ k~θi−~θ∗k 2 2−k~θi +1−~θ∗k 2 2 2η + ηG2 2 . Step 2: 1_tPt i =1f (~θi) − f (~θ∗) ≤ R 2 2η·t + ηG2 2 =⇒ Theorem.

projected gradient descent analysis

Theorem – Projected GD: For convex G -Lipschitz function f ,

and convex set S, Projected GD run with t ≥ R2_G22 iterations,

η = _GR√

t, and starting point within radius R of ~θ∗= min~θ∈Sf (~θ),

outputs ˆθ satisfying:

f (ˆθ) ≤ f (~θ∗) + 

Recall: ~θ(out)_{i +1} = ~θi− η · ~∇f (~θi) and ~θi +1= PS(~θ(out)i +1 ).

Step 1: For all i , f (~θi) − f (~θ∗) ≤

k~θi−θ∗k2 2−k~θ (out) i +1 −~θ∗k 2 2 2η + ηG2 2 .

Step 1.a: For all i , f (~θi) − f (~θ∗) ≤ k~θi−~θ∗k 2 2−k~θi +1−~θ∗k 2 2 2η + ηG2 2 . Step 2: 1_tPt i =1f (~θi) − f (~θ∗) ≤ R 2 2η·t + ηG2 2 =⇒ Theorem.

projected gradient descent analysis

Theorem – Projected GD: For convex G -Lipschitz function f ,

and convex set S, Projected GD run with t ≥ R2_G22 iterations,

η = _GR√

t, and starting point within radius R of ~θ∗= min~θ∈Sf (~θ),

outputs ˆθ satisfying:

f (ˆθ) ≤ f (~θ∗) + 

Recall: ~θ(out)_{i +1} = ~θi− η · ~∇f (~θi) and ~θi +1= PS(~θ(out)i +1 ).

Step 1: For all i , f (~θi) − f (~θ∗) ≤

k~θi−θ∗k2 2−k~θ (out) i +1 −~θ∗k 2 2 2η + ηG2 2 .

Step 1.a: For all i , f (~θi) − f (~θ∗) ≤ k~θi−~θ∗k 2 2−k~θi +1−~θ∗k 2 2 2η + ηG2 2 . Step 2: 1_tPt i =1f (~θi) − f (~θ∗) ≤ R 2 2η·t + ηG2 2 =⇒ Theorem.

online gradient descent

In reality many learning problems are online.

• Websites optimize ads or recommendations to show users, given continuous feedback from these users.

• Spam filters are incrementally updated and adapt as they see more examples of spam over time.

• Face recognition systems, other classification systems, learn from mistakes over time.

Want to minimize some global loss L(~θ, X) =Pn

i =1`(~θ, ~xi), when data

points are presented in an online fashion ~x1, ~x2, . . . , ~xn(similar to

streaming algorithms)

Stochastic gradient descent is a special case: when data points are considered arandom orderfor computational reasons.

online gradient descent

In reality many learning problems are online.

• Websites optimize ads or recommendations to show users, given continuous feedback from these users.

• Spam filters are incrementally updated and adapt as they see more examples of spam over time.

• Face recognition systems, other classification systems, learn from mistakes over time.

Want to minimize some global loss L(~θ, X) =Pn

i =1`(~θ, ~xi), when data

points are presented in an online fashion ~x1, ~x2, . . . , ~xn(similar to

streaming algorithms)

Stochastic gradient descent is a special case: when data points are considered arandom orderfor computational reasons.

online gradient descent

In reality many learning problems are online.

• Websites optimize ads or recommendations to show users, given continuous feedback from these users.

• Spam filters are incrementally updated and adapt as they see more examples of spam over time.

• Face recognition systems, other classification systems, learn from mistakes over time.

Want to minimize some global loss L(~θ, X) =Pn

i =1`(~θ, ~xi), when data

points are presented in an online fashion ~x1, ~x2, . . . , ~xn(similar to

streaming algorithms)

Stochastic gradient descent is a special case: when data points are considered arandom orderfor computational reasons.

online optimization formal setup

Online Optimization: In place of a single function f , we see a

different objective function at each step:

f

, f

, . . . , f

: R

→ R

• At each step, first pick (play) a parameter vector ~θ

_.

• Then are told f

and incur cost f

(~

θ

).

• Goal: Minimize total cost P

i =1

f

(~

θ

).

Our analysis will make no assumptions on how f

, . . . , f

are related

online optimization formal setup

Online Optimization: In place of a single function f , we see a

different objective function at each step:

f

, f

, . . . , f

: R

→ R

• At each step, first pick (play) a parameter vector ~θ

_.

• Then are told f

and incur cost f

(~

θ

).

• Goal: Minimize total cost P

i =1

f

(~

θ

).

Our analysis will make no assumptions on how f

, . . . , f

are related

online optimization formal setup

Online Optimization: In place of a single function f , we see a

different objective function at each step:

f

, f

, . . . , f

: R

→ R

• At each step, first pick (play) a parameter vector ~θ

_.

• Then are told f

and incur cost f

(~

θ

).

• Goal: Minimize total cost P

i =1

f

(~

θ

).

Our analysis will make no assumptions on how f

, . . . , f

are related

online optimization example

UI design via online optimization.

• Parameter vector ~θ

_{: some encoding of the layout at step i .}

• Functions f

, . . . , f

: f

(~

θ

) = 1 if user does not click ‘add to

cart’ and f

(~

θ

) = 0 if they do click.

• Want to maximize number of purchases, i.e., minimize

P

online optimization example

Home pricing tools.

• Parameter vector ~θ

_{: coefficients of linear model at step i .}

• Functions f

, . . . , f

: f

(~

θ

) = (h~

x

, ~

θ

i − price

)

revealed when

home

is listed or sold.

• Want to minimize total squared error P

i =1

f

(~

θ

) (same as

regret

In normal optimization, we seek ˆ

θ satisfying:

f (ˆ

θ) ≤ min

~ θ

f (~

θ) + .

In online optimization we will ask for the same.

t

X

f

(~

θ

) ≤ min

X

f

(~

θ) +  =

X

f

(~

θ

) + 

 is called the

regret

and /t is the average regret.

• This error metric is a bit

unusual

: Comparing online solution to

best fixed solution in hindsight.  can be negative!

regret

In normal optimization, we seek ˆ

θ satisfying:

f (ˆ

θ) ≤ min

~ θ

f (~

θ) + .

In online optimization we will ask for the same.

t

X

f

(~

θ

) ≤ min

X

f

(~

θ) +  =

X

f

(~

θ

) + 

 is called the

regret

and /t is the average regret.

• This error metric is a bit

unusual

: Comparing online solution to

best fixed solution in hindsight.  can be negative!

regret

In normal optimization, we seek ˆ

θ satisfying:

f (ˆ

θ) ≤ min

~ θ

f (~

θ) + .

In online optimization we will ask for the same.

t

X

f

(~

θ

) ≤ min

X

f

(~

θ) +  =

X

f

(~

θ

) + 

 is called the

regret

and /t is the average regret.

• This error metric is a bit

unusual

: Comparing online solution to

best fixed solution in hindsight.  can be negative!

intuition check

What if for i = 1, . . . , t, f

(θ) = |θ − 1000| or f

(θ) = |θ + 1000| in

an alternating pattern?

How small can the regret  be?

P

i =1

f

(~

θ

) ≤

P

f

(~

θ

) + .

What if for i = 1, . . . , t, f

(θ) = |θ − 1000| or f

(θ) = |θ + 1000|

in

no particular pattern

? How can any online learning algorithm hope

intuition check

What if for i = 1, . . . , t, f

(θ) = |θ − 1000| or f

(θ) = |θ + 1000| in

an alternating pattern?

How small can the regret  be?

P

i =1

f

(~

θ

) ≤

P

f

(~

θ

) + .

What if for i = 1, . . . , t, f

(θ) = |θ − 1000| or f

(θ) = |θ + 1000|

in

no particular pattern

? How can any online learning algorithm hope

online gradient descent

Assume that:

• f

, . . . , f

are all convex.

• Each f

is G -Lipschitz (i.e., k ~

∇f

(~

θ)k

≤ G for all ~

θ.)

• k~θ

_{− ~}

_θ

_k

2

≤ R where θ

is the first vector chosen.

Online Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Play ~θ(i ) _{and incur cost f} i(~θ(i )).

• ~_θ(i +1)_{= ~θ}(i )_{− η · ~∇f} i(~θ(i ))

online gradient descent

Assume that:

• f

, . . . , f

are all convex.

• Each f

is G -Lipschitz (i.e., k ~

∇f

(~

θ)k

≤ G for all ~

θ.)

• k~θ

_{− ~}

_θ

_k

2

≤ R where θ

is the first vector chosen.

Online Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Play ~θ(i ) _{and incur cost f} i(~θ(i )).

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Upper bound onaverage regret goes to 0 and t → ∞. No assumptions on f1, . . . , ft!

Step 1.1: For all i , ∇fi(θ(i ))T(θ(i )− θoff) ≤

kθ(i )_−θoff_k2 2−kθ (i +1)_−θoff_k2 2 2η + ηG2 2

Convexity =⇒ Step 1: For all i , fi(θ(i )) − fi(θoff) ≤ kθ(i )_{− θ}off_k2 2− kθ(i +1)− θoffk22 2η + ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Upper bound onaverage regret goes to 0 and t → ∞.

No assumptions on f1, . . . , ft!

Step 1.1: For all i , ∇fi(θ(i ))T(θ(i )− θoff) ≤

kθ(i )_−θoff_k2 2−kθ (i +1)_−θoff_k2 2 2η + ηG2 2

Convexity =⇒ Step 1: For all i , fi(θ(i )) − fi(θoff) ≤ kθ(i )_{− θ}off_k2 2− kθ(i +1)− θoffk22 2η + ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Upper bound onaverage regret goes to 0 and t → ∞. No assumptions on f1, . . . , ft!

Step 1.1: For all i , ∇fi(θ(i ))T(θ(i )− θoff) ≤

kθ(i )_−θoff_k2 2−kθ (i +1)_−θoff_k2 2 2η + ηG2 2

Convexity =⇒ Step 1: For all i , fi(θ(i )) − fi(θoff) ≤ kθ(i )_{− θ}off_k2 2− kθ(i +1)− θoffk22 2η + ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Upper bound onaverage regret goes to 0 and t → ∞. No assumptions on f1, . . . , ft!

Step 1.1: For all i , ∇fi(θ(i ))T(θ(i )− θoff) ≤

kθ(i )_−θoff_k2

2−kθ(i +1)−θoffk22

2η +

ηG2 2

Convexity =⇒ Step 1: For all i , fi(θ(i )) − fi(θoff) ≤ kθ(i )_{− θ}off_k2 2− kθ(i +1)− θoffk22 2η + ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Upper bound onaverage regret goes to 0 and t → ∞. No assumptions on f1, . . . , ft!

Step 1.1: For all i , ∇fi(θ(i ))T(θ(i )− θoff) ≤

kθ(i )_−θoff_k2

2−kθ(i +1)−θoffk22

2η +

ηG2 2

Convexity =⇒ Step 1: For all i , fi(θ(i )) − fi(θoff) ≤ kθ(i )_{− θ}off_k2 2− kθ(i +1)− θoffk22 2η + ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Step 1: For all i , fi(θ(i )) − fi(θoff) ≤

kθ(i )_−θoff_k2 2−kθ(i +1)−θoffk22 2η + ηG2 2 =⇒ " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ t X i =1 kθ(i )_{− θ}off_k2 2− kθ (i +1)_{− θ}off_k2 2 2η + t · ηG2 2 .

online gradient descent analysis

Theorem – OGD on Convex Lipschitz Functions: For con-vex G -Lipschitz f1, . . . , ft, OGD initialized with starting point θ(1)

within radius R of θoff_{, using step size η =} R

G√t, has regret bounded

by: " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ RG√t

Step 1: For all i , fi(θ(i )) − fi(θoff) ≤

kθ(i )_−θoff_k2 2−kθ(i +1)−θoffk22 2η + ηG2 2 =⇒ " t X i =1 fi(θ(i )) − t X i =1 fi(θoff) # ≤ t X i =1 kθ(i )_{− θ}off_k2 2− kθ (i +1)_{− θ}off_k2 2 2η + t · ηG2 2 .

stochastic gradient descent

Stochastic gradient descent is an efficient

offline optimization

method

, seeking ˆ

θ with

f (ˆ

θ) ≤ min

~ θ

f (~

θ) +  = f (~

θ

) + .

• The most popular optimization method in modern machine

learning.

stochastic gradient descent

Stochastic gradient descent is an efficient

offline optimization

method

, seeking ˆ

θ with

f (ˆ

θ) ≤ min

~ θ

f (~

θ) +  = f (~

θ

) + .

• The most popular optimization method in modern machine

learning.

stochastic gradient descent

Assume that:

• f is convex and decomposable as f (~θ) = P

j =1

f

(~

θ).

• E.g., L(~θ, X) = Pn

j =1`(~θ, ~xj).

• Each f

is

-Lipschitz (i.e., k ~

∇f

(~

θ)k

≤

for all ~

θ.)

• What does this imply about how Lipschitz f is?

• Initialize with θ

_{satisfying k~}

_θ

_{− ~}

_θ

_k

≤ R.

Stochastic Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Return

_{θ =}

ˆ

Pt

θ

~

.

stochastic gradient descent

Assume that:

• f is convex and decomposable as f (~θ) = P

j =1

f

(~

θ).

• E.g., L(~θ, X) = Pn

j =1`(~θ, ~xj).

• Each f

is

-Lipschitz (i.e., k ~

∇f

(~

θ)k

≤

for all ~

θ.)

• What does this imply about how Lipschitz f is?

• Initialize with θ

_{satisfying k~}

_θ

_{− ~}

_θ

_k

≤ R.

Stochastic Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Return

_{θ =}

ˆ

Pt

θ

~

.

stochastic gradient descent

Assume that:

• f is convex and decomposable as f (~θ) = P

j =1

f

(~

θ).

• E.g., L(~θ, X) = Pn

j =1`(~θ, ~xj).

• Each f

is

-Lipschitz (i.e., k ~

∇f

(~

θ)k

≤

for all ~

θ.)

• What does this imply about how Lipschitz f is?

• Initialize with θ

_{satisfying k~}

_θ

_{− ~}

_θ

_k

≤ R.

Stochastic Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Return

_{θ =}

ˆ

Pt

θ

~

.

stochastic gradient descent

Assume that:

• f is convex and decomposable as f (~θ) = P

j =1

f

(~

θ).

• E.g., L(~θ, X) = Pn

j =1`(~θ, ~xj).

• Each f

is

-Lipschitz (i.e., k ~

∇f

(~

θ)k

≤

for all ~

θ.)

• What does this imply about how Lipschitz f is?

• Initialize with θ

_{satisfying k~}

_θ

_{− ~}

_θ

_k

≤ R.

Stochastic Gradient Descent

• Pick some initial ~θ

_.

• Set step size η =

.

• For i = 1, . . . , t

• Return

_{θ =}

ˆ

Pt

~

θ

.

stochastic gradient descent

~

θ

= ~

θ

− η · ~

∇f

(~

θ

) vs. ~

θ

= ~

θ

− η · ~

∇f (~

θ

)

Note that: E[~

∇f

(~

θ

)] =

∇f (~

~

θ

).

Analysis extends to

any

algorithm that takes the gradient step

in

expectation

(minibatch SGD, randomly quantized, measurement

stochastic gradient descent analysis

Theorem – SGD on Convex Lipschitz Functions: SGD run with t ≥ R2_G22 iterations, η =

R

G√t, and starting point within radius R

of θ∗, outputs ˆθ satisfying: _{E[f ( ˆ}θ)] ≤ f (θ∗) + .

Step 1: f (ˆθ) − f (θ∗) ≤1_tPt i =1[f (θ (i )_{) − f (θ}∗_)] Step 2: E[f (ˆθ) − f (θ∗_{)] ≤} n t · E Pt i =1[fji(θ(i )) − fji(θ∗)] . Step 3: E[f (ˆθ) − f (θ∗)] ≤ n_t · EPt i =1[fji(θ(i )) − fji(θoff)] . Step 4: E[f (ˆθ) − f (θ∗)] ≤ n_t · R ·G n · √ t | {z } OGD bound = RG√ t.

stochastic gradient descent analysis

Theorem – SGD on Convex Lipschitz Functions: SGD run with t ≥ R2_G22 iterations, η =

R

G√t, and starting point within radius R

of θ∗, outputs ˆθ satisfying: _{E[f ( ˆ}θ)] ≤ f (θ∗) + .

Step 1: f (ˆθ) − f (θ∗) ≤1_tPt i =1[f (θ (i )_{) − f (θ}∗_)] Step 2: E[f (ˆθ) − f (θ∗_{)] ≤} n t · E Pt i =1[fji(θ(i )) − fji(θ∗)] . Step 3: E[f (ˆθ) − f (θ∗)] ≤ n_t · EPt i =1[fji(θ(i )) − fji(θoff)] . Step 4: E[f (ˆθ) − f (θ∗)] ≤ n_t · R ·G n · √ t | {z } OGD bound = RG√ t.

stochastic gradient descent analysis

Theorem – SGD on Convex Lipschitz Functions: SGD run with t ≥ R2_G22 iterations, η =

R

G√t, and starting point within radius R

of θ∗, outputs ˆθ satisfying: _{E[f ( ˆ}θ)] ≤ f (θ∗) + .

Step 1: f (ˆθ) − f (θ∗) ≤1_tPt i =1[f (θ (i )_{) − f (θ}∗_)] Step 2: E[f (ˆθ) − f (θ∗_{)] ≤} n t · E Pt i =1[fji(θ(i )) − fji(θ∗)] . Step 3: E[f (ˆθ) − f (θ∗)] ≤ n_t · EPt i =1[fji(θ(i )) − fji(θoff)] . Step 4: E[f (ˆθ) − f (θ∗)] ≤ n_t · R ·G n · √ t | {z } OGD bound = RG√ t.

stochastic gradient descent analysis

Theorem – SGD on Convex Lipschitz Functions: SGD run with t ≥ R2_G22 iterations, η =

R

G√t, and starting point within radius R

of θ∗, outputs ˆθ satisfying: _{E[f ( ˆ}θ)] ≤ f (θ∗) + .

Step 1: f (ˆθ) − f (θ∗) ≤1_tPt i =1[f (θ (i )_{) − f (θ}∗_)] Step 2: E[f (ˆθ) − f (θ∗_{)] ≤} n t · E Pt i =1[fji(θ(i )) − fji(θ∗)] . Step 3: E[f (ˆθ) − f (θ∗)] ≤ n t · E Pt i =1[fji(θ(i )) − fji(θoff)] . Step 4: E[f (ˆθ) − f (θ∗)] ≤ n_t · R ·G n · √ t | {z } OGD bound = RG√ t.

stochastic gradient descent analysis

Theorem – SGD on Convex Lipschitz Functions: SGD run with t ≥ R2_G22 iterations, η =

R

G√t, and starting point within radius R

of θ∗, outputs ˆθ satisfying: _{E[f ( ˆ}θ)] ≤ f (θ∗) + .

Step 1: f (ˆθ) − f (θ∗) ≤1_tPt i =1[f (θ (i )_{) − f (θ}∗_)] Step 2: E[f (ˆθ) − f (θ∗_{)] ≤} n t · E Pt i =1[fji(θ(i )) − fji(θ∗)] . Step 3: E[f (ˆθ) − f (θ∗)] ≤ n t · E Pt i =1[fji(θ(i )) − fji(θoff)] . Step 4: E[f (ˆθ) − f (θ∗)] ≤ n_t · R ·G n · √ t | {z } OGD bound = RG√ t.

sgd vs. gd

Stochastic gradient descent generally makes more iterations than

gradient descent.

Each iteration is much cheaper (by a factor of n).

~

∇

X

f

(~

θ) vs. ~

∇f

(~

θ)

sgd vs. gd

When f (~θ) =Pn

j =1fj(~θ) and k ~∇fj(~θ)k2≤G_n:

Theorem – SGD: Aftert ≥ R2_G22 iterations outputs ˆθ satisfying:

E[f ( ˆθ)] ≤ f (θ∗) + .

When k ~∇f (~θ)k2≤ ¯G :

Theorem – GD: Aftert ≥ R2_G2¯2 iterations outputs ˆθ satisfying:

summary

• Introduced the online optimization problem and the notion of

regret to measure error in this setting.

• Introduced online gradient descent, which can solve online

convex optimization with average regret approaching 0.

• Introduced stochastic gradient descent, an offline optimization

method that can be analyzed as a special case of online gradient

descent.