REAL-TIME PRICE FORECAST WITH BIG DATA

(1)

CERTS Review

REAL-TIME PRICE FORECAST WITH

BIG DATA

A STATE SPACE APPROACH

Lang Tong (PI), Robert J. Thomas, Yuting Ji, and Jinsub Kim

School of Electrical and Computer Engineering, Cornell

University

Jie Mei, Georgia Institute of Technology

(2)

CERTS Review

DATA QUALITY AND ITS EFFECTS

ON MARKET OPERATIONS

DENY OF SERVICE ATTACK ON REAL-TIME ELECTRICITY

MARKET

COVER UP PROTECTION AGAINST TOPOLOGY ATTACK

Lang Tong (PI), Robert J. Thomas, and Jinsub Kim

Cornell University

(3)

Project overview



Objectives



Accurate short-term

probabilistic

forecasting

of real-time LMP.



Incorporate

real-time measurements

(e.g. SCADA/PMU).



Scalable computation techniques.



Summary of results



A real-time LMP model with forecasting and measurement

uncertainties.



A Markov chain abstraction of real-time LMP computation.



Monte Carlo sampling techniques.

(4)

Outline



Motivation



Load and real-time LMP as random processes



Benchmark techniques



Comments on the state of the art.



Probabilistic forecasting of real-time LMP



Simulation studies

(5)

(6)

(7)

Benchmark techniques



Time series



ARMA, ARIMA, ARMAX, GARCH



Machine learning



Neural networks, support vector machines (SVM)



Hybrid techniques

(8)

Load forecasting

Forecasting

Method

MAPE

ANN [1]

ANN [2]

1.46%

1.24%

SVM [3]

2.09%

[1] P. Mandal, T. Senjyu, N. Urasaki, and T. Funabashi, “A neural network based several-hour-ahead electric load forecasting using similar days approach,” Int. Journal. of Electric Power and Energy System, vol. 28, no. 6, pp. 367–373, July 2006.

[2] P. Mandal, T. Senjyu, N. Urasaki, and T. Funabashi, “Short-Term Price Forecasting for Competitive Electricity Market,” Power

Symposium, 2006. NAPS 2006. 38th_North.

[3] S. Fan, C. Mao and L. Chen, “Next-day electricity-price

forecasting using a hybrid network,” IET Generation, Transmission & Distribution, Volume 1, Issue 1, January, 2007.

(9)

Day ahead LMP forecasting

Forecasting

Method

MAPE

ANN [1]

ANN [2]

ANN [4]

15.96%

12.92%

8.67%

SVM [3]

11.31%

GARCH [5]

11.55%

ARMAX [6]

9.01%

[4] P. Mandal, T. Senjyu, N. Urasaki, and T. Funabashi, “A neural network based several-hour-ahead electric load forecasting using similar days approach,” Int. Journal. of Electric Power and Energy System, vol. 28, no. 6, pp. 367–373, July 2006.

[5] R. Garcia, J. Contreras, “A GARCH Forecasting Model to Predict Day-Ahead Electricity Prices,” Int. Journal of Electric Power and Energy System, vol. 20, no. 2, MAY 2005.

[6] J. Zhang, C. Yu, G. Hou, “Application of Chaotic Particle Swarm Optimization in the Short-term Electricity Price Forecasting,” Int. Journal of Electric Power and Energy System, vol. 28, no. 6, pp. 367 373 July 2006

(10)

Real-time LMP forecasting

Actual and forecast six-hours-ahead electricity price for

Victorian electricity market (Nov 1st_{to 7}th_{, 2003).}

Actual and forecast six-hours-ahead electricity price

for PJM market (Jan 8th_{to 14}th_{, 2006).}

[4]

F. Sarafraz, H. Ghasemi, H. Monsef, “Locational Marginal Price Forecasting by

Locally

(11)

Summary of the state-of-the-art



Extensive literature:



Mostly black box techniques



Primarily providing point forecasting



Rarely deal with on LMP and network effects



Extremely accurate load forecasting (1-3%)



Relatively poor price forecasting (10-20%)

(12)

Outline



Motivation





A stylized real-time

ex post

LMP model



Information structure



LMP states

and Markov chain representations



Probabilistic forecast



Preliminary simulations



Summary and future work

(13)

A stylized real-time ex post LMP

model

(14)

(15)

(16)

(17)

(18)

(19)

Information structure and Markov

chain

(20)

(21)

Outline



Motivation



Preliminary simulations



IEEE 16 bus with NYISO sample load profiles



Varying load errors and correlations



Monte Carlo techniques to estimate transition

matrices

(22)

Load and price distributions

0 15 42 119 200 288 20 22 24 26 28 30 32 34 36 38

time index (5 mins interval in 24 hours)

MW

Scaled average actual load profile (upper/lower bounds for predicted load profile) from NYISO June 1st

actual load profile upper bound lower bound 1 2 3 4 5 6 7 8 9 1011 0 0.5 1

Markov State (LMP) index

P robabi li ty LMP distribution at time 42 1 2 3 4 5 6 7 8 9 1011 0 0.5 1

P

robabi

li

ty

(23)

LMP trajectories

0 50 100 150 200 250 15 20 25 30 35 40 45 50

$/ M W LMP trajectories load bus1 bus2 bus3 bus4 bus5 bus6 bus7 bus8 bus9 bus10 bus11 bus12 bus13 bus14

(24)

Time inhomogeneous Markov

Chain

(25)

Transition Matrices

Ending Markov State (LMP) index

S tar ti ng M ar k ov S tat e ( LM P ) i

ndex Transition matrix at time 15

2 4 6 8 10 12 2 4 6 8 10 12 _-4 -3 -2 -1

ndex Transition matrix at time 29

2 4 6 8 10 12 2 4 6 8 10 12 -3 -2 -1

nde Transition matrix at time 117

2 4 6 8 10 12 2 4 6 8 10 12 _-8 -6 -4 -2 0

S tar ti ng M ar k ov S tat e ( LM P ) i ndex

Transition matrix at time 220

2 4 6 8 10 12 2 4 6 8 10 12 -3 -2 -1 0

(26)

6 hours ahead prediction error

20 40 60 80 100 120 140 160 180 200 10-6 10-4 10-2 100

P robabi lit y of i nc or rec t pr edi c ti

on Prediction error for best and second best prediction

Best prediction 2nd best prediction 20 40 60 80 100 120 140 160 180 200 0 2 4 6 8 10

n A bs ol ut e P er c ent age E rr or ( M A P

E _{MAPE of best (most likely) prediction}

Prediction transition matrix at time 21

2 4 6 8 10 12 2 4 6 8 10 12 -3 -2.5 -2 -1.5 -1 -0.5 0

Prediction transition matrix at time 21

2 4 6 8 10 12 2 4 6 8 10 12 -3 -2.5 -2 -1.5 -1 -0.5 0

(27)

LMP prediction for different time

horizon

1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 10 20 30 40 50 Bus index $/ M W

LMP prediction for 5 mins ahead

Actual Best prediction 2nd best 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 10 20 30 40 50 Bus index $/ M W

LMP prediction for 1 hour ahead

Actual Best prediction 2nd best 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 10 20 30 40 50

LMP prediction for 3 hours ahead

Bus index $/ M W Actual Best prediction 2nd best 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0 10 20 30 40 50

LMP prediction for 6 hours ahead

Bus index $/ M W Actual Best prediction 2nd best

(28)

LMP distribution for different prediction

horizons

1 2 3 4 5 6 7 8 9 10 11 12 13 10-4 10-2 100

P robabi li ty ( Log)

LMP distribution for 5 mins ahead

1 2 3 4 5 6 7 8 9 10 11 12 13 10-4

10-2 100

LMP distribution for 1 hour ahead

1 2 3 4 5 6 7 8 9 10 11 12 13 10-4

10-2 100

LMP distribution for 3 hours ahead

1 2 3 4 5 6 7 8 9 10 11 12 13 10-4

10-2 100

(29)

Planned research



Forecasting techniques



Incorporating load models



More sophisticated Monte Carlo techniques (important

sampling, MCMC)



Generator behavior models



Forecasting with renewable sources



Demand response

(30)

DATA QUALITY AND ITS EFFECTS

ON MARKET OPERATIONS

DENY OF SERVICE ATTACK ON REAL-TIME ELECTRICITY

MARKET

Lang Tong (PI), Robert J. Thomas, and Jinsub Kim

Cornell University

Presented at CERTS

Review

(31)

(32)

A graph theoretic condition

(KCD’80)

(33)

(34)

(35)

(36)

(37)

(38)

(39)

Conclusion



Attacks on cyber physical systems (power

systems) are real:



State attacks



Topology attacks



Dispatch attacks



A key step toward protection is deploy