BaiQing DIAO

Intelligent Operation Analysis and

Application of Power Big Data

Contents

Application of Technology Framework

Application of Technology Framework

Ⅱ

Ⅱ

Introduction of Power Big Data

Introduction of Power Big Data

Ⅰ

Ⅰ

Data Mining and Visualization

Data Mining and Visualization

Ⅳ

Ⅳ

Power Big Data Quality Management

Power Big Data Quality Management

Ⅲ

Introduction of Power Big Data-Types of Business

Operation data of the company includes all kinds of

business information of power grid.

Operation data of the company includes all kinds of

business information of power grid.

Transmission

Transmission _{Transformation}Transformation _DistributionDistribution _UtilizationUtilization _DispatchDispatch

• Condition mo nitoring for transmission line • HVDC Flexible • Intelligent power station • Distribution Automation •MDS Management • Electricity Consumption Information Gathering • Intelligent b uilding • Electric Vehicles • EMS • OMS Resources Guarantee Resources Guarantee •HR • Science and t echnology •……

Introduction of Power Big Data- Business System

With business system deployment and application, data volume increases sharply, various types of data, the data storage, handling and value mining put forward higher requirements, and the need to frame a unified data platform and data management.

With business system deployment and application, data volume increases sharply, various types of data, the data storage, handling and value mining put forward higher requirements, and the need to frame a unified data platform and data management.

• HR System • Finalcial System • Materials System TB->PB ERP •Data volume increases from TB to PB • Amount of data from customer service senter to PB Marketing system •Storage of equipment information, maintenance and order information •Data volume to TB Manufacturing system •Cover scope broad •huge data volume •high data processing performance Dispatching System

•Data volume of data warehouse to TB

•Growth of data volume to TB every month

Structured Data

Platform Unstructured Data Platform •Amount of data to 10 million in existing business systmes •Total data storage to TB

Mass/Real time Data

Platform GIS Data Platform •Electricity consumption and transformation systems information gathering •Data storage to TB

•GIS data platform includes graphic data, attribute data and topological data •Data storage in GIS to TB Data volume from TB to Pb, high data saving performance and extensibility Data volume from TB to Pb, high data saving performance and extensibility Intelligent and subtilized busines s, requires real time and high data processing complexity

Intelligentand

subtilizedbusines s, requires real time and high data processing complexity Data mining capability in cross-business and platform needs to improve Data mining capability in cross-business and platform needs to improve

Introduction of Power Big Data-Business Data

Power Big Data include：Structured Data、Unstructured Data、Mass/Real time Data and GIS Data.

Mass/Real time Data(Timestamp Data)

generated from the

acquisition system and the condition monitoring system.

Mass/Real time Data

Mass/Real time Data

Unstructured Data(Files Data)。

Includes office documents,

pictures, XML, HTML, videos and audios.

Structured Data(Relational Data)，Data from business applications, for horizontal sharing, vertical cascade and data analysis, etc

Structured Data

Structured Data

Unstructured Data

Introduction of Power Big Data-Data

Attribute

Engergy

• Data Factory

• Efficiency Power Plant • Power to Data

Exchange

• Visualization • Real-time • Two-way interaction

Empathy

• Reflect requirements • Cross boundary • Ecological benefit

“4V”

4 V : Volume Variety Velocity Value 3 E : Energy Exchange Empathy

Contents

Application of Technology Framework

Application of Technology Framework

Ⅱ

Ⅱ

Introduction of Power Big Data

Introduction of Power Big Data

Ⅰ

Ⅰ

Data Mining and Visualization

Data Mining and Visualization

Ⅳ

Ⅳ

Power Big Data Quality Management

Power Big Data Quality Management

Ⅲ

Application of Technology Framework

DATA Services Platform

Data Integration and Govemance Data Application

数据管理

Data Integration

Data Integration Data GovernanceData Governance Data Mining

Data Mining _{Data Analysis}Data Analysis _{Data Retrieval}Data Retrieval

Data Computing

Stream Computing

Stream Computing Parallel ComputingParallel Computing

Data Storage

Relational Database Relational

Database NoSQLNoSQL Distributed File System Distributed File System Metadata Management Metadata Management Model ManagementModel Management Data Service Data Service Data Source BI/Report BI/Report Retrieval/Vi sualization Retrieval/Vi

sualization ApplicationApplicationFunctional Functional Professional Professional ApplicationApplication Predictive Predictive _AnalyticsAnalytics …………

Structural Data Structural

Data UnstructuredData Unstructured

Data

Mass Real Time DataMass Real

Time Data GIS DataGIS Data Data Quality

Data Quality

Data Source

Data Source

Data Integration And Governance

Data Integration And Governance

Data Store

Data Store

Data Calculation

Data Calculation

Data Mining And

Data Mining And AnalysisAnalysis Analytics Application Analytics Application Data Management And Service Data Data Management And Service And Service

Using technologies such as ETL, OGG, extraction in the distribution of structured/unstructured/mass/real-time/GIS data in a business system.

Data Source Data Extraction

ETL ETL SQL SQL WebService WebService Copy Copy OPC OPC

··

··

Target Data Target Data Structured data Structured data Unstructure d data Unstructure d data Mass/real-time data Mass/real-time data GIS data GIS data Data Association Data filteringData filtering Extracts features Extracts features Associate metadata Associate metadata Data standardizationData standardization Data Quality Check Integrity Integrity Accuracy Accuracy Timeliness Timeliness Structured data Structured data Unstructured data Unstructured data Mass/real-time data Mass/real-time data GIS data GIS data …… …… …………

Key Technology 2-Data Storage

Improving the capacity of data storage using infrastructure such as Relational database cluster, Distributed real-time database and Distributed file System.

RDB

RDB RDBRDB RDBRDB Relational Database Cluster

Database management Database management in-memory database in-memory database NoSQL Database NoSQL Database

Distributed real-time database _{Distributed file System} Metadata management Metadata management Access control Access control redundancy strategy redundancy strategy Structured Data Management Platform Structured Data

Management Platform Management PlatformUnstructured Data Unstructured Data

Management Platform Management PlatformMass/real-time Data Mass/real-time Data

Management Platform GIS Data ManagementGIS Data Management_{PlatformPlatform}

Cloud Storage Cloud Storage

Key Technology 3-Data Calculation

Improving the capacity of data calculation with technology such as Parallel Processing Technology, Stream Computing Technology and Indexing Technology.

Parallel Processing

Key Technology 4-Data Mining

Using data mining technology, to build the business analysis model, find business value.

Analytical Application

Data Mining Technology

Power grid state tracking Power grid state

tracking

Load forecast of distribution AnalyticalLoad forecast of distribution Analytical Multi-dimensional Analysis Multi-dimensional Analysis MapReduce MapReduce Electricity sales Analytical Electricity sales Analytical User behavior Analytical User behavior Analytical Parallel computing Parallel computing Data Mining Data Mining Cluster Cluster Association Rules Association Rules Classify Classify Regression Analysis Regression Analysis Time Series Time Series Memory computingMemory computing Intelligence knowledge baseIntelligence knowledge base

··· ··· Semantic engines

Semantic engines

Data Mining Model

State grid analysis Model

State grid analysis Model

Load forecast of distribution ModelLoad forecast of distribution Model

Electricity sales forecast Model Electricity sales

forecast Model User behavior Model User behavior Model Optimize purchasing structure Model Optimize purchasing structure Model ··· ···

Key Technology 5-Data Application and Presentation

Association Association Predictive Predictive Clustering Clustering Display Form Display Vector Variance Variance …

Data presentation: By using different kinds of data presentation, such as charts, GIS, video, and map. To display data analysis results through PC, large screen, tablet, and mobile devices.

GIS Data GIS Data Unstructured Data

Unstructured Data Mass/Real time Data_{Mass/Real time Data}

Integration enterprise layer management model Integration enterprise layer management model Metadata

Metadata Structured Data

Master Data Master Data

Data Quality Management Platform

Data interface monitoring Data interface monitoring

Monitor class

management Data interface monitoring Data access monitoring

Metadata Management

Metadata Management _{Master Data Management}Master Data Management Data Quality Management

Data Quality Management Data schedule management Data schedule

management managementmanagementData service Data service

Data operations manageme nt Data operations manageme nt Code verify management Interface verify

management Data quality statistic Data quality report Verify scene

management Verify result board

Data model manageme nt Data model manageme nt Data stream monitoring Automati c dispatch Connecti on Service Access Service

Key Technology 6-Data Management and Services

Data management and services: By establishing data service platform, to unified manage data storage, calculation, and data mining.

Contents

Application of Technology Framework

Application of Technology Framework

Ⅱ

Ⅱ

Introduction of Power Big Data

Introduction of Power Big Data

Ⅰ

Ⅰ

Data Mining and Visualization

Data Mining and Visualization

Ⅳ

Ⅳ

Power Big Data Quality Management

Power Big Data Quality Management

Ⅲ

Power Big Data Quality Management-Data Link

Business Storage

Stream

Application Storage

Power big data come from various business systems. Through organizing data link, to make sure the management responsibility of each point in data link, to draw the data distribution and flow map.

Power Big Data Quality

Management-Data Link Monitoring

To design monitor rule of data link points in creation, calculation, transformation, and circulation, so that to locate data quality related issues and realize closed-loop

Power Big Data Quality

Management-Technical Standard

Development of data quality technical standard, including data model, data access, application development, and utilization.

Power Big Data Quality

Management-Management Standard

Development of data quality management standard, including data connection, alteration, and utilization.

Contents

Application of Technology Framework

Application of Technology Framework

Ⅱ

Ⅱ

Introduction of Power Big Data

Introduction of Power Big Data

Ⅰ

Ⅰ

Data Mining and Visualization

Data Mining and Visualization

Ⅳ

Ⅳ

Power Big Data Quality Management

Power Big Data Quality Management

Ⅲ

Data Mining-Application Framework

1 1 2 2 3 4 4

The historical data of the lower

layer indicator as the basis,through the data mining for the

2. Vertical Indicator relationship

Data mining of single indicator of historical data, find change law of indicators ,monitor indicator of data quality and business tansaction.

Suitable data mining algorithms: time series, linear fitting.

Through studying the

relevance between the index of two two types of data, build the indicator data correlation network di agram, to find potential business association t o provide decision basis.

Suitable data mining algorithms: correlation analysis, association rules, clustering. 1. Indicator 3. Horizontal

Data Mining1-Indicator

Short-term Power Load Forecasting based on Fuzzy Information Granulation SVM

Predicted Results：

[L,R,U]=[595.24,680.69,727.26]

716 703 690 669 658 648 638 637 644 646 648 635 623 619 602 604 627 637 669 667 669 693 701 698 711 712 727 723 721 708 692 713 716 718 722 710 705 725 706 718 Examp le

Indicator Set

Examp le

weight？% weight ？% weight ？% weight ？% weight ？% weight ？%

Data Mining2-Vertical Indicator Relationship

power distribution reliability Power supply reliability rate User average failure times of interruption User average scheduled interruption time Transformer fault outage rate The average outage time of user Customer average interru ption time …… Customer average insufficient power supply

(27indicators，6 notes，21 parameters)

weight ？%

This scene aims at studying “electricity average interruption duration“ and 27 indicators, to analyze the weights and score of power distribution reliability.

24

Examp le

业务稳定性分析

Through factor

analysis can obtain

the factor of

original 27

indicators:

The average outage time of user Transform er fault outag e rate User average failure times of interruptio n …… Customer average insufficient power supply Factor total value varianc e 0.066 0.052 0.048 …… 0.040 0.5305 weight factor1/facto_{r total} Factor2/fac_{tor total} Factor3/factor _total factor4/factor _{total n/factor total}Factor 1

Data Mining2-Vertical Indicator Relationship

power distribution reli ability The average outage time of user User average failure times of interruption 0.124 0.09 …… …… Transformer fault outage rate Supply

Radius _Service Cost of

Facility

0.10 _0.13

0.15 0.09

Among the indicator rule is starting from one or more concern, research the data regularity between the two indicators and business rules, so as to construct the indicators correlation network diagram, improve business personnel’s understanding of the indicator rule.

Monitoring the lifting point:

(1) through the positive and negative correlation between indicators provide monitoring means for monitoring the quality of data

(2) through the positive and negative correlation between business state can monitor the current integrated service quality

Analysis of lifting points:

(1) the strength of the correlation between indicators, provide the data analysis range of thematic analysis later

(2) the correlation between indicators found potential business rules, business optimization and upgrading to provide analysis basis

Detailed monitoring and data analysis

聚类一 聚类二 聚类三 A市 47 1 B市 48 C市 48 D市 48 E市 18 30 F市 48 G市 48 H市 48 I市 47 1 J市 48 K市 1 46 1 L市 1 47 M市 47 1 N市 48 10年 11年 12年 13年 10年 11年 12年 13年 10年 11年 12年 13年 E市

Power feature with area:

The city E has 60% samples in the cluster-3, 40% samples in cluster-1. From the time trend point of view, city E has the trend to change from cluster-3 to cluster-1 recently.

The city E has 60% samples in the cluster-3, 40% samples in cluster-1. From the time trend point of view, city E has the trend to change from cluster-3 to cluster-1 recently.

Big Data Application Technology

electric power load change.

Power load flow analysis

Power load flow analysis

Usage: (1) each ribbon r epresents the load of corresponding city; (2) ribbon width

represents load value or the percentage of load in the province; (3) by moving the

Usage: (1) each ribbon r epresents the load of corresponding city; (2) ribbon width

represents load value or the percentage of load in the province; (3) by moving the Use the data flow diagram to show the area of enterprise power load variation with time changes.

Big Data Application Technology

Treemap Analysis

Treemap Analysis

(1) the block area on behalf of

the blocks

represent the total applied electricity capacity;

(2) using the nested model conveniently

show multilevel data constitute relationship.

(1) the block area on behalf of

the blocks

represent the total applied electricity capacity;

(2) using the nested model conveniently

show multilevel data constitute relationship.

Using Treemap to show the detailed data of various categories.

Examp le

Bubble chart analysis

Bubble chart analysis

The chart records different electricity capacity increase of cities. As we can see, the pink

one(rural residents living electricity) is the largest part of every city

Big Data Application Technology

To access different electricity

capacity increase of cities, through the analysis of the data, for

each city using a bubble

chart display, a bubble chart size represents electricity increase that appeared, the bubble number withi n a block represents the number of classification of the

electricity increase that the staff had made.