Big Data
-
Business, Math, Technology – Best combination for big
data
商业理解，数据科学，技术实践之完美结合
Agenda
1. Big Data Trends
Big Data and Analysis Gradually Infiltrate into Every Aspect of Enterprises
大数据与分析逐渐渗透到企业各个方面
Application-oriented Data analysis-oriented
Transform Architecture transformation
Platform
moving, cloud
computing
DataApplication
Analysis capability Analysis capability Analysis capability Data Data Data+
2. A comprehensive big data
technology platform is required.
需要全方位的大数据
技术平台
3. Data science analysis technology is complex and
has high requirements on skills.
数据科学分析技术复杂和对技能要求高
Industry solutions
quickly generate business income. Low TCO, SaaS
Platforms Low maintenance costs Performance Scalability Tools IT-driven service automation Services Competitive, standardized, and industry experience Vertical BigData
1. Business Analysis will determine
the business transformation.
Application
Application Application
Big Data and Analysis Trends
大数据趋势
New Buyers and Users
New Sources of Data New Actionable Insights New IT Economics
Transformational Market Drivers
Non -Traditional Players/ Open Source Specialized Offerings Agility Lighter Footprint (offering, economic) Faster Evaluation to Deployment
Disruptive
Competitive Forces
Client Challenges
Limited Analytic Skills & Understanding
Information Blind Spots Time-to-Value Falls Outside Window of Opportunity Lacking Analytics Strategy/ No Roadmap
Demand for New Economic Models
3
2
1
4
Drive Agility in the Information Supply Chain
accelerate time-to-value - foundation for Analyticscurate across data sources
Create Deeper Business Relevance
solutions by role & industry - whole product experienceRedefine the Experience
fits the user’s work - engaging & obvious – collaborative automated insights
Transform Delivery of Business Outcomes
cloud - mobile - deployment - open source - fits economic modelResolving Big Data Problems from Three Dimensions
从
BMT
三个维度出发解决大数据问题
Business: Analyze customer service
problems
Math: Methodology and tools
Technology : Big data platform
technologies
Business
Math
Technology
Big data industry
solution service
Continuous
iteration
To resolve customer problems, Huawei provides big data analysis consulting services, big data
platform services, and big data industry solution services based on business understanding,
data science, and IT practices, reflecting the service concept.
Agenda
1. Big Data Trends
Huawei Big Data Solution: Business, Math, Technology –
Best combination
Mass data storage, batch processing, iterative processing, and real-time stream processing
Manager
Unified
management
RH2288 Generic X86 server OceanStor 9000Big data storage
Data insight
platform
Data processing
platform
Big data
infrastructure
FusionInsight
Data integration
platform
Collect
Clean
Change
Feature/Model/Mining/Visualization/ServiceService-related application suite (service logic/decision-making/security/data open/ visualization...)
Application
suite layer
Telecommunication
CDR query, operation analysis, and precision marketing
Bank
Entire life cycle analysis, historical details, precision marketing, online credit investigation and risk
control, etc.
Industry
applications
Public security
Gate data analysis Intelligence analysis Population management
Math - Huawei Labs Focus on Key Technologies in the Big Data
数据科学
–
华为各试验室全面覆盖大数据领域关键技术
Nanjing Research Center:
ETL/ Real-time decision-making
(Application and Software R&D
Center)
Shenzhen R&D center/Hong Kong:
Big data analysis platform
(IT R&D Center)
Big data management platform
(IT R&D Center)
Data mining algorithm (Noah Lab)
India Research Center:
Hadoop (Central Software Institute)
Data visualization
(Central Software Institute)
MOLAP (Central Software Institute)
America Research Center:
MPP DB and TP DB (Gauss Lab)
MOLAP (Central Software Institute)
Spark (Central Software Institute)
European Research Center:
Distributed memory DB
(Central Software Institute)
Distributed computing
algorithm (France, Central
Software Institute)
Hangzhou Research Center:
Spark (Central Software Institute)
Stream computing (Central
Math - Huawei's Innovative Research Achievements in the Data Analysis and Mining
数据科学
–
华为在数据分析和挖掘领域多项创新研究成果
Tendency prediction
Character portrayal
Topic extraction
Relationship estimation
Feature management analysis Automatic feature construction
倾向预测
人物刻画
主题提取
关系估计
Technology - Hadoop Has Become the Standard for Big Data
技术实践
- Hadoop
已成为大数据事实标准
•
First phase (2005 to 2009): Simulation of the Google troika. The leaders were Internet vendors such as Yahoo and Facebook. The related items are
①②③④
.
•
Second phase (2009 -): Simulation of the new Google troika. The leaders are Hadoop distribution vendors such as Cloudera and Hortonworks.
Traditional IT vendors, such as IBM, EMC, Intel, and Huawei start to integrate Hadoop. Hadoop enters the enterprise market. The related item is
⑤
.
•
Third phase (2012 -): Combination of successful experience. The AMP Lab and stream computing achievements are adopted to construct a big
data ecosystem by using matching tools, forming a de facto standard. The related item is
⑥
.
Six Hadoop distribution vendors
Technology - FusionInsight Hadoop
技术实践
- FusionInsight Hadoop
由来
2007
2011
2013
Continuous optimization and feedback to the community
Telecom performance-oriented
Reliable secure self-management, enterprise distributions
Open source tracing research
Component matching GalaX HD
Independently released FusionInsight HD
Yarn HIVE/Impala HBASE HDFS M/R Spark Porter Miner DataFarm Hadoop Storm Solr System management Farmer Oozie Metadata management Manager
Hadoop API Plugin API OpenAPI/SDK
Application layer
REST/SNMP/Syslog
Zookeeper
Data Information Knowledge wisdom
FusionInsight Hadoop provides a big data
processing environment.
FusionInsight DataFarm provides an
end-to-end data farm for data insight and supports a
closed-loop data value-added life cycle:
data-information-knowledge-intelligence.
Under fierce Internet financial competition, a decision-making and service system based on big data analysis must be constructed to improve self-competitiveness and customer satisfaction.
With the increase of financial data amount and types, traditional data processing platforms cannot meet the requirements of the big data era because these platforms apply only to structured data and have low scalability and high capacity expansion costs.
Customer challenges
Huawei solutions
Enterprise-level big data platform: High reliability, high security, easy management, and easy development.
The first big data analysis and mining platform supports classified protection for
financial data and over 1000 km remote disaster recovery, seamlessly integrating with enterprise applications.
Auxiliary capabilities: Data insight platform Miner and data service platform Farmer. A core development project team, and consulting and customization service
capabilities.
Customer benefits
Unified solution covering the entire life cycle of financial big data.
A variety of innovative services: Online details, real-time credit investigation, and
precision marketing.
Potential customer forecast for small and micro financial credit, which improves
the conversion rate by 40 times; the error rate of contingent financial assets is reduced by half; the credit investigation duration for credit cards is reduced from 15 days to 30 minutes.
Business - Huawei Assists CMB with Internet Financial Transformation by Constructing a Big Data
Platform
商业理解
–
协助招商银行构筑大数据平台向互联网金融转型
我们把大数据应用视作是生命线，一定是采用大数据
平台企业版，搞开源软件不是我们的主业。在选合作
伙伴的时候，我们一定考虑门当户对，因为强有力的
合作伙伴才能保证
3~5
年的供应、合作安全”
——
招商银行
CIO
Under the chimney construction, multiple applications are independently stored and data cannot be shared. The cross-department data obtaining takesseveral months.
A lack of effective data assets management. The accurate amount of data or numbers of models and rules are unknown, and the data security is under threats.
The data amount supported by the existing system is limited. The larger the data amount is, the slower the analyzing and processing speed is.
Customer challenges
Huawei solutions
Unified enterprise-level big data platform, which supports the data hierarchy storage. Different data is stored in different storage.
Unified data asset management and data security management. Shared data access interfaces and capability open interfaces.
Linear capacity expansion, which retains the concurrent processing speed for a large amount of data.
Customer benefits
PB-level data storage capability, which ensures high concurrent processing speed for multiple applications.
The platform and application are decoupled, and data between applications is shared, speeding up the application development and deployment.
Effective data asset management, which speeds up the data value mining.
Business - Huawei Helps Shanghai Unicom Improve Service Innovation Efficiency
Huawei big data solution
Hadoop Enterprise Edition
Huawei advantages
Commercial cases
1 year -> 5 years
1 day -> 15 minutes
1 day -> 1 hour
Inventory customer loyalty:
86%
Half a year -> 1 year
Real-time credit investigation
2 weeks -> 30 minutes
The first full-component HA
in the industry
The first one under
financial-level protection in the industry
Five times of performance
improvement compared with
open source
1st
Performance
Ranks first in community
1st Input 1st Reliability 1st Security
Capacity expansion cost
xxK$/T -> xK$/T
Business + Math + Technology = Customer Innovation
商业理解
+
数据科学
+
技术实践
=
成就客户创新
Historical service details
Internet banking customer
behavior analysis
Data inspection service
Shanghai Unicom unified big
data platform
Jiangsu Mobile CDR cloud
Philippines PLDT intelligent
big data platform
历史明细业务 网银用户行为分析 数据稽查业务 上海联通统一大数据平台 江苏移动详单云 实时征信 菲律宾PLDT智能大数据平台
