Tectonic Change, Big Data and the Test & Evaluation

(1)

Tectonic Change, “Big Data”

g ,

g

and the Test & Evaluation

Ch ll

Challenge

(2)

Food & Water

Covert Action Nuclear Programs

Hybrid Warfare

Islamic Extremism

UNCLASSIFIED

Cloud Computing

Arab Spring

Hybrid Warfare

Energy Demands

Terrorist Attacks

BIG DATA

Amplifiers…Seismic…..TECTONIC

Arab Spring

A Transformed World

••

Globalization….

Virtualization

••

Demographic Trends

••

Resource Competition

••

WMD Proliferation

••

Failed / Failing States

••

Climate Change

Find Fix Finish

Units / Individuals / NETWORKS

--

Knowledge Management

-- Power of Anonymity…Identity Resolution

Power of Anonymity…Identity Resolution

-- Age of the Strategic Private LES/CES

Age of the Strategic Private LES/CES

-- Age of the Strategic Innovator

Age of the Strategic Innovator

Culture

…….

ACCESS

••

Expertise has moved to the Edge

••

No more Order of Battle

PREDICTIVE ANALYTICS

Every Paradigm Has Changed

No more Order of Battle

F3EAD

(3)

We live in a society

exquisitely dependent on

science and technology,

in which hardly anyone

knows anything about

science and technology

(4)

902.4 PB – Size of YouTube

_{$187/second – Cost of last Ebay Outage}

282 Milli

I t

t

b it

6.9 Billion People….4.3 Billion Mobile Devices

800 Million Facebook Users

294 Billion – Average emails/Day

282 Million – Internet websites

30 Billion – Pieces shared on Facebook/Month

_{F-35 A Terabyte per Sortie}

460,000 New Twitter Accounts per Day

y

p

1 Billion PCs and Laptops

100 Million Gigabytes – Size of Google’s Index

1 Billion Tweets/Week, 177 Million 3/1/11

152 Million – Internet Blogs

107 Trillion – emails sent in 2010

340x10

23 _{– Size of IPv6 Address Space}

100 Million Gigabytes Size of Google s Index

6,939 1/1/11 Current Tweets Per Second record - Japan 4 seconds after Midnight*

1.98 Billion – Internet Users

(5)

The Information Being Created Today is “Big Data”…

Massive Amounts of File-based, Unstructured Data

Video Surveillance

and Graphics

Satellite Imagery

UNCLASSIFIED

(6)

If a Grain of Sand were One Byte of Information . . .

A Vocabulary for Measuring Information Data

1 Megabyte =

1 million bytes

a tablespoon of sand

1 Exabyte =

1,000 petabytes

the same beach—

from Maine to North Carolina

1 Gigabyte =

1 billion bytes

patch of sand—

9” square 1’ deep

a tablespoon of sand

1 Zettabyte =

1,000 exabytes

the same beach—

along the entire US coast

9 square, 1 deep

1 Terabyte =

1 trillion bytes

a sandbox—

24’

1’ d

g

1 Yottabyte =

1,000 zetabytes (24 Zeroes)

enough info to bury the entire

US under 296 feet of sand

24’ square, 1’ deep

1 Petabyte =

1,000 terabytes

a mile long beach—

US under 296 feet of sand

……And if you’re talking about

communications…….bits…….

MULTIPLY BY EIGHT!

g

(7)

The Ever Growing Volume and Complexity of

Data Collection, Warehousing and Discovery

Tectonic

Shift

,

g

y

• A DIGITAL FOOTPRINT...

IN THE MODERN WORLD, EVERY HUMAN AND MECHANICAL

ACTIVITY LEAVES A TRAIL OF DATA....

• Phone calls, credit card transactions, blogs, medical records, cash withdrawals, bar code

swipes, bank interaction, passports, visas, Tweets, web use, e-mail messages, Government

data…

CONSTELLATIONS OF PLATFORMS TIED TO NETWORKS OF SENSORS ALL

INTEGRATED INTO THE INTERNET

• High- capacity microwave radio-relay systems, satellite links, fibre optic submarine cables

carrying mixed traffic of voice/VOIP/video/fax/e-mail and internet data video surveillance

carrying mixed traffic of voice/VOIP/video/fax/e-mail and internet data, video surveillance

cameras, fingerprint scanners, CBRNE (chemical, biological, radiological, nuclear, explosive)

detection devices, Airline Passenger Name Records, Phone call detail records, satellite and

UAV images,

4.5 million CCTV cameras in UK ....30 million in US

• New families of sensors are constantly being created, each sending an increasing amount of

data (High Density Images, Video, Voice, Data, Multispectral).

NETWORK SECURITY, DATA FORENSICS AND TRACKINGIS INCREASINGLY DIFFICULT –

SMALLER DATA PACKETS – MULTIPLE ROUTES – DEEP PACKET INSPECTION (DPI) AND

CLOUD TRUST ARE KEYs TO FUTURE SUCCESS

INTELLIGENCE

(8)

“Our problem is we have historically been platform

and sensor-centric giving only lip-service to the

and sensor centric, giving only lip service to the

crucial processing and analytic activities which

transform information into intelligence. More and

more constellations of platforms each with

increasingly more capable families of sensors only

provide us with more mountains of data, making

actionable intelligence even more difficult to

g

derive. Developing technologies to sift and

analyze mind-numbing amounts of data while

interacting with analysts in real time has got to be

our priority in the future.”

General John P. Abizaid

Commander, US Central Command

March 2007

(9)

The ISR T&E Dilemma

Cloud

NETWORK SECURITY

BIG DATA

Intelligence

Cloud

BIG DATA

BACK-UP &

RECOVERY

TAGGING

REAL-TIME

CONUS

PREDICTIVE

ANALYSIS

BIG DATA

STORAGE

ANALYTIC

TOOLS

Launch &

Recovery

Ground

Control

Station

Tactical Operations Center

C2 and Analysis

How do you Test and Evaluate BIG DATA Systems of Systems Architectures

based on Multi-Platform, Joint, and Interagency Interoperability?

(10)

Where We Stand Today

• AN EMERGING ERA OF CLOUD COMPUTING AND

BIG DATA

• How do we Test in a New Data Paradigm?

• Can Data Overwhelm Testing?

• Requires a Relevant and Contested Environment

• What does PreDeployment Testing Look Like?

at does

e ep oy e t est g oo

e

• Speeds and Capabilities of Operational Systems

• How to Evaluate and Validate ‘The Cloud?”

• A Ubiquitous Cyber Security Envelope

• Real-Time Analytics will Define the Era

• Data Minutes Old is Stale

• SIMULATING AND EVALUATING KNOWLEDGE

MGMT

• Cyber Security Design Test requirements

• By 2013…50% of Devices on Internet will not be PCs…By

2015 only 20% PCs

In Every Future War we will use

2015 only 20% PCs

• Transitioning to Billions of Concurrent Users

• Platforms Will Produce a Petabyte a Day

• No Future US Monopoly on Situational Awareness…ISR will

Globalize and Commoditize

• Storage is a Commodity…but Raw vs Usable matters

In Every Future War we will use

ONE TENTH the Force utilized

in the Previous Conflict and

ONE HUNDRED Times the

Storage is a Commodity…but Raw vs Usable matters

Petabytes…Hyperscale…The New Normal

1.

-Bandwidth

(11)

T&E Infrastructure Study

• The Under Secretary of Defense (Acquisition,

Technology and Logistics) has requested TRMC to

assemble an objective expert team to conduct a

study to

determine how T&E resource costs can

study to

determine how T&E resource costs can

be significantly reduced to provide

adequate and robust testing at incrementally

lower budget levels.

(12)

Identify

high payoff

areas for potential

savings

Scope of the T&E Infrastructure Study

Identify

high-payoff

areas for potential

savings

Determine DoD T&E

infrastructure requirements

and proper

stewardship

of specific test capabilities to include:

Capabilities at other Federal Agencies

Capabilities at commercial contractors

Based on infrastructure requirements,

identify capabilities required

to meet future weapon system needs

Identify

alternatives

and develop

recommendations for operation

and maintenance support of test facilities

Develop recommendations with advantages/disadvantages of alternatives

Assess potential business model alternatives, funding policy

alternatives and technology initiatives

(13)

The Testing Capability

Gap

How Do You Test a Petabyte

Sensor Network?

BBBB

Sensors and Data

Growing Exponentially

(14)

Around The Corner

• KEY DRIVERS OF THE FUTURE

–

Virtualization VDC/VDI

–

Containerized Data Centers Enable the

Cloud

–

Broadband will Enable the Video

Explosion

–

Hi-Fidelity Search

y

• “Predictive Learning Curiosity

Engines”

–

Increasingly Intelligent and Independent

NETWORKS

THE ENTERPRISE IS THE EDGE

• THE ENTERPRISE IS THE EDGE

–

The Network is the Formation of the

Future

• All Information Will Be Digital

• Users Will Employ Any and All

How Do We Create a

N

T

t

d

Devices

p y

y

• All Devices will be Network Nodes

• Stand Alone Products will Die

–

CONVERGED INFRASTRUCTURE

• Virtualized Integrated and

New Test and

Evaluation Regime?

Virtualized ,Integrated and

Preconfigured Networks, Compute,

Storage and Security

(15)

Implications for Test and Evaluation

Volume

• PETABYTES are the New Normal

PETABYTES are the New Normal ….HYPERSCALE ….How Long Until EXABYTES?

HYPERSCALE

How Long Until EXABYTES?

• Store, Manage and Analyze Vast Amounts of Data

• Scale-out Virtualized, Dynamic, Elastic Architectures

Velocity

• Real Time Situational Awareness

• Converged Compute, Storage, Networking and Security….Encrypted Enterprises

Variety

• Structured and Unstructured Data

• Subtle Networks of Connections

Simulated Assets and Feeds

• Subtle Networks of Connections…Simulated Assets and Feeds

• Detect Non-Obvious Patterns and Links between Systems, Events, Places and Timeframes

Complexity and Speed of Analysis

• Rapid, Rich, Enhanced Analytics for Big Data Sets…

CHAFF IS THE NEW WHEAT

• Correlation NOT Search…Ad Hoc, Interactive Analysis ….NOT Simple Structured Reports

• Provide Analysts the Ability to Explore and Manipulate the Data

Budget

• Maintain SLAs as Volume Dramatically Expands….$$$

UNCLASSIFIED

a ta

S

s as o u e

a at ca y

pa ds

$$$

(16)

MetaData Challenges for

21 st

Century Test and Evaluation

21 Century Test and Evaluation

• Exponential growth in data quantity, sources,

types and required analysis capability

• Metadata Creation must be Automated for

Test and Evaluation…Must be a New Priority

• Insufficient bandwidth to move data for

Insufficient bandwidth to move data for

centralized analysis necessitates federated

massively parallel analytics

• Lack of performance for advanced analytics

Lack of performance for advanced analytics,

complex queries and predictive results

• Necessary shift from Network Centricity to

Data Centricity

(17)

Down The Road

• COLLABORATION….PRODUCTIVITY

TOOL OF THE FUTURE

–

Collaboration -No Longer an Option

–

Smart Networks…Collaboration will

Dictate T&E Efficiencies

&

–

SIPR and NIPR will vanish….replaced

by Functional CoEs in the Cloud

–

Data-Centric NOT Net-Centric—

Operate Under the Assumption of

Breach

–

Network Technology…so deeply

embedded it can no longer be

separated from the Mission

–

IT is driving a Confluence of

Intelligence - Operations – Logistics

across the Military

(18)

Down The Road

• TEST & EVALUATION

CHALLENGES

–

Modern Telemetry

–

Fifth Generation Aircraft

–

Cyberspace

–

Autonomous Vehicles

–

Electronic Warfare

H

D W C

t

ALL BIG DATA

PROBLEMS

-

How Do We Create a

Contested Architecture for

Testing?

PROBLEMS

• Finding Emerging

Technologies to Assess and

Evaluate Emergent Threats

g

-

What Are The Tenets of

21 st

_{Century T&E Cyber}

Security?

Evaluate Emergent Threats

(19)

A Look Ahead

1. Prepare for Disruption

2. Cloud will enable

Next Generation IT

3. Next Killer App:

Big Data Analytics

4. Trust Will Be The Key

The Toughest

C lt

t Ch

i

4. Trust Will Be The Key

5. BIG Data Demands a

New T&E Paradigm

Culture to Change is

One that’s been

Successful

e

&

a ad g

(20)