Low-Level Verification of Embedded Software: Addressing the Challenge

(1)

Low-Level Verification of

Embedded Software:

Addressing the Challenge

Low-Level Verification of

Embedded Software:

Addressing the Challenge

Sanjit A. Seshia

Assistant Professor Assistant Professor EECS, UC Berkeley EECS, UC Berkeley FMCAD 2010 Panel October 2010

(2)

– 2 –

Abstraction Layers in Computing

Algorithms, Protocols, Models

Application Software

Systems Software / Firmware

Architecture

Circuits

Devices

HW

SW

(3)

– 3 –

What makes Software “Low-Level”?

(from Verification perspective)

What makes Software “Low-Level”?

(from Verification perspective)



Properties



Software is low

-

level

if the behavior

of the software system is defined

significantly by lower levels

of

abstraction (hardware platform)



(4)

– 4 –

Quantitative Analysis / Verification

Does the brake-by-wire software

always actuate the brakes within 1 ms?

Safety-critical embedded systems

Can this new app drain my iPhone battery in an hour?

Consumer devices

How much energy must the sensor node harvest for RSA encryption?

Energy-limited sensor nets, bio-medical apps, etc.

(5)

– 5 –

Cyber-Physical Systems (CPS):

Orchestrating networked computation with physical systems

Cyber-Physical Systems (CPS):

Courtesy of Kuka Robotics Corp.

Courtesy of Doug Schmidt

Power generation and distribution Courtesy of General Electric Military systems: E-Corner, Siemens Transportation (Air traffic control at SFO) Avionics Telecommunications Factory automation Instrumentation (Soleil Synchrotron) Daimler-Chrysler Automotive Building Systems

(6)

– 6 –

Cyber-Physical Systems (CPS):

Cyber-Physical Systems (CPS):

Courtesy of Kuka Robotics Corp.

Courtesy of Doug Schmidt

Power generation and distribution Courtesy of General Electric Military systems: E-Corner, Siemens Transportation (Air traffic control at SFO) Avionics Telecommunications Instrumentation (Soleil Synchrotron) Daimler-Chrysler Automotive Building Systems Factory automation

(7)

– 8 –

Time is Central to Cyber-Physical Systems

Several timing analysis problems:

Several timing analysis problems: 

 Worst_Worst-_-case execution time (_{case execution time (}WCET_WCET) estimation_{) estimation}



 Estimating _Estimatingdistribution_distribution of execution times_{of execution times}



 Threshold_Threshold property: can you produce a test case _{property: can you produce a test case}

that causes a program to violate its deadline?

that causes a program to violate its deadline? 

 Software_Software-_-in_in-_-the_the-_-loop simulation_{loop simulation}: predict _{: predict}

execution time of particular program path

(8)

– 9 –

Challenge: Environment Modeling

(Timing Analysis)

Challenge: Environment Modeling

(Timing Analysis)



 Timing properties of the Program depend heavily _{Timing properties of the Program depend heavily}

on its environment

on its environment 

 Environment =_{Environment =}

Processor & Memory Hierarchy

+

+ Operating System, other processes/threads, Operating System, other processes/threads, ……

+

+ NetworkNetwork

+

+ I/O DevicesI/O Devices

+

+ ……



 Modeling the full environment is hard!_{Modeling the full environment is hard!}



 However, we need a _{However, we need a}‘_‘reasonably_reasonably’_’ precise _precise

environment model

–

(9)

– 10 –

Success of “High-Level” Software

Verification

Success of “High-Level” Software

Verification



 From theoretical ideas to industrial practice in _{From theoretical ideas to industrial practice in}

~ 15 yrs

Some Reasons:

Some Reasons: 

 Availability of open source software_{Availability of open source software}



 Well_Well-_-defined target problems: Device drivers, _{defined target problems: Device drivers,}

memory safety, security vulnerabilities,

concurrency,

concurrency, ……



 Value of bug findingValue of bug finding



(10)

– 11 –

Challenge of Timing Analysis: Example

flag==1 flag==1 flag=1; (*x)++; CFG unrolled to a DAG *x += 2; On a single-core processor with a data cache x

Timing of an edge (basic block) depends on:

• Program path it lies on • Initial platform state

Challenges:

• Exponential number of paths and platform states! • Lack of visibility into

(11)

– 12 –

Current State-of-the-art for

Timing Analysis

Current State-of-the-art for

Timing Analysis



 Program = Sequential, _{Program = Sequential,}

terminating program

terminating program 

 Runs uninterrupted_{Runs uninterrupted}



 Environment = _{Environment =}

Single

Single--core Processor + core Processor + Memory Hierarchy

Memory Hierarchy

Timing Model

PROBLEM:

Can take several

man-months to construct!

Also: limited to extreme-case analysis

(12)

– 13 –

Existing Approaches: One-size-fits-all?



Why construct a

SINGLE timing model

for ALL programs?



Only interested in

analyzing a specific

program.



Why not

automatically

synthesize

a

program

-

-specific

(13)

– 14 –

Promising Direction

(for timing analysis and low-level verification in general)

Promising Direction

(for timing analysis and low-level verification in general)



 Inductive Synthesis_{Inductive Synthesis}

–

– Automatically generate environment model Automatically generate environment model through

through active learning active learning



 Active = Select behaviors from which to learn_{Active = Select behaviors from which to learn}



 Use core verification techniques (SAT, SMT, _{Use core verification techniques (SAT, SMT,}

model checking,

model checking, ……) to generate selected ) to generate selected behaviors

behaviors 

Example: _Example:GameTime_GameTime for timing analysis of _{for timing analysis of}

software

S. A. Seshia and A. Rakhlin, “Quantitative Analysis of Embedded Systems Using Game-Theoretic Learning”, ACM Trans. Embedded Systems.

(14)

– 18 –

Estimating the Distribution of Times

for Modular Exponentiation: predictions from 9 measurements in blue,

actual 256 measurements in red

Estimating the Distribution of Times

for Modular Exponentiation: predictions from 9 measurements in blue,

actual 256 measurements in red

For StrongARM processor

(15)

– 19 –

Potential Barriers

(

from Academic Perspective)

Potential Barriers

(

from Academic Perspective)



 Lack of OpenLack of Open--Source BenchmarksSource Benchmarks

–

– Recent progress in software verification was Recent progress in software verification was driven by wide availability of open

driven by wide availability of open--source source software

software

–

– More challenging for More challenging for ““low levellow level”” software software verification!

verification!

–

– Heavy dependence on platform makes it more Heavy dependence on platform makes it more challenging

challenging



 Hardware + Software Skills_{Hardware + Software Skills}

–

– Students need crossStudents need cross--cutting skills (or willingness cutting skills (or willingness to learn) to work in this area

(16)

– 20 –

Summary



“

Low level

”

software =

Software

whose

behavior is significantly defined by hardware

–

– HardwareHardware--Software Verification?Software Verification? 



Challenge:

Environment modeling

–

– Current manual methods too tedious and errorCurrent manual methods too tedious and error- -prone

prone



Proposed Approach:

Automatic model

generation by Inductive Synthesis

–

– Active Learning + Traditional verification Active Learning + Traditional verification techniques

techniques (e.g., SAT/SMT)(e.g., SAT/SMT)

–

One instance:

GameTime

for timing analysis

of software

–