SIGNALING PROCESSOR NETWORK OPERATIONS CENTER SIGNALING PROCESSOR

(1)

QoS-enabled Middleware for Managing and Controlling High-Speed Networks

Douglas C. Schmidt

Associate Professor & Computer Science Dept.

Director of the Center for Washington University, St. Louis Distributed Object computing www.cs.wustl.edu/

schmidt/

Sponsors

NSF, DARPA, Bellcore/Telcordia, BBN, Boeing, CDI/GDIS, Hughes, Kodak, Lockheed, Lucent, Microsoft, Motorola, Nokia, Nortel, OCI, OTI,

SAIC, Siemens SCR, Siemens MED, Siemens ZT, Sprint, USENIX

April 13th, 1999

Embedded Middleware Douglas C. Schmidt

Motivation: the High-speed Network Software Crisis

www.arl.wustl.edu/arl/

Symptoms

– Network element

hardware

gets smaller, faster, cheaper

– Control/management

software

gets larger, slower, more expensive

Culprits

–

Inherent

and

accidental

complexity

Solution Approach

– Manage & control network elements via QoS-enabled embedded

middleware

Washington University, St. Louis 1

General Switch Management Protocol (GSMP)

SIGNALING CLIENT

SIGNALING PROCESSOR

GSMP SERVER Middleware

IIOP ADD_BRANCH SIGNALING

CLIENT

GSMP SERVER GSMP Messages

NETWORK ELEMENT GSMP LIB

Conventional Implementation CORBA Based Implementation ADD_BRANCH

NETWORK ELEMENT SIGNALING PROCESSOR

T A O

Features

Setup & release connections

Add & delete

point-to-multipoint leaves

Manage ATM switch ports

Request configuration information & statistics Low-level C APIs send RFC 2297 GSMP ATM messages

GSMP Proxy Server Configuration

A T M Switch C O N T R O L P R O C E S S O R

Network Switch G S M P S E R V E R

P R O X Y G S M P SERVER

PROXY CONTROL PROCESSOR SIGNALING

S E R V E R

SIGNALING PROCCESSOR

TAO - Middleware connection request

Inter-ORB Protocol

Establish Connect method

A T M Switch

Network Switch

A T M Switch

Network Switch Control cells G S M P

P R O C E S S O R GSMP message

add_branch

config port SIGNALING

S E R V E R

SIGNALING PROCCESSOR

C O N T R O L P R O C E S S O R

Features

Supports standard CORBA programming API

Can use standard ORB

Transparent to existing GSMP servers

Scales to distributed configuration

– i.e., one CP can control multiple switches

(2)

GSMP Embedded ORB Configuration

M A N A G E M E N T A G E N T

N E T W O R K O P E R A T I O N S C E N T E R

SIGNALING AGENT

S I G N A L I N G P R O C E S S O R

A T M Switch Control Processor

Network Switch G S M P server TAO - Middleware

Host A Host B

Connect Request

add_branch

Connect Request I N T E R-ORB P R O T O C O L

get stat

Features

Leverages middleware flexibility and

standardization

Multiple protocols can be supported

– GSMP in-line bridging, IIOP, etc.

Washington University, St. Louis 4 EmbeddedMiddlewareDo

Goal: Appl y E mbed ded Mid dle ware to Netw ork Element M ana g ment & Contr o l

MANAGEMENT CLIENT RIO

TAO

NETWORK OPERATIONS CENTER SIGNALING SERVER RIO

TAOSIGNALING PROCESSOR SIGNALING SERVER RIO

TAOSIGNALING PROCESSOR GSMP SERVER SWITCH CONTROLLERTAORIO WUGSWUGSWUGS

ADD_BRANCH ADD_BRANCHGET STATS

EVENT PORT DOWN CONNECT_REQUEST ADD_BRANCH GET STATS

ENDSYSTEM A UNI CLIENT

TAOInter-ORB Protocol

CONNECT_REQUEST ENDSYSTEM A UNI CLIENT

TAO ATM SWITCHATM SWITCHATM SWITCH

GSMP SERVER SWITCH CONTROLLER

TAORIO GSMP SERVER SWITCH CONTROLLERTAORIO DomainChallenges

High-speed (20 G bps) A TM s w itches w/embedded controllers

Lo w-latency and statistical real-time deadlines

CO TS infr astr ucture , open systems , and small footpr int

WashingtonUniversity,St.Louis

Problem: Lack of QoS-enabled Middleware

LOW LOW--LEVELLEVEL MIDDLEWARE MIDDLEWARE OPERATING OPERATING SYSTEMS SYSTEMS &&

PROTOCOLS PROTOCOLS HIGHER HIGHER--LEVELLEVEL MIDDLEWARE MIDDLEWARE

COMMON COMMON SERVICES SERVICES APPLICATIONS APPLICATIONS

Cons EVENT Cons EVENT CHANNEL

CHANNEL ConsCons Cons Cons

Many telecom applications require QoS guarantees

– e.g., call-processing,

network/switch management, wireless

Building these applications manually is hard

Existing middleware doesn’t support QoS effectively

– e.g., CORBA, DCOM, DCE, Java

Solutions must be integrated horizontally & vertically

Candidate Solution: CORBA

INTERFACE

REPOSITORY IMPLEMENTATION

REPOSITORY COMPILERIDL

DII ORB

INTERFACE

ORB

ORB CORECORE ^GIOP^GIOP//IIOPIIOP//ESIOPSESIOPS IDL

IDL

STUBS STUBS

operation() operation()^{in args}

out args + return value

CLIENT OBJECT

(SERVANT)

OBJ REF

STANDARD INTERFACE STANDARD LANGUAGE MAPPING

ORB-SPECIFIC INTERFACE STANDARD PROTOCOL INTERFACE

IDL

SKELETON DSI

OBJECT ADAPTER

www.cs.wustl.edu/

schmidt/corba.html

Goals of CORBA

Simplify distribution by automating – Object location &

activation – Parameter

marshaling – Demultiplexing – Error handling

Provide foundation

for higher-level

services

(3)

Caveat: Limitations of CORBA for QoS

INTERFACE

DII ORB

INTERFACE

ORB

ORB CORECORE ^GIOP^GIOP//IIOPIIOP//ESIOPSESIOPS IDL

IDL

STUBS STUBS

operation() operation()^{in args}

out args + return value

CLIENT OBJECT

(SERVANT)

OBJ REF

STANDARD INTERFACE STANDARD LANGUAGE MAPPING

ORB-SPECIFIC INTERFACE STANDARD PROTOCOL INTERFACE

IDL

SKELETON DSI

OBJECT ADAPTER

www.cs.wustl.edu/

schmidt/RT-ORB.ps.gz

Limitations

Lack of QoS specifications

Lack of QoS enforcement

Lack of real-time programming features

Lack of performance optimizations

Overview of the Real-time CORBA Specification

OS KERNEL

OS I

OS I//O SUBSYSTEMO SUBSYSTEM NETWORK ADAPTERS NETWORK ADAPTERS

MUTEX MUTEX IDL IDL END END--TOTO--ENDEND

PRIORITY PRIORITY PROPAGATION PROPAGATION

ORB

ORB CORECORE

OBJECT OBJECT ADAPTER ADAPTER

CLIENT CLIENT

GIOP GIOP

OBJECT OBJECT ((^SERVANT^SERVANT))

PROTOCOL PROTOCOL PROPERTIES PROPERTIES

THREAD THREAD POOLSPOOLS EXPLICIT

EXPLICIT BINDING BINDING

NETWORK NETWORK

OS KERNEL OS KERNEL

OS I

OS I//O SUBSYSTEMO SUBSYSTEM NETWORK ADAPTERS NETWORK ADAPTERS

Features

1. End-to-end priority propagation 2. Protocol properties 3. Thread pools 4. Explicit binding 5. Mutex IDL

Our Approach: The ACE ORB (TAO)

NETWORK NETWORK ORB

ORB^RUN^RUN--^TIME^TIME SCHEDULER SCHEDULER

operation() operation()

IDL IDL STUBS STUBS

IDL SKELETONIDL SKELETON in args

in args out args + return value out args + return value CLIENT

CLIENT

OS KERNEL OS KERNEL

HIGH HIGH--^SPEED^SPEED NETWORK INTERFACE NETWORK INTERFACE

REAL REAL--^{TIME I}^{TIME I}//^O^O

SUBSYSTEM SUBSYSTEM

OBJECT OBJECT ((SERVANTSERVANT))

OS KERNEL OS KERNEL

HIGH HIGH--^SPEED^SPEED NETWORK INTERFACE NETWORK INTERFACE

REAL REAL--^{TIME I}^{TIME I}//^O^O

SUBSYSTEM SUBSYSTEM ACE

ACE COMPONENTSCOMPONENTS OBJ

OBJ REF REF

REAL

REAL--TIMETIME ORBORB CORECORE IOP

IOP _PLUGGABLE_PLUGGABLE

ORB ORB & & XPORTXPORT

PROTOCOLS PROTOCOLS

IOP

PLUGGABLE IOP

PLUGGABLE ORB ORB & & XPORTXPORT

PROTOCOLS PROTOCOLS

REAL REAL--^TIME^TIME

www.cs.wustl.edu/

schmidt/TAO.html

TAO Overview

^!

An open-source, standards-based, real-time,

high-performance CORBA ORB

Runs on POSIX, Win32, &

embedded RT platforms – e.g., VxWorks,

Chorus, LynxOS

Leverages ACE

The ADAPTIVE Communication Environment (ACE)

PROCESSES//

THREADS THREADS

DYNAMIC DYNAMIC LINKING LINKING

MEMORY MEMORY MAPPING MAPPING SELECT

SELECT//

IO COMP IO COMP

SYSTEM SYSTEM V V IPCIPC STREAM

STREAM PIPES PIPES

NAMED NAMED PIPES PIPES APICSS SOCKETSSOCKETS//

TLI TLI

COMMUNICATION COMMUNICATION SUBSYSTEM SUBSYSTEM

VIRTUAL MEMORY VIRTUAL MEMORY SUBSYSTEM SUBSYSTEM GENERAL POSIX AND WIN32 SERVICES PROCESS

PROCESS//THREADTHREAD SUBSYSTEM SUBSYSTEM

FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR SELF-CONTAINED

DISTRIBUTED SERVICE COMPONENTS

NAME NAME SERVER SERVER TOKEN TOKEN SERVER SERVER LOGGING LOGGING SERVER SERVER

GATEWAY GATEWAY SERVER SERVER

SOCK SOCK__SAPSAP//

TLI TLI__SAPSAP

FIFO FIFO SAP SAP LOG LOG MSG MSG SERVICE

SERVICE HANDLER HANDLER

TIME TIME SERVER SERVER

C++

WRAPPER FACADES

SPIPE SPIPE SAP SAP

CORBA CORBA HANDLER HANDLER

SYSV SYSV WRAPPERS WRAPPERS SHARED SHARED MALLOC MALLOC

THE ACE ORB THE ACE ORB ((TAOTAO)) JAWS ADAPTIVE

JAWS ADAPTIVE WEB SERVER WEB SERVER

MIDDLEWARE APPLICATIONS

REACTOR REACTOR//

PROACTOR PROACTOR PROCESS

PROCESS//

THREAD THREAD MANAGERS MANAGERS

STREAMS STREAMS

SERVICE SERVICE CONFIG CONFIG-- URATOR URATOR SYNCH

SYNCH WRAPPERS WRAPPERS

MEM MEM MAP MAP OS ADAPTATION LAYER

www.cs.wustl.edu/

schmidt/ACE.html

ACE Overview

^!

A concurrent OO networking framework

Available in C++ and Java

Ported to POSIX, Win32, and RTOSs Related work

^!

x-Kernel

SysV STREAMS

(4)

ACE and TAO Statistics

Over 30 person-years of effort – ACE

^>

185,000 LOC – TAO

^>

100,000 LOC

– TAO IDL compiler

^>

100,000 LOC – TAO CORBA Object Services

^>

150,000 LOC

Ported to UNIX, Win32, MVS, and RTOS platforms

Large user community

– www.cs.wustl.edu/

schmidt/ACE- users.html

Currently used by dozens of companies

– Bellcore, Boeing, Ericsson, Kodak, Lockheed, Lucent, Motorola, Nokia, Nortel, Raytheon, SAIC, Siemens, etc.

Supported commercially – ACE

^!

www.riverace.com – TAO

^!

www.ociweb.com

Optimization Challenges for QoS-enabled ORBs

ORB COREORB

PLUGGABLE CORE

PLUGGABLE PROTOCOLS

PROTOCOLS PLUGGABLEPLUGGABLE

PROTOCOLS PROTOCOLS

OS KERNEL OS KERNEL OS I

OS I//O SUBSYSTEMO SUBSYSTEM NETWORK INTERFACES NETWORK INTERFACES

OS KERNEL OS KERNEL OS I

OS I//O SUBSYSTEMO SUBSYSTEM NETWORK INTERFACES NETWORK INTERFACES NETWORK

NETWORK ORB ORB INTERFACE INTERFACE

operation() operation()

IDL IDL STUBS

STUBS OBJECTOBJECT

ADAPTER ADAPTER IDL

IDL SKELETON SKELETON in args

in args out args + return value out args + return value

11

22 33

44 55

66 77

1)

1) CLIENT MARSHALING CLIENT MARSHALING

2)

2) CLIENT PROTOCOL CLIENT PROTOCOL

3)

3) NETWORK LATENCY NETWORK LATENCY

4)

4) SERVER PROTOCOL SERVER PROTOCOL

5) THREAD DISPATCHING

6) REQUEST DEMUXING

7) OPERATION DEMUXING

8) SERVANT DEMARSHALING OBJECT OBJECT ((SERVANTSERVANT))

88

CLIENT CLIENT

IOP

IOP IOPIOP

Key Challenges

Alleviate priority inversion and non-determinism

Reduce demultiplexing latency/jitter

Ensure protocol flexibility

Specify QoS requirements

Schedule operations

Eliminate (de)marshaling overhead

Minimize footprint

Problem: Optimizing Complex Software

DIAGNOSTIC STATIONS DIAGNOSTIC STATIONS

ATM MANATM MAN ATMLAN

ATM LAN

MODALITIES

(CT, MR, CR) BLOB STORE^CENTRAL CLUSTER

STOREBLOB DX

STOREBLOB

www.cs.wustl.edu/

schmidt/

JSAC-99.ps.gz

Common Problems

^!

Optimizing complex software is hard

Small “mistakes” can be costly Solution Approach (Iterative)

^!

Pinpoint overhead via white-box metrics – e.g.,

Quantify

and

VMEtro

Apply patterns and framework components

Revalidate via white-box and black-box metrics

Solution: Patterns and Framework Components

ACCEPTOR CONNECTOR

ABSTRACT FACTORY

SERVANT CLIENT

OS KERNEL OS KERNEL

ACTIVE OBJECT THREAD-SPECIFIC

STORAGE SERVICE CONFIGURATOR

STRATEGY

REACTOR REACTOR WRAPPER FACADES WRAPPER FACADES

www.cs.wustl.edu/

schmidt/ORB- patterns.ps.gz

Definitions

Pattern

– A solution to a problem in a context

Framework

– A “semi-complete”

application built with components

Components

– Self-contained, “pluggable”

ADTs

(5)

EmbeddedMiddlewareDo

ORB O ptimization Principle P a tterns

Definition Optimizationprinciplepatterns

document ru les for a v oiding common design and implementation p rob lems that can deg rade the perf o rm ance , scalability , and predictability o f comple x systems

KeyPrinciplePatternsUsedinTAO #PrinciplePattern

1 Opt imiz e fo r the common c ase 2 Remo v e g ratuitous w a ste 3 Replace inefficient gener al-pur pose functions with efficient special-pur pose ones 4 Shift c omputation in time ,

e.g.,

p recompute 5 Store redundant state to s peed-up e x pensiv e oper ations 6 P a ss hints betw een la y e rs and components 7 Don’t b e tied to ref erence implementations/models 8 Use e fficient/predictab le data s tr uctures

ORB Latency and Priority Inversion Experiments

000 111 C₁

C₀ Requests

C_n

0 10011001101 00

1100110011001101

Client

...

2 ATM Switch

Ultra 2 Ultra 2

I/O SUBSYSTEM

Server

Object Adapter

ORB Core

Servants ^SCHEDULER ^RUNTIME

www.cs.wustl.edu/

schmidt/RT-perf.ps.gz

Method

Vary ORBs, hold OS constant

Solaris real-time threads

High priority client^C0 connects to servant^S0

with matching priorities

Clients^C1 :::C

nhave same lower priority

Clients^C1 :::C

n

connect to servant^S1

Clients invoke twoway CORBA calls that cube a number on the servant and returns result

ORB Latency and Priority Inversion Results

0 4 8 12 16 20 24 28 32 36 40 44 48 52

1 5 10 15 20 25 30 35 40 45 50

Number of Low Priority Clients

Latency per Two-way Request in Milliseconds

CORBAplus High Priority CORBAplus Low Priority MT-ORBIX High Priority MT-ORBIX Low Priority miniCOOL High Priority miniCOOL Low Priority TAO High Priority TAO Low Priority

Synopsis of Results

TAO’s latency is lowest for large # of clients

TAO avoids priority inversion – i.e., high priority client

always has lowest latency

Primary overhead stems from concurrency and connection architecture – e.g., synchronization and

context switching

ORB Jitter Results

1 5 10 15 20 25 30 35 40 45 50

TAO High Priority TAO Low Priority

miniCOOL High Priority miniCOOL Low Priority

MT-ORBIX High Priority MT-ORBIX Low Priority

CORBAplus High Priority CORBAplus Low Priority

0 50 100 150 200 250 300

Jitter in milliseconds

Number of Low Priority Clients

Definition

Jitter

^!

standard deviation from average latency Synopsis of Results

TAO’s jitter is lowest and most consistent

CORBAplus’ jitter is highest and most variable

(6)

Problem: Improper ORB Concurrency Models

SERVANT SERVANT DEMUXERDEMUXER

SERVANT SERVANT

SKELETONS SKELETONS^U

ORB ORB CORECORE

FILTER FILTER FILTER FILTER FILTER FILTER

SERVANT SERVANT

SKELETONS SKELETONS^U

2: enqueue(data) 2: enqueue(data) 3: dequeue,

3: dequeue, filter filter request, request,

&enqueue

4: dispatch 4: dispatch upcall() upcall()

II//OO SUBSYSTEMSUBSYSTEM

THREAD THREAD FILTER FILTER

1: recv() 1: recv()

CONNECTION THREADS CONNECTION THREADS

Common Problems

High context switching and synchronization overhead

Thread-level and packet-level priority inversions

Lack of application control over concurrency model www.cs.wustl.edu/

schmidt/

CACM-arch.ps.gz

Problem: ORB Shared Connection Models

SERVER SERVER ORB ORB CORECORE II//OO SUBSYSTEMSUBSYSTEM

II//OO SUBSYSTEMSUBSYSTEM

COMMUNICATION COMMUNICATION LINKLINK

II//OO SUBSYSTEMSUBSYSTEM SERVANTS SERVANTS

ONE ONE TCPTCP CONNECTION CONNECTION

ORB ORB CORECORE

SEMAPHORES SEMAPHORES 2: select()

2: select() 4: release() 3: read()

BORROWED THREAD BORROWED THREADS

APPLICATION

5: dispatch_return() 1: invoke_twoway()

www.cs.wustl.edu/

schmidt/

RTAS-98.ps.gz

Common Problems

Request-level priority inversions – Sharing multiple

priorities on a single connection

Complex connection multiplexing

Synchronization overhead

Problem: High Locking Overhead

0

94

199

175

0

231

216

599

0 100 200 300 400 500 600 700

TAO miniCOOL CORBAplus MT ORBIX

ORBs Tested

User Level Lock Operations per Request

client server

Common Problems

Locking overhead affects latency and jitter significantly

Memory management commonly involves locking www.cs.wustl.edu/

schmidt/

RTAS-98.ps.gz

Solution: TAO’s ORB Endsystem Architecture

R R U U N N T T II M M E E S S C C H H E E D D U U L L E E R R

HIGH-SPEED NETWORK INTERFACES (e.g., APIC, VME)

Z Z E E R R O O C C O O PP Y Y B B U U FF FF E E R R

RT SS

RT I/O I/O SUBSYSTEM SUBSYSTEM

RT

RT OBJECTOBJECT ADAPTER ADAPTER

O

O(1) (1) REQUESTREQUEST DEMUXERDEMUXER CLIENTS

CLIENTS

STUBS STUBS

SERVANTS SERVANTS

SKELETONS SKELETONS

U

RT

RT ORBORB CORECORE

REACTOR REACTOR ((PP11))

SOCKET

SOCKET QUEUEQUEUE DEMUXERDEMUXER PLUGGABLE PLUGGABLE PROTOCOLSPROTOCOLS

Solution Approach

^!

Integrate scheduler into ORB endsystem

Support multiple scheduling strategies

Co-schedule threads Principle Patterns

^!

Pass hints, precompute, optimize common case, remove gratuitous waste, store state, don’t be tied to reference implementations &

models

(7)

Problem: Reducing Demultiplexing Latency

OPERATION1 OPERATION2 OPERATIONK

2: DEMUX TO I/O HANDLE

...

POA1

...

OS I/O SUBSYSTEM NETWORK ADAPTERS SERVANT 1

5: DEMUX TO SKELETON 6:^DISPATCH OPERATION

1: DEMUX THRU PROTOCOL STACK 4: DEMUX TO SERVANT

ORB CORE ORB CORE ROOT POA 3:^{DEMUX TO}

OBJECT ADAPTER

POA2 POA_N SERVANT N

...

SKEL 1 ^SKEL2 ^{SKEL N}

SERVANT 2

KERNELOS LAYER

ORB LAYER SERVANT

LAYER

Design Challenges

Minimize demuxing layers

Provide

^O(1)

operation demuxing through all layers

Avoid priority inversions

Remain CORBA-compliant www.cs.wustl.edu/

schmidt/

POA.ps.gz

Solution: TAO’s Request Demultiplexing Optimizations

...

... ...

...

IDL IDL SKEL SKEL 22

OBJECT ADAPTER

SERVANT SERVANT 500500 (A)

(A) LAYERED DEMUXING LAYERED DEMUXING,, LINEAR SEARCH LINEAR SEARCH

SERVANT

SERVANT 11 ^SERVANT^SERVANT22

OPERATIONOPERATION100100

...

IDL IDL SKEL N

SKEL N

... ... ...

SERVANTSERVANT500::500::OPERATIONOPERATION100100

OBJECT ADAPTER

...

OBJECT ADAPTER (C) LAYERED DEMUXING,

PERFECT HASHING

SERVANT

SERVANT 11 ^SERVANT^SERVANT22

... ...

...

IDL IDL SKEL N SKEL N

search(object key)

hash(operation)

(D) DE-LAYERED ACTIVE DEMUXING

hash(object key) search(operation)

index(object key/operation) (B) LAYERED DEMUXING,

DYNAMIC HASHING

SERVANT SERVANT 500500

Demuxing

www.cs.wustl.edu/

schmidt/

f

ieee tc-97,COOTS-99

^g

.ps.gz

Perfect hashing

www.cs.wustl.edu/

schmidt/

gperf.ps.gz

Ser v a nt Dem ultiple xing Results

1100200300 400500

050

100150

200 Latency (us)

No. of Objects

Active Demux Dynamic Demux Linear Demux SynopsisofResults

Linear dem ux is costly

Activ e dem ux is most efficient & predictab le

Principles

Precompute , pass hints , use special-pur pose & predictab le data str u ctures

Operation Demultiplexing Results

1 10

20 30

40 50

0 5 10 15 20 25

Latency (us)

No. of Methods Perfect Hashing Binary Search Dynamic Hashing Linear Search

Synopsis of Results

^!

Perfect Hashing – Highly predictable – Low-latency

Others strategies slower

Principle Patterns

^!

Precompute, use predictable data structures, remove gratuitous waste

(8)

T A O R equest Dem ultiple xing Summar y ... ...

POAPOA11

SERVANT SERVANT 11 ORB COREORB COREROOT POA ACTIVE DEMUXING

POA2POAN SERVANT N

...

SKEL 1SKEL 2SKEL N SERVANT 2 ACTIVE DEMUXINGPERFECT HASHING

DemultiplexingStageAbsoluteTime(s)

1. Request parsing 2 2. PO A dem ux 2 3. Ser v ant dem ux 3 4. Oper ation dem ux 2 5. P a ra meter demarshaling oper ation dependent 6. User upcall ser v ant dependent 7. Results m arshaling oper ation dependent

Real-time ORB/OS Performance Experiments

[P ]₀ [P ]₁

SCHEDULER

0 RUNTIME

[P ]

I/O SUBSYSTEM

Server

0 ₁ Sn

Cn

00 1100110011001101

Pentium II

S S

C0 C1

...

...

Object Adapter Servants

ORB Core

Client

...

[P]

Requests Priority

...

[P ] [P ]₁

[P ] n

n

www.cs.wustl.edu/

schmidt/RT-OS.ps.gz

Method

Vary OS, hold ORBs constant

Single-processor Intel Pentium II 450 Mhz, 256 Mbytes of RAM

Client and servant run on the same machine

Client^Ciconnects to servant^Siwith priority^Pi

– ⁱranges from¹^::^:⁵⁰

Clients invoke twoway CORBA calls that cube a number on the servant and returns result

Real-time ORB/OS Performance Results

'

&

$

%

0 500 1000 1500 2000 2500 3000 3500

0 1 2 5 10 15 20 25 30 35 40 45 50 Low Priority Clients

Two-way Request Latency, usec

Linux LynxOS NT Solaris VxWorks

'

&

$

%

0 2000 4000 6000 8000 10000 12000

0 1 2 5 10 15 20 25 30 35 40 45 50 Low Priority Clients

Two-way Request Latency, usec

Linux LynxOS NT Solaris VxWorks

High-priority Client Latency Low-priority Clients Latency

Real-time ORB/OS Jitter Results

'

&

$

%

0 2 10 20 30 40 50

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Two-way Jitter, usec

Low Priority Clients LynxOS Linux Solaris NT VxWorks

'

&

$

%

0 2 10 20 30 40 50

0 2000 4000 6000 8000 10000 12000

Two-way Jitter, usec

Low Priority Clients

LynxOS Linux VxWorks Solaris NT

High-priority Client Jitter Low-priority Clients Jitter

(9)

EmbeddedMiddlewareDouglasC.Schmidt

Pr ob lem: Har d -coded O RB Messa ging and T ranspor t P ro tocols

BOARD BOARD 11

VMEVME 15531553

BOARD BOARD 22

GIOP/IIOP a re not sufficient, e. g .:

– GIOP message footpr int m a y be too large – TCP lac ks necessar y QoS – Legacy c ommitments to e xisting protocols

Existing O RBs don’t s uppor t “pluggab le protocols”

– This mak e s O RBs infle xib le and inefficient

WashingtonUniversity,St.Louis32

Embedded Middleware Do

One Solution: Hacking GIOP

GIOP requests include fields that aren’t needed

in homogeneous embedded applications

– e.g., GIOP magic #, GIOP version, byte order,

request principal, etc.

TAO’s

gioplite

option save 15 bytes

per-request, yielding these calls-per-second:

Marshaling-enabled Marshaling-disabled

min max avg min max avg

GIOP 2,878 2,937 2,906 2,912 2,976 2,949

GIOPlite 2,883 2,978 2,943 2,911 3,003 2,967

The result is a measureable, but small (2%),

improvement in throughput/latency

Our pluggable protocols framework will all much

greater decreases in IOP request sizes, as well as more flexible support for multiple transport protocols

However, there will be no changes required to

the standard CORBA programming model

Washington University, St. Louis

Embedded Middleware Do

Better Solution: TAO’s Pluggable Protocols Framework

CLIENT

STUBS SKELETONS

TCP

MULTICAST IOP

VME UDP

ORB MESSAGING COMPONENT

ORB TRANSPORT ADAPTER COMPONENT

ESIOP

REAL-TIME IOP

EMBEDDED IOP

RELIABLE, BYTE-STREAM UDP ATM

OTHER

ORB

CORE SERVICES

COMMUNICATION INFRASTRUCTURE HIGH SPEED NETWORK INTERFACE REAL-TIME I/O SUBSYSTEM

ORB MESSAGE

FACTORY

ORB TRANSPORT ADAPTER FACTORY

OBJECT ADAPTER

GIOP GIOPLITE

ADAPTIVE Communication Environment (ACE)

OBJECT (SERVANT)

operation (args)

IN ARGS

OUT ARGS & RETURN VALUE

RELIABLE, BYTE-STREAM

POLICY CONTROL MEMORY MANAGEMENT CONCURRENCY

MODEL

Features

Pluggable

ORB

messaging and transport protocols

Highly efficient and

predictable behavior

Principle Patterns

Replace

general-purpose functions

(protocols) with special-purpose ones

Washington University, St. Louis

EmbeddedMiddlewareDouglasC.Schmidt

CORB A P ro tocol Inter operability A rc hitecture

ORB MESSAGING

COMPONENT

ORB TRANSPORT

ADAPTER COMPONENT

TRANSPORT LAYER

NETWORK LAYER GIOPIIOPTCP

IP VMEDRIVER ATM

(

AAL

5)

ATM GIOPLITE

VME

-

IOP ESIOP

ATM

-

IOP

RELIABLE

SEQUENCED

PROTOCOL CONFIGURATIONS STANDARD CORBA PROGRAMMING API

Features

!

Presentation la y er – e. g ., CDR

Message fo rm ats – e. g ., G IOP

T ranspor t assumptions – e. g .,T C P

Object addressing – e. g ., IIOP IOR

www .cs .wustl.edu/

schmidt/ p luggab le protocols .ps .gz

WashingtonUniversity,St.Louis35