Acceptor-Connector an Object Creational Pattern for Connecting

(1)

Acceptor-Connector

An Object Creational Pattern for Connecting

and Initializing Communication Services

Douglas C. Schmidt

[email protected]

Department of Computer Science Department of Computer Science

Washington University Washington University St.

St. LouisLouis, MO , MO 63130, USA63130, USA An earlier version of this paper appeared in a chapter in

An earlier version of this paper appeared in a chapter in the book

the book Pattern Languages of Program Design 3Pattern Languages of Program Design 3, edited, edited by Robert Martin, Frank Buschmann, and Dirke Riehle by Robert Martin, Frank Buschmann, and Dirke Riehle pub-lished by

lished by Addison-WAddison-Wesleyesley, 1997., 1997.

1 1 IIn

ntteen

ntt

The Acceptor-Connector design pattern decouples The Acceptor-Connector design pattern decouples connec-tion establishment and service initializaconnec-tion in a distributed tion establishment and service initialization in a distributed system from the processing performed once a service is system from the processing performed once a service is ini-tia

tializlized. ed. ThiThis s decdecoupouplinling g is is achachieieved with three compoved with three compo--nents:

nents: acceptorsacceptors,, connectorsconnectors, and, and service handlersservice handlers. A con-. A con-nector

nector activelyactively establishes a connection with a remote ac-establishes a connection with a remote ac-ceptor component and initializes a service handler to ceptor component and initializes a service handler to pro-cess data ex

cess data exchangechanged on the connectd on the connection. ion. LikeLikewisewise, , an ac-an ac-ceptor

ceptor passivelypassivelywaits for connection requests from remotewaits for connection requests from remote connectors, establishing a connection upon arrival of such a connectors, establishing a connection upon arrival of such a request, and initializing a service handler to process data request, and initializing a service handler to process data ex-changed on the connection. The initialized service handlers changed on the connection. The initialized service handlers then perform application-speciﬁc processing and then perform application-speciﬁc processing and communi-cate via the connection established by the connector and cate via the connection established by the connector and ac-ceptor components.

ceptor components.

2 2 E

Ex

xa

am

mp

pllee

T

To o illusillustrate the trate the AccepAcceptor-tor-ConneConnector ctor pattepattern, rn, consiconsider der thethe multi-service,

multi-service, application-levapplication-levelel GatewayGateway shoshown wn in in Fig Fig--ure 1.

ure 1. In geneIn generalral, a, a GatewayGatewaydecouples cooperating com-decouples cooperating com-ponents in a distributed system and allows them to interact ponents in a distributed system and allows them to interact without having direct dependencies on each other. The without having direct dependencies on each other. The par-ticular

ticular GatewayGateway in Figure 1 routes data between differentin Figure 1 routes data between different service endpoints running on remote

service endpoints running on remote PeersPeers used to mon-used to mon-itor and contro

itor and control a l a satesatellitllite constelle constellationation. . Each serviEach service ince in the

thePeersPeers sends and receives several types of data via thesends and receives several types of data via the

Gateway

Gateway, such as status information, bulk data, and com-, such as status information, bulk data, and com-mands. In general,

mands. In general, PeersPeerscan be distributed throughout lo-can be distributed throughout lo-cal area networks (LANs) and wide-area networks (WANs). cal area networks (LANs) and wide-area networks (WANs).

The

TheGatewayGateway is a router that coordinates the communi-is a router that coordinates the communi-cation among its

cation among its PeersPeersFrom the˙˙From the GatewayGateway’s ’s perspectivperspective,e, the

thePeerPeerservices whose data it routes differ solely in termsservices whose data it routes differ solely in terms of their application-level communication protocols, which of their application-level communication protocols, which may use different framing formats and payload types. may use different framing formats and payload types.

WIDE AREA WIDE AREA NETWORK NETWORK SATELLITES SATELLITES SATELLITES SATELLITES TRACKING TRACKING TRACKING TRACKING STATION STATION STATION STATION PEERS PEERS PEERS PEERS STATUS INFO STATUS INFO COMMANDS

COMMANDS BULK DATABULK DATA TRANSFER TRANSFER

LOCAL AREA NETWORK LOCAL AREA NETWORK

GROUND GROUND STATION STATION PEERS PEERS GATEWAY GATEWAY Fi

Figugure re 1: 1: ThThe e PhPhysysicical al ArArchchititecectuture re of of a a CoConnnnecectition on--oriented

oriented Application-levApplication-level Gatewayel Gateway

The

The GatewayGateway transmits data between itstransmits data between its PeersPeers usingusing the connecti

the connection-oron-orienteiented TCP/IP protocol [1]. d TCP/IP protocol [1]. In our In our exam- exam-ple network conﬁguration, each service is bound to a ple network conﬁguration, each service is bound to a con-necti

nectionon endpoiendpointnt desigdesignatednated by an IP hby an IP host aost addresddresss and a TCand a TCPP port numbe

port numberr. . The port numberThe port number uniqueuniquelyly identidentiﬁes the type of iﬁes the type of servi

service. ce. MaintMaintaininaining g separseparate connectiate connections for each ons for each type of type of service/port increases the ﬂexibility of routing strategies and service/port increases the ﬂexibility of routing strategies and provides more robust error handling if network connections provides more robust error handling if network connections shutdown

shutdown unexpectedlyunexpectedly.. In

In ourour distdistribributeduted applapplicatiication,on,GatewayGatewayandandPeersPeersmustmust be able to change their connection roles to support different be able to change their connection roles to support different use-cases. In particular, either may initiate a connection use-cases. In particular, either may initiate a connection ac-tiv

tively or ely or may wait passimay wait passively for connectvely for connection requesion requests. ts. ForFor example, in one conﬁguration, the

example, in one conﬁguration, the GatewayGateway may activelymay actively initiate connections to remote

initiate connections to remote PeersPeersin order to route datain order to route data to them.

to them. In another conﬁIn another conﬁguratiguration, theon, the GatewayGateway may pas-may pas-sively receive connection requests from

sively receive connection requests from PeersPeers which thenwhich then route data through the

route data through the GatewayGateway to anotherto another PeerPeer. . LiLike ke--wise,

wise,PeersPeersmay be active connection initiators in one use-may be active connection initiators in one use-cas

casee andand thethenn bebe paspassisiveve conconnecnectiotionn accaccepteptorsors inin anoanothetherr use use--case.

(2)

Due

Due to to the nature of the nature of our our disdistritributbuted ed appappliclicatiation, on, con con--venti

ventional onal desigdesigns ns that that desigdesignate nate connecconnection tion estabestablishlishmentment and service initialization roles a priori and hard-code them and service initialization roles a priori and hard-code them into the

into the GatewayGateway andand PeerPeer components are too inﬂexi-components are too inﬂexi-ble.

ble. Such a Such a desidesign overly couples the gn overly couples the connecconnection estabtion establish lish--ment, service initialization, and service processing ment, service initialization, and service processing compo-nents. This tight coupling make it hard to change connection nents. This tight coupling make it hard to change connection roles independently of the communication roles.

roles independently of the communication roles.

3 3 C

Co

on

ntteex

xtt

A client/server application in a distributed system that A client/server application in a distributed system that uti-lizes

lizes connecconnectiontion-orie-orientednted protoprotocolscols to to commucommunicatnicatee betwebetweenen service endpoints.

service endpoints.

4 4 P

Prro

ob

blleem

m

Dist

Distribuributedted appliapplicatiocationsns oftenoften contacontainin complcomplexex codecode thatthat per- per-forms connection establishment and service initialization. In forms connection establishment and service initialization. In general, the processing of data exchanged between service general, the processing of data exchanged between service endpoints in a distributed application is largely independent endpoints in a distributed application is largely independent of conﬁguration issues such as (1) which endpoint initiated of conﬁguration issues such as (1) which endpoint initiated the connection,

the connection, i.e.i.e., the, the connection roleconnection role vs. thevs. the communca- communca-tion

tion rroleole andand (2)(2) thethe conconnecnectiotionn manmanageagemenmentt proprotoctocolol vs. vs. thethe network programming

network programming API. These issues are outlined below:API. These issues are outlined below: Connection role vs.

Connection role vs. communication rolcommunication role:e: ConnectionConnection estab

establishlishmentment rolesroles areare inherinherentlyently asymmetricalasymmetrical,, i.e.i.e.,, ththee papas- s-sive service endpoint

sive service endpoint waitswaits and the active service endpointand the active service endpoint initiates

initiatesthe connection. Once the connection is established,the connection. Once the connection is established, however, the communication role can be

however, the communication role can be orthogonalorthogonal to theto the connection role.

connection role. Thus, data can be transferred between ser-Thus, data can be transferred between ser-vice endpoints in any manner that obeys the serser-vice’s vice endpoints in any manner that obeys the service’s com-munication pr

munication protocol. Common communicationotocol. Common communication protocols in-protocols in-clude peer-to-peer

clude peer-to-peer,, request-response, and oneway streaming.request-response, and oneway streaming. The connection management p

The connection management protocol vs. rotocol vs. the networkthe network programming API:

programming API: Different network programming in-Different network programming in-terfaces, such as sockets or TLI, provide different APIs to terfaces, such as sockets or TLI, provide different APIs to establish connections using various connection management establish connections using various connection management protocols. Regardless

protocols. Regardless of the of the protocolprotocol used to used to establish establish a con-a con-nection, however, data can be transferred between endpoints nection, however, data can be transferred between endpoints using

using unifouniformrm messmessageage passpassinging operatoperations,ions, e.g.e.g.,,sendsend / / recvrecv

calls. calls.

In general, the strategies for connection establishment and In general, the strategies for connection establishment and service initialization change much less frequently than service initialization change much less frequently than ap-plication service implementations and communication plication service implementations and communication pro-tocols. Thus, decoupling these aspects so that they can vary tocols. Thus, decoupling these aspects so that they can vary independently is essential for developing and maintaining independently is essential for developing and maintaining distributed applications. The following

distributed applications. The following forcesforcesimpact the so-impact the so-lution to the problem of separating the connection and lution to the problem of separating the connection and ini-tialization protocols from the communication protocol: tialization protocols from the communication protocol:

It should be easy to add new types of services, new It should be easy to add new types of services, new ser-vice implementations, and new communication vice implementations, and new communication proto-cols without affecti

cols without affectingng the the exisexistingting connecconnectiontion estaestablisblish- h-ment and service initialization software. For

ment and service initialization software. For instance, itinstance, it

may be necessary to extend the

may be necessary to extend the GatewayGatewayto interoper-to interoper-ate with a directory service that runs over the IPX/SPX ate with a directory service that runs over the IPX/SPX communication protocol, rather than TCP/IP..

communication protocol, rather than TCP/IP.. It should be possible to decouple (1) the

It should be possible to decouple (1) the connectionconnection roles

roles,, i.e.i.e., , which procwhich process initiaess initiates a tes a connecconnection vs. tion vs. ac- ac-cepts

cepts the the connecconnection,tion, fromfrom (2) (2) thethe communicationcommunication rolesroles,, i.e.

i.e., which service endpoint, which service endpoint is the client or the servis the client or the server. Iner. In general, the distinction between “client” and “server” general, the distinction between “client” and “server” refer to communication roles, which may be refer to communication roles, which may be orthogo-nal to connection roles. For instance, clients often play nal to connection roles. For instance, clients often play the active role when initiating connections with a the active role when initiating connections with a pas-sive server. However, these connection roles can be sive server. However, these connection roles can be re-versed. For example, a client that plays an active versed. For example, a client that plays an active com-munication role may wait passively for another process munication role may wait passively for another process to connect to it.

to connect to it. The exampThe example in Section 2 illustrle in Section 2 illustratesates this latter use-case.

this latter use-case.

It should be possible to write communication software It should be possible to write communication software that is portable to many OS platforms in order to that is portable to many OS platforms in order to max-imiz

imize e avavailabailabilitility y and market share. and market share. Many low-Many low-levlevelel netw

network ork prograprogramminmming g APIs have APIs have semasemantics that ntics that areare only superﬁcially dif

only superﬁcially different. ferent. Therefore, it hard to writeTherefore, it hard to write porta

portableble appliapplicaticationon usingusing low-low-levlevelel APIs, APIs, suchsuch asas sock- sock-ets and TLI, due to syntactic incompatibilities.

ets and TLI, due to syntactic incompatibilities.

It should be possible to shield programmers from the It should be possible to shield programmers from the lack of typesafety in low-level network programming lack of typesafety in low-level network programming APIs like sockets or

APIs like sockets or TLI. For TLI. For exampexample, connectile, connection es-on es-tablishment code should be completely decoupled from tablishment code should be completely decoupled from subsequent data transport code to ensure that endpoints subsequent data transport code to ensure that endpoints are used correctly.

are used correctly. Without tWithout this strong decoupling, forhis strong decoupling, for instance, services may mistakenly read or write data on instance, services may mistakenly read or write data on passive-mode transport endpoint factories that should passive-mode transport endpoint factories that should only be used to accept connections.

only be used to accept connections.

It should be possible to reduce connection latency by It should be possible to reduce connection latency by using OS

using OS featufeatures like res like asyncasynchronouhronous connection es-s connection es-tabl

tablishmeishment. nt. For instFor instance, applicance, applicationations with a larges with a large number of peers may need to asynchronously establish number of peers may need to asynchronously establish many connections concurrently

many connections concurrently. . Efﬁcient and scalableEfﬁcient and scalable connection establishment is particularly important for connection establishment is particularly important for applications that

applications that run over run over long-latency Wlong-latency WANs.ANs.

It should be possible to reuse as much general-purpose It should be possible to reuse as much general-purpose connec

connectiontion estaestablishblishmentment andand servservice ice initiinitializalizationation soft- soft-ware as possible in order to leverage prior development ware as possible in order to leverage prior development effort.

effort.

5 5 So

Sollu

uti

tion

on

For each

For each serviceserviceoffered by a distributed application, use theoffered by a distributed application, use the Acceptor-Connector

Acceptor-Connector pattern to decouple connection estab-pattern to decouple connection estab-lishment and service initialization from subsequent lishment and service initialization from subsequent process-ing performed by the two endpoints of a service once they ing performed by the two endpoints of a service once they are connected and initialized.

are connected and initialized.

Introduce two factories that produce connected and Introduce two factories that produce connected and ini-tialized service handlers, which implement the application tialized service handlers, which implement the application servi

services. ces. The ﬁrst The ﬁrst factfactoryory, , callcalled acceptored acceptor, , creatcreates and es and ini-

ini-               

(3)

Connector

connect(sh, addr) connect(sh, addr) complete() complete() ACTIVATES ACTIVATES ACTIVATES ACTIVATES NOTIFY WHEN NOTIFY WHEN CONNECTION COMPLETES CONNECTION COMPLETES

Service

Handler

peer_stream_ peer_stream_ open() open() Service Handler Service Handler Service Handler Service Handler

Dispatcher

1 1 1 1

n

Acceptor

peer_acceptor_ peer_acceptor_ accept() accept() open() open() Service Handler Service Handler Service Handler Service Handler

n

1 1 1 1 NOTIFY WHEN NOTIFY WHEN CONNECTION ARRIVES CONNECTION ARRIVES ACTIVATES ACTIVATES

Figure 2: Structure of Participants in the Acceptor

Figure 2: Structure of Participants in the Acceptor-Connector-Connector PatternPattern

tializes a transport endpoint that passively listens at a tializes a transport endpoint that passively listens at a par-ticular address for connection requests from remote ticular address for connection requests from remote connec-tors.

tors. TheThe seconsecondd factfactoryory,, a a connecconnectortor,, actiactivelyvely initiinitiatesates aa con- con-nection to a remote acceptor. Acceptor and connectors both nection to a remote acceptor. Acceptor and connectors both initialize the corresponding

initialize the corresponding service handlers that processervice handlers that process thes the data exc

data exchangedhanged onon the connethe connectioction. n. OnceOnce the servthe service handlice handlersers are connected and initialized they perform the are connected and initialized they perform the application-speciﬁc processing, and generally do not interact with the speciﬁc processing, and generally do not interact with the acceptor and connector any further.

acceptor and connector any further.

6 6 St

Stru

ruct

ctur

uree

The structure of the participants in the Acceptor-Connector The structure of the participants in the Acceptor-Connector pattern is illustrated by the Booch class diagram [2] in pattern is illustrated by the Booch class diagram [2] in Fig-ure 2.

ure 2. Servi

Servicece HandlHandler:er: AAService HandlerService Handlerimplimplemenementsts anan application service, typically playing the client role, server application service, typically playing the client role, server role, or both roles.

role, or both roles. It provideIt provides a s a hook methohook method that is d that is callecalledd byan

byanAcceptorAcceptorororConnectorConnectorto activto activateate thethe appliapplicatiocationn serv

service when ice when the connectiothe connection is n is estabestablishlished. ed. In addition, theIn addition, the

Service Handler

Service Handleroffeoffersrs a data data-moda-modee transtransportport endpoiendpointnt that encapsulates an I/O handle, such as a socket. Once that encapsulates an I/O handle, such as a socket. Once con-nected

nected and initiand initialized, this alized, this endpointendpoint is used is used by theby theServiceService Handler

Handlerto exchange data with its connected peer.to exchange data with its connected peer. Acceptor:

Acceptor: AnAn AcceptorAcceptor is a is a factfactory that ory that implimplementementss the

the ststratrategy egy forfor passivelypassively estaestablisblishing hing a a connecconnection andtion and initializi

initializing ng its associatedits associated Service HandlerService Handler. . In In adaddi di--tion, the

tion, theAcceptorAcceptorcontains a passive-mode transport end-contains a passive-mode transport end-point factory that creates new data-mode endend-points used by point factory that creates new data-mode endpoints used by

Service Handler

Service Handler’s to transmit data between connected’s to transmit data between connected peer

peers. s. TheTheAcceptorAcceptor’s’sopenopenmethod initializes its trans-method initializes its trans-por

portt endendpoipointnt facfactortoryy onconcee byby binbindindingg itit toto aa netnetworworkk addaddresress,s, such as the TCP port number the

such as the TCP port number theAcceptorAcceptoris listening on.is listening on. Once initialized, the passive-mode transport endpoint Once initialized, the passive-mode transport endpoint fac-tory listens for connection requests from peers. When a tory listens for connection requests from peers. When a con-nection request arrives, the

nection request arrives, the AcceptorAcceptorcreates acreates aServiceService

In this

In this diagdiagram dashed ram dashed clouclouds ds indindicaticate e classclasses; es; dashdashed ed boxboxes es in thein the clouds indicate template parameters; a solid undirected edge with a hollow clouds indicate template parameters; a solid undirected edge with a hollow circle at one end indicates a uses relation between two classes.

circle at one end indicates a uses relation between two classes.

Handler

Handlerand uses its transport endpoint factory to accept aand uses its transport endpoint factory to accept a new connection into the

new connection into the Service HandlerService Handler.. Connector:

Connector: AA ConnectorConnector is a factory that implementsis a factory that implements the strategy for

the strategy for activelyactively establishing a connection and ini-establishing a connection and ini-tializing its associated

tializing its associated Service HandlerService Handler. . It proIt providvides aes a method that initiates a connection to a remote

method that initiates a connection to a remote AcceptorAcceptor.. Likewise, it provides another method that ﬁnishes activating Likewise, it provides another method that ﬁnishes activating

Service

Service HandlerHandlerss whose whose connecconnections were tions were initinitiateiatedd either synchronously or asynchronously

either synchronously or asynchronously. . TheThe ConnectorConnector

uses two separate methods

uses two separate methods to support asynchronousto support asynchronous connec- connec-tion

tion establishment transparentlyestablishment transparently.. Bo

Both th ththee AcceptorAcceptor andand ConnectorConnector actactivivate ate aa

Service Handler

Service Handlerby calling its activation hook methodby calling its activation hook method wh

when a en a coconnnnecectition is eston is estabablilishsheded. . OnOnce ace a ServiceService Handler

Handler is complis completeletely y initiinitializalized ed by by anan AcceptorAcceptor oror

Connector

Connectorfactory it typically does not interact with thesefactory it typically does not interact with these components any

components any further.further. Dispatcher:

Dispatcher: For theFor the AcceptorAcceptor, the, the DispatcherDispatcher de- de-multiplexes connection requests received on one or more multiplexes connection requests received on one or more trans

transport port endpoendpoints to ints to the the appropappropriatriatee AcceptorAcceptor. . TThhee

Dispatcher

Dispatcheralloallowsws multmultipleipleAcceptorsAcceptorstoto reregisgisterter witwithh it in order to listen for connections from different peers on it in order to listen for connections from different peers on different ports

different ports simultaneouslysimultaneously.. For the

For the ConnectorConnector, , tthehe DispatcherDispatcher handlhandles es thethe co

compmpleletition on of of coconnennectctioions ns ththat at wewere re ininititiaiateted d asasyn yn--chrono

chronouslyusly. . In this caIn this case, these, the DispatcherDispatcher calls back tocalls back to the

the AcceptorAcceptor when an when an asyncasynchronouhronous connectis connection is on is es- es-tab

tablilisheshed. d. TheThe DispatcherDispatcher allows multipleallows multiple ServiceService Handlers

Handlers to to havhave e theitheir r conneconnectionctions initiates initiated d and com-and com-plete

pleted d asyncasynchronouhronously by sly by aa ConnectorConnector. . NoNote tte thahat tht thee

Dispatcher

Dispatcheris not necessary for synchronous connectionis not necessary for synchronous connection establishment since the thread of control that initiates the establishment since the thread of control that initiates the connection also completes the service handler activation. connection also completes the service handler activation.

A

ADispatcherDispatcheris typically implemented using an eventis typically implemented using an event dem

demultultipliplexiexing ng patpattertern, n, sucsuch h thothose se proprovidvided ed by by the the Re- Re-act

actor or [3] [3] or or ProProactactor or [4][4], , whiwhich ch hanhandle synchrdle synchronoonous andus and async

asynchronohronous demultiplus demultiplexinexing, g, resperespectictivelyvely. . LikeLikewise, wise, thethe

Dispatcher

Dispatcher can be implemented as a separate thread orcan be implemented as a separate thread or process using the Active Object pattern [5].

process using the Active Object pattern [5].

1 1

(4)

7 7 D

Dyn

ynam

amic

icss

The

The folfollolowinwing g secsectiotion n desdescricribes bes the the colcollablaboraoratitions ons per per--formed by the

formed by the AcceptorAcceptor andand ConnectorConnector componentscomponents in the

in the AccepAcceptor-tor-ConneConnector pattctor pattern. ern. WWe e examexamine the threeine the three can

canonionical scecal scenarnariosios: : for thefor the AcceptorAcceptor, , asyncasynchronoushronous

Connector

Connector, and synchronous, and synchronousConnectorConnector..

7.1 7.1 Acce

Acceptor

ptor Comp

Component

onent Collab

Collaborat

orations

ions

Fi

Figugure re 3 3 ililluluststraratetes s ththe e cocollllababororatatioion n bebetwtweeeen n ththee

Acceptor

Acceptor andand Service HandlerService Handlerpartiparticipancipants. ts. TheseThese

Server Server Server Server INSERT IN DISPATCHER INSERT IN DISPATCHER INSERT IN DISPATCHER INSERT IN DISPATCHER

START EVENT LOOP START EVENT LOOP START EVENT LOOP START EVENT LOOP

CONNECTION EVENT CONNECTION EVENT CONNECTION EVENT CONNECTION EVENT REGISTER HANDLER REGISTER HANDLER REGISTER HANDLER REGISTER HANDLER FOR CLIENT FOR CLIENT FOR CLIENT FOR CLIENTI/OI/OI/OI/O FOREACH EVENT DO FOREACH EVENT DO FOREACH EVENT DO FOREACH EVENT DO INITIALIZE PASSIVE INITIALIZE PASSIVE INITIALIZE PASSIVE INITIALIZE PASSIVE ENDPOINT ENDPOINT ENDPOINT ENDPOINT acc : acc : acc : acc : Acceptor Acceptor Acceptor Acceptor handle_event() handle_event() : disp : disp : disp : disp Dispatcher Dispatcher Dispatcher Dispatcher select() select() sh: sh: sh: sh: Service Service Service Service Handler Handler Handler Handler accept() accept() register_handler(sh) register_handler(sh) DATA EVENT DATA EVENT service() service() PROCESS MSG PROCESS MSG open() open() CREATE CREATE,, ACCEPT ACCEPT,, AND ACTIVATE AND ACTIVATE OBJECT OBJECT handle_events() handle_events() register_handler(acc) register_handler(acc) sh = new Service_Handler sh = new Service_Handler peer_acceptor.accept peer_acceptor.accept (sh->peer_stream_) (sh->peer_stream_) sh->open() sh->open() E E N N D D P P O O I I N N T T E E N N D D P P O O I I N N T T I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N P P H H A A S S E E P P H H A A S S E E S S E E R R V V I I C C E E S S E E R R V V I I C C E E I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N P P H H A A S S E E P P H H A A S S E E S S E E R R V V I I C C E E S S E E R R V V I I C C E E P P R R O O C C E E S S S S I I N N G G P P R R O O C C E E S S S S I I N N G G P P H H A A S S E E P P H H A A S S E E

Figure 3: Collaborations Among Acceptor Participants Figure 3: Collaborations Among Acceptor Participants collaborations are divided into three phases:

collaborations are divided into three phases: 1.

1. Endpoint initiEndpoint initialization phasealization phase:: To initialize a connec-To initialize a connec-tion passively, an applicaconnec-tion calls the

tion passively, an application calls the openopenmethod of themethod of the

Acceptor

Acceptor. . ThiThis s metmethod crehod createates s a a paspassisive-ve-modmode transe trans--port endpoint and binds it to a network address,

port endpoint and binds it to a network address, e.g.e.g. thethe loc

local host’al host’s s IP IP namname e and a and a TCP port numbeTCP port number, r, and thenand then liste

listens ns for connectiofor connection n requerequests from sts from peerpeer ConnectorConnectors.s. Nex

Next, t, thethe openopen methomethod d regiregistersters s thethe AcceptorAcceptor objectobject with a

with a DispatcherDispatcher so that the dispatcher can call back so that the dispatcher can call back to the

to the AcceptorAcceptor when connection eventwhen connection events arrive. s arrive. FinallyFinally,, the

the appliapplicatication on initinitiateiates s thethe DispatcherDispatcher’’s s evevent ent looloop,p, whi

which ch waiwaits ts for for conconnecnectiotion n reqrequesuests ts to to arrarrivive e frofrom m peepeerr

Connector Connectors.s. 2.

2. SerServicvice inie inititialializatzation pion phashase:e: WheWhen n a a conconnecnectiotionn req

requesuest t arrarrivives, es, thethe DispatcherDispatcher cacalllls s baback ck to to ththee

Acceptor

Acceptor’s’s acceptaccept memeththodod. . ThThee acceptaccept methodmethod ass

assembembles les the the resresourources necesces necessarsary y to to (1) (1) crecreate ate a a neneww

Service Handler

Service Handler, , (2) (2) use use its its paspassisive-ve-modmode e tratranspnsportort end

endpoipoint nt fafactoctory ry to to accaccept ept the the conconnecnectiotion n intinto o the the datdata- a-mode transport endpoint of this handler, and (3) activate the mode transport endpoint of this handler, and (3) activate the

Service Handler

Service Handlerby calling itsby calling its openopenhook. Thehook. Theopenopen

ho

hook of ok of ththee Service HandlerService Handler can can perfoperform rm servservice- ice-speci

speciﬁc ﬁc initinitialiializatization, on, such such as as alloallocaticating ng lockslocks, , spawspawningning

threa

threads, ds, openinopening g log log ﬁlesﬁles, , and/or regisand/or registeritering ng thethe ServiceService Handler

Handlerwith awith aDispatcherDispatcher.. 3.

3. ServService price procesocessing phsing phase:ase: After the connection hasAfter the connection has been establis

been established hed passipassively and vely and thethe Service HandlerService Handler

has

has been initialibeen initialized, zed, the service processithe service processing ng phase begins.phase begins. In this phase, an application-level communication protocol, In this phase, an application-level communication protocol, such as HTTP or IIOP, is used to exchange data between the such as HTTP or IIOP, is used to exchange data between the local

localService HandlerService Handlerand its connected remoteand its connected remote PeerPeer

via its

via its peer streampeer stream endpoiendpoint. nt. When thiWhen this exchangs exchange ise is complete the connection and

complete the connection and Service HandlerService Handlers can bes can be shut down and resources released.

shut down and resources released.

7.2 7.2 Conne

Connector

ctor Compo

Component

nent Coll

Collaborat

aborations

ions

The

The ConnectorConnector compocomponent can nent can initiinitialize alize itsits ServiceService Handler

Handler using two using two genergeneral al scheschemes:mes: synchronoussynchronous andand asynchronous

asynchronous. . SynchSynchronouronous service initis service initializaalization is tion is usefusefulul for the following situations:

for the following situations:

If the latency for establishing a connection is very low, If the latency for establishing a connection is very low, e.g.

e.g.,, estaestablishblishinging a ca connectonnectionion with with a sa serveerverr onon the the samesame host via the loopback device; or

host via the loopback device; or

If multiple threads of control are available and it is If multiple threads of control are available and it is efﬁ-ci

cienentt toto ususee aa didiffffererenentt ththrereadad toto coconnnnecectt eaeachchServiceService Handler

Handlersynchronously; orsynchronously; or

If the services must be initialized in a ﬁxed order and If the services must be initialized in a ﬁxed order and the client cannot perform useful work until connections the client cannot perform useful work until connections are established.

are established.

Likewise, asynchronous service initialization is useful in Likewise, asynchronous service initialization is useful in opposite situations:

opposite situations: If

If coconnnnecectitionlateonlatencncyy isis hihighgh anandd ththereree araree mamanyny pepeererss toto connect with,

connect with, e.g.e.g., , estaestablisblishing a hing a larglarge e number of con-number of con-nections over a high-latency WAN; or

nections over a high-latency WAN; or If only a single thread of control is available,

If only a single thread of control is available, e.g.e.g., if the, if the OS platform

OS platform does not does not provideprovide application-levapplication-levelel threads;threads; or

or

If the order in which services are initialized is not If the order in which services are initialized is not im-portant and if the client application must perform portant and if the client application must perform addi-tional work, such as refreshing a GUI, while the tional work, such as refreshing a GUI, while the con-nection is being established.

nection is being established. The

The collacollaboratborations among the ions among the partiparticipancipants in ts in thethe syn- syn-chronous

chronousConnectorConnectorscenario can be divided into the fol-scenario can be divided into the fol-lowing three phases:

lowing three phases: 1.

1. ConneConnection iction initinitiation phation phase:ase: TTo o synchronously ini-synchronously ini-tia

tiate te a a conconnecnectiotion n betbetweeween n aa Service HandlerService Handler andand its

its remotremotee PeerPeer, , an an appappliclicatiation on calcalls ls thethe ConnectorConnector’s’s

connect

connect methomethod. d. This metThis method actihod actively esvely establitablishes theshes the connection by blocking the thread of control of the calling connection by blocking the thread of control of the calling thread until the connection completes synchronously.

thread until the connection completes synchronously.

           

(5)

Client Client Client Client FOREACH CONNECTION FOREACH CONNECTION FOREACH CONNECTION FOREACH CONNECTION INITIATE CONNECTION INITIATE CONNECTION INITIATE CONNECTION INITIATE CONNECTION INSERT IN DISPATHER INSERT IN DISPATHER INSERT IN DISPATHER INSERT IN DISPATHER con : con : con : con : Connector Connector Connector Connector handle_event() handle_event() : disp : disp : disp : disp Dispatcher Dispatcher Dispatcher Dispatcher sh: sh: sh: sh: Service Handler Service Handler Service Handler Service Handler register_handler(sh) register_handler(sh) DATA ARRIVES DATA ARRIVES DATA ARRIVES DATA ARRIVES service() service() PROCESS DATA PROCESS DATA PROCESS DATA PROCESS DATA connect(sh, addr) connect(sh, addr) S S E E R R V V I I C C E E S S E E R R V V I I C C E E P P R R O O C C E E S S S S I I N N G G P P R R O O C C E E S S S S I I N N G G P P H H A A S S E E P P H H A A S S E E C C O O N N N N E E C C T T I I O O N N C C O O N N N N E E C C T T I I O O N N I I N N I I T T I I A A T T I I O O N N I I N N I I T T I I A A T T I I O O N N / / S S E E V V I I C C E E I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N P P H H A A S S E E

START EVENT LOOP START EVENT LOOP FOREACH EVENT DO FOREACH EVENT DO handle_events() handle_events() select() select() open() open() ACTIVATE OBJECT ACTIVATE OBJECT

Figure 4: Collaborations Among Connector Participants for Figure 4: Collaborations Among Connector Participants for Synchr

Synchronousonous ConneConnectionctionss

2.

2. SerServicvice initie initialializatzation phion phasease:: After the After the conneconnectionction completes, the

completes, the ConnectorConnector’s’sconnectconnect method calls themethod calls the

complete

complete method to activates themethod to activates the Service HandlerService Handler.. The

The completecomplete method performs the activation by invok-method performs the activation by invok-ing the

ing the Service HandlerService Handler’s’sopenopenhook method, whichhook method, which performs service-speciﬁc

performs service-speciﬁc initializatiinitialization.on. 3. Service processing phase:

3. Service processing phase: This phase is similar to theThis phase is similar to the service processing phase a

service processing phase a Service HandlerService Handlerperformsperforms once it is created by an

once it is created by an AcceptorAcceptor. . In partIn particuicularlar, once a, once a

Service Handler

Service Handler is activated it performs application-is activated it performs application-speciﬁc service processing using data exchanged with the speciﬁc service processing using data exchanged with the remote

remoteService HandlerService Handlerit is connected to.it is connected to. The

The collcollaboraaboration tion for for synchsynchronouronous s servservice ice initiinitializaalizationtion is sho

is shown in Figwn in Figurure 4. e 4. In thiIn this s scschehememe, , ththee ConnectorConnector

combines the connection initiation and service initialization combines the connection initiation and service initialization phases in

phases into a to a singlsingle blocking opere blocking operationation. . In this scenariIn this scenario,o, only one connection is established for every invocation of only one connection is established for every invocation of

connect

connectin each thread of control.in each thread of control.

The collaborations among the participants in the

The collaborations among the participants in the asyn- asyn-chronous

chronous ConnectorConnector can be can be dividivided into ded into the followithe followingng three phases:

three phases: 1.

1. ConneConnection iniction initiatitiation phase:on phase: To asynchronously ini-To asynchronously ini-ti

tiate ate a a conconnecnectiotion n betbetweeween n aa Service HandlerService Handler andand its

its remotremotee PeerPeer, , an an appappliclicatiation on calcalls ls thethe ConnectorConnector’s’s

connect

connect methomethod. d. As with the synchronouAs with the synchronous scenars scenario, theio, the

Connector

Connector actiactively esvely establitablishes the coshes the connectnnection. ion. How- How-ever, it does not block the thread of control of the caller ever, it does not block the thread of control of the caller while the

while the connecconnection completion completes asynchrotes asynchronouslnouslyy. . InsteInstead,ad, it registers the

it registers the Service HandlerService Handler’s transport endpoint,’s transport endpoint, which we call

which we call peer streampeer stream in in thithis s exaexamplmple, e, witwith h thethe

Dispatcher

Dispatcherand returns control to its caller.and returns control to its caller. 2.

2. ServiServicece initinitializializationation phasephase:: AftAfterer thethe conconnecnectiotionn com com--plete

pletes s asyncasynchronouhronouslysly, , thethe DispatcherDispatcher calcalls ls bacback k thethe

Connector

Connector’s’scompletecompletemethod. This method actimethod. This method activatesvates the

the Service HandlerService Handler by calling itsby calling its openopen hoohook. k. ThiThiss

open

openhook performs hook performs service-speciﬁc initialization.service-speciﬁc initialization.

3.

3. Service prService processing phase:ocessing phase: This phase is similar to theThis phase is similar to the other service processing phases describe

other service processing phases described earlierd earlier. . Once theOnce the

Service Handler

Service Handler is activated it performs application-is activated it performs application-speci

speciﬁcﬁc serviservicece proceprocessingssing usingusing thethe data data exchaexchangednged with with thethe remote

remoteService HandlerService Handlerit is connected to.it is connected to.

Figure 5 illustrates these three phases of collaboration Figure 5 illustrates these three phases of collaboration us-ing asynchr

ing asynchronous connonous connectiection establon establishmeishment. nt. In the asyn-In the

asyn-Client Client Client Client FOREACH CONNECTION FOREACH CONNECTION FOREACH CONNECTION FOREACH CONNECTION INITIATE CONNECTION INITIATE CONNECTION INITIATE CONNECTION INITIATE CONNECTION INSERT IN DISPATCHER INSERT IN DISPATCHER INSERT IN DISPATCHER INSERT IN DISPATCHER

START EVENT LOOP START EVENT LOOP START EVENT LOOP START EVENT LOOP FOREACH EVENT DO FOREACH EVENT DO FOREACH EVENT DO FOREACH EVENT DO handle_events() handle_events() select() select() CONNECTION COMPLETE CONNECTION COMPLETE CONNECTION COMPLETE CONNECTION COMPLETE INSERT IN REACTOR INSERT IN REACTOR INSERT IN REACTOR INSERT IN REACTOR con : con : con : con : Connector Connector Connector Connector handle_event() handle_event() : disp : disp : disp : disp Dispatcher Dispatcher Dispatcher Dispatcher sh: sh: sh: sh: Service_Handler Service_Handler Service_Handler Service_Handler register_handler(sh) register_handler(sh) DATA ARRIVES DATA ARRIVES DATA ARRIVES DATA ARRIVES service() service() PROCESS DATA PROCESS DATAPROCESS DATA PROCESS DATA connect(sh, addr) connect(sh, addr) ACTIVATE OBJECT ACTIVATE OBJECT ACTIVATE OBJECT ACTIVATE OBJECT register_handler(con) register_handler(con) C C O O N N N N E E C C T T I I O O N N C C O O N N N N E E C C T T I I O O N N I I N N I I T T I I A A T T I I O O N N I I N N I I T T I I A A T T I I O O N N P P H H A A S S E E P P H H A A S S E E S S E E R R V V I I C C E E S S E E R R V V I I C C E E I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N I I N N I I T T I I A A L L I I Z Z A A T T I I O O N N P P H H A A S S E E P P H H A A S S E E S S E E R R V V I I C C E E S S E E R R V V I I C C E E P P R R O O C C E E S S S S I I N N G G P P R R O O C C E E S S S S I I N N G G P P H H A A S S E E P P H H A A S S E E complete() complete() open() open()

Figure 5: Collaborations Among Connector Participants for Figure 5: Collaborations Among Connector Participants for Async

Asynchronoushronous ConnecConnectiontionss

chronous scenario, note how the connection initiation phase chronous scenario, note how the connection initiation phase is separated temporally from the service initialization phase. is separated temporally from the service initialization phase. This decoupling enables multiple connection initiations (via This decoupling enables multiple connection initiations (via

connect

connect) and completions (via) and completions (viacompletecomplete) to proceed in) to proceed in parallel within each thread of control.

parallel within each thread of control.

8 8 Im

Impl

plem

emen

enta

tati

tion

on

This section explains the steps involved in building This section explains the steps involved in building commu-nicat

nicationion softwsoftware are appliapplicatiocationsns usinusingg the the AccepAcceptor-tor-ConnecConnectortor patte

pattern. rn. The implemThe implementatentation in ion in this sectithis section on is based on is based on thethe reusable components and applications in the ACE OO reusable components and applications in the ACE OO net-work progra

work programminmming toolkig toolkit [6]. t [6]. ACE provACE provides a rich set of ides a rich set of reusable C++ wrappers and fram

reusable C++ wrappers and frameworkework components that per-components that per-form common communication software tasks across a range form common communication software tasks across a range of OS platforms.

of OS platforms.

The participants in the Acceptor

The participants in the Acceptor-Connector-Connector pattern are di-pattern are di-vided into the

vided into the ReactiveReactive,, ConnectionConnection, and, and ApplicationApplication layers,layers, as shown in Figure 6.

as shown in Figure 6. The

The ReactReactive and ive and ConnecConnection layers tion layers perforperform m genergeneric,ic, appli

applicatiocation-indn-independependent ent strastrategitegies es for for dispdispatchiatching ng eveeventsnts and initializing services, respectively. The Application layer and initializing services, respectively. The Application layer inst

instantiaantiates tes thesthese e genergeneric ic stratstrategieegies s by by proviproviding ding concrconcreteete This diagram illustrates addition

This diagram illustrates additional Booch notation: al Booch notation: directed edges in-directed edges dicate inheritance relationships between classes; a dashed directed edge dicate inheritance relationships between classes; a dashed directed edge in-dicates template instantiatio

dicates template instantiation; n; and a and a solid circle illustrates a solid circle illustrates a compositicompositionon relatio

relationship between nship between two classes.two classes.

2 2

(6)

Initiation

Dispatcher

1 1 1 1

Acceptor

SERVICE_HANDLER SERVICE_HANDLER SERVICE_HANDLER SERVICE_HANDLER PEER_ACCEPTOR PEER_ACCEPTOR PEER_ACCEPTOR PEER_ACCEPTOR

Concrete

Acceptor

Concrete_Service_Handler Concrete_Service_Handler Concrete_Service_Handler Concrete_Service_Handler SOCK_Acceptor SOCK_Acceptor SOCK_Acceptor SOCK_Acceptor 1 1 1 1

Concrete

Service

Handler

SOCK Stream SOCK Stream SOCK Stream SOCK Stream

n

R R E E A A C C T T I I V V E E R R E E A A C C T T I I V V E E L L A A Y Y E E R R L L A A Y Y E E R R C C O O N N N N E E C C T T I I O O N N C C O O N N N N E E C C T T I I O O N N L L A A Y Y E E R R L L A A Y Y E E R R A A P P P P L L I I C C A A T T I I O O N N A A P P P P L L I I C C A A T T I I O O N N L L A A Y Y E E R R L L A A Y Y E E R R sh = make_service_handler(); sh = make_service_handler(); sh = make_service_handler(); sh = make_service_handler(); accept_service_handler (sh); accept_service_handler (sh); accept_service_handler (sh); accept_service_handler (sh); activate_service_handler (sh); activate_service_handler (sh); activate_service_handler (sh); activate_service_handler (sh);

n

Event

Handler

A A make_service_handler() make_service_handler() accept_service_handler() accept_service_handler() activate_service_handler() activate_service_handler() open() open() accept() accept()

Connector

connect_service_handler() connect_service_handler() activate_service_handler() activate_service_handler() complete() complete() connect(sh, addr) connect(sh, addr) SERVICE_HANDLER SERVICE_HANDLER SERVICE_HANDLER SERVICE_HANDLER PEER_CONNECTOR PEER_CONNECTOR PEER_CONNECTOR PEER_CONNECTOR

Concrete

Connector

Concrete_Service_Handler Concrete_Service_Handler Concrete_Service_Handler Concrete_Service_Handler SOCK_Connector SOCK_Connector SOCK_Connector SOCK_Connector 1 1 1 1 A A A A connect_service_handler connect_service_handler connect_service_handler connect_service_handler (sh, addr); (sh, addr); (sh, addr); (sh, addr);

1:

activate_service_handler activate_service_handler activate_service_handler activate_service_handler (sh); (sh); (sh); (sh);

2:

ACTIVATES ACTIVATES ACTIVATES ACTIVATES ACTIVATES ACTIVATES CONNECTOR CONNECTOR CONNECTOR CONNECTOR ROLE ROLE ROLE ROLE ACCEPTOR ACCEPTOR ACCEPTOR ACCEPTOR ROLE ROLE ROLE ROLE SERVICE HANDLER SERVICE HANDLER SERVICE HANDLER SERVICE HANDLER ROLE ROLE ROLE ROLE

Service

Handler

PEER_STREAM PEER_STREAM PEER_STREAM PEER_STREAM open() open() AAAA

Figure 6: Layering and Partitioning of Participants in the Acceptor-Connector Pattern Implementation Figure 6: Layering and Partitioning of Participants in the Acceptor-Connector Pattern Implementation

classes that establish connections and perform service classes that establish connections and perform service pro-cessing.

cessing. This separation of concerns increThis separation of concerns increases the reusabil-ases the reusabil-ity, portabi

ity, portabilitylity,, and extensibilitand extensibility in the implementy in the implementationation of theof the Acceptor-Connector pattern.

Acceptor-Connector pattern.

The following discussion of the Acceptor-Connector The following discussion of the Acceptor-Connector pat-tern implementation starts at the bottom with the Reactive tern implementation starts at the bottom with the Reactive layer and

layer and works upwardworks upwards through the s through the ConnecConnection layer andtion layer and Application

Application layerlayer..

8.1 8.1 Rea

React

ctive

ive Lay

Layer

er

The Reactive layer handles events that occur on transport The Reactive layer handles events that occur on transport endpoints represented by I/O handles, such as socket endpoints represented by I/O handles, such as socket end-points. The two participants in this layer, the

points. The two participants in this layer, the InitiationInitiation Dispatcher

Dispatcher andand EvenEvent t HandHandlerler, are deﬁned by the, are deﬁned by the React

Reactor pattern [3]. or pattern [3]. This patteThis pattern enables efﬁrn enables efﬁcient demucient demul- l-tiplexing of multiple types of events from multiple sources tiplexing of multiple types of events from multiple sources within a single thread of control.

within a single thread of control.

The two main roles in the Reactive layer are: The two main roles in the Reactive layer are: Event Handler:

Event Handler: Speciﬁes an interface consisting of hook Speciﬁes an interface consisting of hook met

methodhodss [7][7] thathatt absabstratractlctlyy reprepresresenttheentthe eveventent proprocescessisingng op- op-erations

erations that can that can be providedbe provided by an aby an application. For instpplication. For instance,ance, these hook methods signify events such as a new these hook methods signify events such as a new connec-tion request

tion request, , a a complcompletion of etion of a a conneconnection requesction request t starstartedted asynchronously

asynchronously,, or the arrival of data from a connected peer.or the arrival of data from a connected peer. The

TheAcceptorAcceptorandandConnectorConnectorcomponcomponentsents are concreteare concrete event handlers that derive from

event handlers that derive from EveEventnt HanHandlerdler..

Initiation Dispatcher:

Initiation Dispatcher: DeﬁDeﬁnes nes an an intinterferface ace for for reregis gis--ter

teringing, , remremovioving, ng, and and disdispatpatchichingng EvenEvent t HandHandlerlers.s. The

The SynchronSynchronous ous Event Event DemultipDemultiplexerlexer, such as, such as

select

select [8[8] ] oror WaitForMultipleObjectsWaitForMultipleObjects [9[9], ], in in--forms the

forms the Initiation DispatcherInitiation Dispatcherwhen to call back when to call back appli

applicatication-spon-speciﬁc event eciﬁc event handlhandlers ers in in resporesponse nse to to certcertainain types of event

types of events. s. CommoCommon eventn events include conns include connectioection ac-n ac-ceptance events, data input and output events, and timeout ceptance events, data input and output events, and timeout events.

events.

Note that the

Note that theInitiation DispatcherInitiation Dispatcheris an imple-is an imple-men

mentattationion ofof thetheDispatcherDispatcherpartparticipaicipantnt descrdescribedibed inin Sec- Sec-tion 6.

tion 6. In generalIn general, , the Acceptthe Acceptor-Cor-Conneconnectortor DispatcherDispatcher

participant can be rea

participant can be reactive, proactictive, proactive,ve, or multi-threaded. Theor multi-threaded. The particular

particularInitiatInitiationion DispatchDispatchererinin thisthis implimplemenementa- ta-tion

tion usesuses the the reacreactivtivee modelmodel to to demuldemultipltiplexex andand dispadispatchtch con- con-crete eve

crete event handlernt handlers in s in a single thread of control. a single thread of control. In our In our ex- ex-ample the

ample theInitiatiInitiationon DispatcDispatcherheris a Singleton [10]is a Singleton [10] since we only need one instance of it for the entire process. since we only need one instance of it for the entire process.

8.2 8.2 Con

Connec

nectio

tion

n Lay

Layer

er

The Connection layer The Connection layer

1.

1. CreatCreatesesService HandlerService Handlers;s; 2.

2. PassPassiveively ly or or actiactively connectvely connectss Service HandlersService Handlers

to their remote peers; and to their remote peers; and 3.

3. ActiActivatvateses Service Service HandlerHandlerss once they are con-once they are con-nected.