Design of an embedded inverse-feedforward biomolecular tracking controller for enzymatic reaction processes

(1)

ContentslistsavailableatScienceDirect

Computers

and

Chemical

Engineering

jo u r n al h om ep age :w w w . e l s e v i e r . c o m / l o c a t e / c o m p c h e m e n g

Design

of

an

embedded

inverse-feedforward

biomolecular

tracking

controller

for

enzymatic

reaction

processes

Mathias

Foo

a,∗

_,

_Jongrae

_Kim

b

_,

_Rucha

_Sawlekar

a

_,

_Declan

_G.

_Bates

a a_Warwick_Integrative_Synthetic_Biology_Centre,_School_of_Engineering,_University_of_Warwick,_Coventry_CV4_7AL,_UK b_School_of_Mechanical_Engineering,_University_of_Leeds,_Leeds_LS2_9JT,_UK

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received14October2016

Receivedinrevisedform10January2017 Accepted16January2017

Availableonline17January2017

Keywords: Processcontrol

Enzymaticreactionprocess Chemicalreactionnetworktheory Syntheticbiology

Biologicalengineering

a

b

s

t

r

a

c

t

Feedbackcontroliswidelyusedinchemicalengineeringtoimprovetheperformanceandrobustness ofchemicalprocesses.Feedbackcontrollersrequirea‘subtractor’ thatisabletocomputetheerror betweentheprocessoutputandthereferencesignal.Inthecaseofembeddedbiomolecularcontrol circuits,subtractorsdesignedusingstandardchemicalreactionnetworktheorycanonlyrealise one-sidedsubtraction,renderingstandardcontrollerdesignapproachesinadequate.Here,weshowhowa biomolecularcontrollerthatallowstrackingofrequiredchangesintheoutputsofenzymaticreaction processescanbedesignedandimplementedwithintheframeworkofchemicalreactionnetworktheory. Thecontrollerarchitectureemploysaninversion-basedfeedforwardcontrollerthatcompensatesforthe limitationsoftheone-sidedsubtractorthatgeneratestheerrorsignalsforafeedbackcontroller.The pro-posedapproachrequiressigniﬁcantlyfewerchemicalreactionstoimplementthanalternativedesigns, andshouldhavewideapplicabilitythroughouttheﬁeldsofsyntheticbiologyandbiologicalengineering. ©2017TheAuthor(s).PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCCBYlicense (http://creativecommons.org/licenses/by/4.0/).

1. Introduction

Amajorchallengeinsyntheticbiologyistodeveloppractically implementabledesignmethodsforthesynthesisoffeedback con-trollersthatachievereferencetracking,i.e.forcetheoutputofa biomolecularprocessofinteresttotrackdesiredchangesinits con-centrationovertime(Hsiaoetal.,2015).Thedesignoffeedback controllerstocontrolbiochemicalprocesseshasreceived signifi-cantattentionintheliterature(see.e.g.Henson,2003;Baldeaetal., 2013),andtheconstructionofsyntheticcontrolcircuitshasbecome a majorfocusofresearchin thenewfieldof syntheticbiology. Ideally, suchcircuits should be madeup of well-defined mod-ulesconsistingonlyofmolecularreactions,inordertoallowthe realisationofembeddedbiomolecularcontrolsystems(Cosentino etal., 2016).A promising approachtofacilitating thedesign of suchcircuitsisprovidedbynucleicacid-basedchemistry,wherein thedesign of biomolecular circuits can be doneusing abstract

Abbreviations: CRN,chemicalreactionnetwork;DNA,deoxyribonucleicacid; LHS,left-hand-side;RHS,right-hand-side;ODE,ordinarydifferentialequation;PI, proportional-integral;FF,feedforward;IMC,internalmodelcontrol;DSD,DNA stranddisplacement.

∗Correspondingauthor.

E-mailaddresses:[email protected](M.Foo),[email protected](J.Kim), [email protected](R.Sawlekar),[email protected](D.G.Bates).

chemical reactionnetwork (CRN)theory(e.g.Soloveichiketal., 2010),andthentranslatedtodeoxyribonucleicacid(DNA)using stranddisplacementreactionsforimplementations(Chenetal., 2013).ACRNisacollectionofchemicalreactionswritteninthe form

X+Y+...

Reactants

→A

+B

+..

.

Products

(1)

whereisthereactionrate,theleft-hand-side(LHS)ofthereaction consistsofreactantsandtheright-hand-side(RHS)ofthereaction consistsofproducts.Mostofthechemicalreactionsconsideredin thispaperareeitherunimolecular(i.e.onereactantontheLHSof

(1))orbimolecular(i.e.tworeactantsontheLHSof(1)).According tostandardCRNtheory(seee.g.Feinberg,1986,1988)aCRNwith nspeciesandmreactionscanberepresentedbyanordinary differ-entialequation(ODE)followinggeneralisedmass-actionkineticin theformof

dx dt =Pf(x) wherex ∈Rn

≥0isthespeciesconcentration,f(x)∈Rm isa

func-tion describing thereaction rates of the CRN, P ∈Rn×m _is _the

stoichiometricmatrix that describe thedynamics of thespecies concentrationsfollowingtheirassociatedreactionrates,R≥0isthe

non-negativerealnumberset,Ristherealnumberset,andnand marepositiveintegers.

http://dx.doi.org/10.1016/j.compchemeng.2017.01.027

(2)

One-sided subtraction

e

≥ 0, when

r ≥

y

e

= 0, when

r <

y

Two-sided subtraction

e

≥ 0, when

r ≥

y

e

< 0, when

r <

y

Subtraction

r

y

e

Fig.1.Subtractionoperator.

Inanyreferencetrackingfeedbacksystem,itisimperativethat anappropriateerrorsignalcanbecomputedsuchthatthedesigned controllercantakerelevantcontrolactiontodrivetheprocess out-puttowardstheintendedstate.Whilesucharequirementistrivial tosatisfyinstandardcontroltheory,itisnotinthecontextofCRN theory.Thisisbecauseatwo-sidedsubtractor(Fig.1),whichisan operatorthatisabletocomputethedifferencebetweentwoinput signalsregardlessoftheirrelativemagnitude,isyettoberealised usingstandard CRN’s.For accuratereferencetracking,theerror eshouldbeabletotakeboth positive(r>y)and negative(r<y) values.Thus, theaforementionedconstraintis aserious imped-imenttothedesign offunctionalbiomolecularfeedbackcontrol systemsthatwillinevitablyleadtopoorqualityreferencetracking andpotentiallyeveninstability.

To the best of the authors’ knowledge, almost all previous designsforbiomolecularsubtractorsusingCRNshaveresultedin onlyone-sidedsubtraction.Wenotethatthereisaliteratureon thedesignofhalf-subtractorsorfull-subtractorsusingdigitallogic gatesrealisedusingCRNs(seee.g.Xuetal.,2013;Linetal.,2015), however,asourfocusisonthedesignofanalogbiomolecular cir-cuitry,weexcludethisworkfromourdiscussion.

Thesubtractionoperatorusedinourpaperisbasedonthedesign presentedinBuismanetal.(2009),whichcanberealisedusingaset offourchemicalreactions(seePage6ofBuismanetal.(2009)).The authorsanalysedtheJacobianmatrixoftheODEassociatedwiththe subtractionoperatorandfoundthatwhentheresultingsubtraction iszero,thefixedpointdoesnotexist.Additionally,whenthe result-ingsubtractionisnegative,theoverallsystemdiverges,asthefixed pointisunstable.Inviewofthis,thesubtractionoutputsapositive valuewhenthemagnitudeofthefirstcomponentisgreaterthan thesecondcomponentandzerowhentheconditionisreversed. WefurtherillustratethispointinSection2.2ofourpaper.Some otherrelevantresultsonbiomolecularsubtractioncanbefoundin

Salehietal.(2016)andSongetal.(2016).Thedesignconsidered

intheformerusedthesubtractionoperatortorealisea biomolec-ularcomputationofaBersteinpolynomialandtheirsubtraction isequivalent tothedesign of Buismanet al. (2009).The latter paperproposedframeworkstobuildoperatorsusingDNAstrand displacement,andexplicitlymentionedthattheirsubtraction oper-atorisone-sided.In Harrisetal.(2015),theauthorsdesigned a feedbackcontrollerforgeneexpressionregulation,whichrequires theuseofasubtraction,butdonotproposeadetailedbiomolecular implementationofthesubtractionoperator.

Analternativeapproachtothedesignofbiomolecular subtrac-tionoperatorscanbefoundinCosentinoetal.(2013,2016)and

Bilottaet al.(2015, 2016). In this work,theauthors developed

subtractorsthatareusedtocomputethedifferencebetweentwo molecularfluxes,ratherthantwomolecularconcentrations.By sat-isfyingcertainconditions,andassumingallthereactionsinvolve unitarystoichiometriccoefficientswithinputfluxesconstant,the outputfluxisshowntoconvergetothedifferencebetweenthetwo inputfluxesinanasymptoticmanner.Whilethechemicalreactions requiredtorealisethesubtractionoperatorinthisframeworkare slightlydifferenttotheonesproposedinBuismanetal.(2009),the

ﬁnalODErepresentationisexactlythesame,andthusalsoyieldsa one-sidedsubtraction.

Theonlyavailablepartialsolutiontotheproblemmentioned aboveis toadoptthe designframework proposedin Oishi and

Klavins(2011).Inthisframework,eachsignalinthe

biomolecu-larcircuitisimplementedasthedifferenceintheconcentrationof twochemicalspecies.Inthisway,atwo-sidedsubtractionoperator canthenberealised.Asweshowinthefollowingsection, how-ever,thisapproachatleastdoublesthetotalnumberofchemical reactionsrequiredtoimplementtheentirefeedbackcircuit.This increasein thenumber ofchemical reactionsis highly undesir-ableasitpresentsamajorchallengeforwetlabimplementation, andstronglylimitsthescalabilityofthedesign.Moreover,large numbersof chemical reactionspotentially increases the proba-bilityof unwantedcrosstalk interactions. Forinstance, a circuit whoseimplementationrequiresnmolecularspecieswillincrease thepotential bimolecular crosstalkinteractions byn2_._This _has

promptedresearcherstolookintowaystoreducecrosstalk,such asrequiringacertainnumberofmismatchesforanytwodistinct recognitiondomains(seee.g.QianandWinfree,2011). Nonethe-less,obtaininglargenumbersofwell-behavedsequenceswithlong domainsisextremelydifﬁculttoachieveinpractice.

Inthispaper,usingtheavailableframeworkforrealising one-sidedbiomolecularsubtractionusingCRNs(seee.g.Buismanetal., 2009),weproposeadesignstrategythatusesamodel-inversion feedforwardcontroller(seee.g.Devasia,2002;FranklinandPowell, 2014)tocircumventthelimitationsofthefeedbackcontrollerwhen usingaone-sidedsubtractionoperator.Inaddition,ourcontroller designstrategyalsoaimstoutilisetheminimalnumberof chem-icalreactions,toallowfora morescalableandfeasiblewetlab implementation.

2. Materialsandmethods

2.1. One-sidedsubtractionoperator

Tothebestofauthors’knowledge,allcurrentexistingdesigns forbiomolecularsubtractionoperatorsthatutilisestandardCRN theorycanonlyimplementone-sidedsubtraction.A comprehen-sivelistofmathematicaloperatorsthatcanbeimplementedusing CRN’s,whichincludestheone-sidedsubtractionanditsdetailed analysescanbefoundinBuismanetal.(2009).Followingthedesign

ofBuismanetal.(2009),thesubtractionoperatorcanberealised

usingthefollowingabstractchemicalreactions:

xi,1

→xi,1+xo, xitd+xo →∅

xi,2

→xi,2+xitd, xo →∅

(2)

wherexi,1andxi,2arethetwoinputs,xoistheresultingoutput,

xitdistheintermediatestatesandisthereactionrate.Notethat

thisone-sidedsubtractionoperatorisrealisedusingfourabstract chemicalreactions.Usinggeneralisedmass-actionkinetics(seee.g.

Feinberg,1986),theseabstractchemicalreactionscanbe

repre-sentedbyODE’s,wherethecorrespondingODE’sfor(2)aregiven by

dxo

dt =(xi,1−xoxitd−xo) dxitd

dt =(xi,2−xoxitd)

(3)

Atsteadystate,xi,2=xoxitd,leading toxo=xi,1−xi,2.By analysing

(3)

anunstableﬁxedpoint,respectively.Hence,xo=0whenxi,1<xi,2

andxo=xi,1−xi,2>0whenxi,1≥xi,2,makingthesubtraction

one-sided.Otherexampleofone-sidedsubtraction operatorscanbe foundinCosentinoetal.(2016)–inthesecasestheoperatorsare usedtocomputethedifferenceofmolecularﬂuxes,ratherthan concentrations.

2.2. Two-sidedsubtractionoperator

AsmentionedinSection1,theonlyknownframeworktodate thatoffersapartialsolutiontotherealisationofatwo-sided sub-tractionoperatorcanbefoundinOishiandKlavins(2011).Inthat proposedframework, a signal,u is representedas a difference betweentwo chemicalspecies i.e. u:=u+₋_u−_. _This

representa-tionresultsinthechemicalspecieshavingpositiveandnegative components.Toillustratehowsuchrepresentationcanachievea two-sidedsubtraction,weconsiderﬁrsthowthesummation oper-atorcanberealisedusingabstractchemicalreactions.Theabstract chemicalreactionsforthesummationoperatoraregivenby

x+_i,1

→x+_i,1+x+o, x−i,1

→x−_i,1+xo−, x+i,1+x−i,1

→∅

x+_i,₂→x+_i,₂+x+o, x−i,2

→x−_i,₂+xo−, x+i,2+x−i,2

→∅

x+o →∅, x−o

→∅, x+o +x−o →∅

(4)

whereisareactionratesuchthat.ThecorrespondingODEs aregivenby

dx+o

dt =(x+i,1+x+i,2−x+o)−xo+x−o

dx−o

dt =(x−i,1+x−i,2−x−o)−xo+x−o

dxo

dt = dx+o

dt − dx−o

dt =(xi,1+xi,2−xo)

(5)

whereatsteadystate(i.e.dxo/dt=0),xo=xi,1+xi,2.

Now,forthesubtractionoperator,itsabstractchemical reac-tionsaregivenby

x+_i,₁→x+_i,₁+x+o, x−i,1

→x−_i,₁+xo−, x+i,1+x−i,1

→∅

x+_i,₂→x+_i,₂+x−o, x−i,2

→x−_i,₂+xo+, x+i,2+x−i,2

→∅

x+o →∅, x−o

→∅, x+o +x−o →∅

(6)

Noticethedifferenceofthesuperscripts+and−intheabstract chemicalreactioncomparedto(4).ThecorrespondingODEsare givenby

dx+o

dt =(x

+

i,1+x−i,2−x+o)−xo+x−o

dx−o

dt =(x

−

i,1+x+i,2−x−o)−xo+x−o

dxo

dt = dx+o

dt − dx−o

dt =(xi,1−xi,2−xo)

(7)

whereatsteady state,xo=xi,1−xi,2.Becausethesignalis

repre-sentedasthedifferenceofpositiveandnegativecomponents,xocan

beproperlycomputedregardlessoftherelativemagnitudeofxi,1

andxi,2.Notethatboththesummationandsubtractionoperators

in(4)and(6)arerealisedusingnineabstractchemicalreactions. Now, note that the summation of two concentrations that is equivalentto (5) with positive signals could alsohave been realisedbyemployingthefollowingthreeabstractchemical reac-tions(Buismanetal.,2009):xi,1

→xi,1+xo,xi,2

→xi,2+xoandxo →∅. Thisrealisationdoesnotrequirethesignaltoberepresentedusing positive/negativecomponents.Surprisingly,however,an equiva-lentwayofrepresentingthesubtractionoperatorasin(7)seems

nottoexist,astherearenoassociatedabstractchemicalreactions torealiseit.Weshallfurtherdemonstratethispointbelow.

Considerthefollowingtwo reactions:xi,1

→xi,1+yandy

→∅. TheircorrespondingODEsaredy/dt=+xi,1anddy/dt=−y

respec-tively. Their ﬁnal ODE expression can then be obtained by summing these two equations together, i.e. dy/dt=(xi,1−y).

Now, for the subtraction operator, the corresponding ODE is given by dy/dt=(xi,1−xi,2−y). We have already shown how

dy/dt=(xi,1−y) can be obtained. Therefore, we simply need

anotherabstractchemicalreactionthatcanachievedy/dt=−xi,2.

WiththesignontheRHSoftheODEbeingnegative,onewould expecttowriteyontheLHSoftheabstractchemicalreaction.In addition,werequirethemultiplicationofxi,2with,whichmeans

xi,2hastobeontheLHSoftheabstractchemicalreactionaswell.A

naturalﬁrstattemptwouldthenbetowritexi,2+y

→∅.However, asumofreactantsinanabstractchemicalreactionleadsto multi-plicationinthecorrespondingODE,i.e.dy/dt=−xi,2y.Ifweareto

moveytotheRHSoftheabstractchemicalreaction,i.e.xi,2

→xi,2+

y,itscorrespondingODEwouldthenbedy/dt=+xi,2.Thus,there

isnowaytorealisedy/dt=−xi,2usingstandardabstractchemical

reactions.Thisisthereasonwhythepositive/negativecomponents formalismintroducedby Oishi andKlavins(2011) isneeded to realiseatwo-sidedsubtractionoperator.

WhiletheformalismproposedinOishiandKlavins(2011)isable torealisetwo-sidedsubtraction,italsoinevitablyleadstoalarge increase inthenumber ofabstractchemical reactionsrequired, sinceitmustbeusedtorealisealloperatorsandcomponentsinthe circuit,eventhoughinmanycasestherearesimpleralternatives available(e.g.thesummationoperator).Fromthepoint ofview ofexperimentalimplementation,itishighlydesirabletokeepthe numberofabstractchemicalreactionsrequiredassmallas possi-ble,thusmotivatingourattempttoformulateadesignstrategythat cancopewiththelimitationsofaone-sidedsubtractionoperator andthusallowustoavoidusingpositive/negativecomponents.

2.3. Enzymaticreactionprocesses

Inthispaper,wefocusontheproblemofcontrollingenzymatic reactionprocesses.Suchprocessesareubiquitousincellbiology, withsomenotableexamplesincludingproteinphosphorylationby kinases,metabolicsynthesispathwaysandanaerobicfermentation ofglucosetoethanolinyeast(seee.g.Segel,1975;Galazzoand

Bailey,1990;Heinrichetal.,2002;Faederetal.,2003;Hatakeyama

etal.,2003;Charusantietal.,2004;GershonandShaked,2008).

Themainfunctionofenzymesistoactasthecatalystsof bio-chemicalsystems(seee.g.Tayloretal.,2008).Disruptiontothe regulationofenzymaticreactionprocessescanthereforehave sig-niﬁcantadverse effectonbiological function,and theability to accuratelycontrolthedynamicsofenzymaticreactionprocesses attheiroptimallevelsis crucialtoawide rangeof naturaland engineeredbiochemicalsystems.

Thechemicalreactionsdescribingtheenzymaticreaction pro-cessinitssimplestform(seee.g.Segel,1975;GershonandShaked, 2008)aregivenby

xin+xek→r1xi

xi kr2 →xout+xe

xoutk→∅r3

(8)

wherekr1,kr2andkr3aretheprocessbinding,catalyticand

degra-dationratesrespectively.xin,xe,xiandxoutusuallyrepresentthe

(4)

speciﬁcsubstrate toformthe enzyme-substrate complex.After thereaction,thisenzyme–substratecomplexbreaksupresulting intheassociatedproductand theenzymeitself.Thisenzymeis unchangedandonceseparatedfromthecomplexisfreetointeract againwithmoresubstrate.

ThecorrespondingODE’sfor(8)arethengivenby

dxi

dt =kr1xinxe−kr2xi dxout

dt =kr2xi−kr3xout

(9)

Here,weassumethatthetotalconcentrationofxeandxi

repre-sentedasxT=xe+xiisconstant.

3. Theory/calculation

3.1. Derivationofinverse-feedforwardcontroller

Thebasicaimoftheinverse-feedforwardcontrolleristoinvert therelevantdynamicsoftheprocessatsteady stateinorderto provideaccuratesteady-statetrackingofreferencesignals.From

(9),atsteady-state,(i.e.settingbothdxi/dtanddxout/dttozero),we

have

kr1xinxe=kr2xi

kr2xi=kr3xout

(10)

Aftersomealgebraicmanipulation,weget

xin=₍_ˇ˛x₋_ıxout out)

(11)

where˛=kr2kr3,ˇ=kr1kr2xTandı=kr1kr3.Forreferencetracking,

thesteadystatevalueoftheoutput,xoutshouldreachthereference

signal,r.Bysubstitutingxout=randrewritingxin=uinto(11),we

obtaintherequiredcontrolinputsignalsuchthattheprocessis abletotrackthereferencesignalatsteadystate,whichisgivenby

u= ₍ ˛r

ˇ−ır) (12)

Thecontrolsignalgivenin(12)essentiallyamountstoanopen loopcontrolstrategy,inwhichtherelevantdynamicsoftheprocess areinvertedinordertoachieveperfectsteady-statetrackingof ref-erencesignals.Inpractice,ofcourse,theinversionwillnotbeexact, duetomodeluncertainty,andthereisalsonowaytocontrolthe transientdynamicsoftheclosedloopsystemusingthisapproach, whichcouldleadtothepresenceofsteadystateerrors.Toaddress theselimitations,thefeedforwardcontrollerisusedtogetherwith aclassicalproportional-integral(PI)feedbackcontroller,asshown inFig.2.Themainpurposeofthefeedbackcontrolleristocorrect forerrorsintroducedbymodelmismatchandtocontrolthe tran-sientbehaviouroftheprocess.Thus,theﬁnalcontrolsignalacting ontheprocessisgivenbythesumoftheoutputsofthefeedforward andfeedbackcontrollers.NotethatSubtractionIandSubtractionII showninFig.2arebothone-sidedsubtractionoperators.

3.2. Abstractchemicalreactionrepresentationofthetracking controller

Here,weshowhowthetrackingcontrollermayberealisedusing abstractchemicalreactions.AsshowninFig.2,thefeedforward controllerrequirestwogainoperators,onedivisionoperatorand one(one-sided)subtractionoperator.Followingthedesign pro-cedureforallthoseoperatorsgiveninBuismanetal.(2009),the associatedabstractchemical reactionsfor thefeedforward con-trollerpartaregivenasfollows:

α

δ

β

Inverse-Feedforward Controller

Process

K

_I

K

P

∫

PI Controller

r

x

1

x

2

x

₃

x

4

x

5

x

6

x

₆

r

x

7

x

₈

x

₉

x

₁₀

Subtraction I

Division Summation II

Subtraction II Summation I

Fig.2.Blockdiagramconﬁgurationoftheproposedtrackingcontroller.

[Gain,˛:]

r˛→G˛r+x1

r→G˛∅

(13)

whereG˛is˛gainreactionrate.

[Gain,ı:]

rı→Gır+x2

r→∅Gı

(14)

whereG_ıistheıgainreactionrate.

[SubtractionI:]

ˇ→SbIˇ+x3

x2

_SbI →x2+xs

x3

_SbI → ∅

x3+xs _SbI

→ ∅

(15)

whereSbI isSubtractionIreactionrateandxsistheintermediate

speciesinvolvedinSubtractionI. [Division:]

x1

D →x1+x4

x3+x4

_D →x3

(16)

whereDisthedivisionreactionrate.

ForthePIfeedbackcontroller,theassociatedabstractchemical reactionsaregivenasfollows:

[Integralgain:]

x7

KI

→x7+x8 (17)

whereKIistheintegralgain.

[Proportionalgain:]

x7

GKPKP

→ x7+x9

x9

_GKP → ∅

(18)

(5)

[SummationI:]

x8

_SmI →x8+x10

x9

_SmI →x9+x10

x10

_SmI

→∅

(19)

whereSm_IisSummationIreactionrate.

Theerrorresultingfrommodelmismatchandtransienteffects iscomputedbySubtractionII,wherethecorrespondingabstract chemicalreactionsaregivenby

[SubtractionII:]

r→SbIIr+x7

x6

_SbII →x6+xt

x7

_SbII

→∅

xt+x7

_SbII

→∅

(20)

whereSbIIisSubtractionIIreactionrateandxtistheintermediate

speciesinvolvedinSubtractionII.

Theoverallcontrolsignaltobeappliedtotheprocessisthe summationofthecontrolsignalsfromboththefeedforwardand feedbackcontrollers, where theabstractchemical reactionsare givenby

[SummationII:]

x4

_SmII →x4+x5

x10

_SmII →x10+x5

x5

_SmII

→∅

(21)

whereSmIIisSummationIIreactionrate.

With thecontrol signal, x5 as theinput to theprocess, the

abstractchemical reactionsfor theprocesstobecontrolledare givenby(8)withxin=x5andxout=x6.Thus,thenumberof

chem-icalreactionsneededtorealisethecompletecontrolcircuitis26. Ontheotherhand,ifthedesignframeworkthatutilisestwo-sided subtractionasproposedinOishiandKlavins(2011)istobeusedto designjustastandardPIfeedbackcontroller,36chemicalreactions wouldberequiredinthecircuit,anincreaseincomplexityof28%.

3.3. Ordinarydifferentialequationrepresentationofthetracking controller

Using generalised mass-action kinetics, the corresponding ODE’sforalltheabstractchemicalreactionsdescribedinSection3.2

aregivenby

[Feedforwardcontroller:]

dx1

dt =G˛(˛r−x1)

dx2

dt =Gı(ır−x2)

dx3

dt =SbI(ˇ−xsx3−x3)

dxs

dt =SbI(x2−xsx3)

dx4

dt =D(x1−x3x4)

(22)

[Feedbackcontroller:]

dx8

dt =KIx7 dx9

dt =GKP(KPx7−x9)

dx10

dt =SmI(x8+x9−x10)

(23)

[SubtractionII:]

dx7

dt =SbII(r−xtx7−x7)

dxt

dt =SbII(x6−xtx7)

(24)

[SummationII:]

dx5

dt =SmII(x4+x10−x5) (25)

[Process:]

dxi

dt =kr1x5xe−kr2xi dx6

dt =kr2xi−kr3x6

(26)

withxT=xe+xiisconstant.

4. Resultsanddiscussion

4.1. Limitationoffeedbackcontrolwithone-sidedsubtraction

Inthissection,weﬁrstillustratetheeffectofusingaone-sided subtractionoperatorwhenastandardtrackingfeedbackcontroller isused.Fig.3(A)showstheconﬁgurationofastandardfeedback systemwithaPIcontrollerusingaone-sidedsubtractionoperator andthePIcontrolleristunedusingthestandardZiegler–Nicholor InternalModelControl(IMC)methods(seee.g.Ogata,2010;Rivera et al.,1986).FromFig.3(B),attime 0 to40,000s,thefeedback controllerattemptstotrackthereferencevaluebutasovershoot occurs,wehavethesituationthatthereferencevalueissmaller than theoutputvalueresulting innocontrol action beforethe undershootwherethesituationisreversed.Thisleadstothe oscil-latorybehaviourobservedwithinthattimespan.Attime40,000s to80,000s,thereferencevalueissteppeddownandwehavethe situationofthereferencevaluebeingsmallerthantheoutputvalue. Thismeanstheerrorgivenbytheone-sidedsubtractionisalways zeroandthePIcontrollerexertsnocontrolaction,resultinginthe largesteadystateerror.Theperformancedoesnotimprovedespite repeatedattemptstotunethePIcontrollergain–oneofthebest tuningsisshowninFig.3(C),wherethePIcontrollerisstillunable toachieveproperreferencetracking.

4.2. Inverse-feedforwardcontrollerwithone-sidedsubtraction

InFig.4,weshowtheresultsofrepeatingtheabovesimulation withourinverse-feedforwardcontrollerarchitecturefromFig.2. Alltherelevantparametersusedinthesimulationaresummarised

inTable1.Allunitsareassumedtobedeﬁnedappropriately.The

performanceofthecontrollerisgoodasitisabletotrackthe refer-encesignalproperly.Thecontributionofthetwocontrollersisas expected,wheremostofthecontrolactionisgivenbythe feedfor-wardcontrollerwhilethePIcontrollerisoperativewhendealing withtransients.TheparametersofthePIcontrollerareobtained usingstandard tuningmethods e.g.,Ziegler–Nichol/IMCmethod

(seee.g.Ogata,2010;Riveraetal.,1986)followedbyfurtherﬁne

(6)

K

_I

K

_P

_Process

PI Controller

r

y

e

m

n

u

y

∫

(A)

One-sided Subtraction

Summation

0 1 2 3 4 5 6 7 8

x 10 0

2 4 6 8

time [s]

[a.u.]

0 1 2 3 4 5 6 7 8

x 10 0

2 4 6 8 10

time [s]

[a.u.]

output response calculated error reference (set−point)

(B)

(C)

Fig.3. (A)Blockdiagramconﬁgurationofastandardclosed-loopfeedbackcontrolsystemusingaPIcontrollerwithaone-sidedsubtractionoperator.(B)Systemresponse withPIfeedbackcontrollerandone-sidedsubtractionoperatorwithlowercontrolgain,(C)withhighercontrolgain.

0 1 2 3 4 5 6 7 8

x 104 0

2 4 6

time [s]

[a.u.]

0 1 2 3 4 5 6 7 8

x 104 0

0.05 0.1 0.15 0.2

time [s]

[a.u.]

control signal (FF) control signal (PI) reference (set−point) output response

(B) (A)

Fig.4.Systemresponseswiththetrackingcontroller.(A)Outputandreference signals.(B)Controlsignalsfrominverse-feedforward(FF)andPIcontrollers.

Table1

Parametersusedintheclosed-loopfeedbackcontrolsystem.

Parameters Values

Process

kr1 0.005

kr2 1.6

kr3 0.0008

xT 5.5

Inverse-feedforwardcontroller

G˛ 1.0

Gı 1.0

G_SbI 1.0

GD 1.0

PIcontroller

G_KP 1.0

KP 0.02

KI 2.5×10−8

SmI 0.0004

SummationIIandSubtractionII

SmII 1.0

SbII 3.0

0 1 2 3 4 5 6 7 8

x 104 0

1 2 3 4 5 6

time [s]

[a.u.]

output with uncertainties reference (set−point) output response

Fig.5.Robustnessanalysisofthetrackingcontroller.

4.3. Robustnessandsensitivityanalysis

Here,therobustnessof thecontrollerwhen implementedin closed-loopisinvestigatedusingMonteCarlosimulations,where alltheparametersin(22)–(26)arerandomlydrawnfromauniform distributionand theabove simulationsare repeated.The num-berofMonteCarlosimulationsrequiredtoobtainvariouslevels ofestimationuncertaintywithknownprobabilityaredetermined followingChernoffbound(Vidyasagar,1998).Followingthe guide-linesgiveninWilliams(2001),atotalnumberof1060simulations arerequiredtoaccomplishanaccuracylevelof0.05 with conﬁ-dencelevelof99%(Vidyasagar,1998;Menonetal.,2009).Allthe parametersarevariedwithinrangesof10% aroundtheir nomi-nalvalues.Mathematically,thisiswrittenas(1+0.1), where ∈{G˛,i,Gı,i,SbI,j,D,i,GKP,i,SmI,k,SbII,j,SmII,k, KP, KI, kr1, kr2,kr3},wherei∈{1,2},j∈{1,2,3,4,5}andk∈{1,2,3}.isa

randomnumberfromtheuniformdistributionin[−1,1].Notethat alltheassociatedreactionratesaresplitaccordingtothenumber ofchemicalreactionsinwhichtheyareinvolved.

(7)

Table2

Parametersensitivityanalysisofthetrackingcontroller.Themaximumpercentagerelativesteadystateerrorhastworowsforeachparameter,wheretheupperandbottom rowsdenoteRelativeess,UandRelativeess,Drespectively.

Parameters Rel.ess(%) Parameters Rel.ess(%) Parameters Rel.ess(%)

SubtractionII Inverse-feedforwardcontroller SummationII

SbII,1 29.58 G˛,1 50.00 SmII,1 50.00

31.97 50.00 50.00

SbII,2 20.95 G˛,2 50.00 SmII,2 0.25

23.08 50.00 0.99

SbII,3 0.25 Gı,1 0.25 SmII,3 50.00

0.99 0.00 50.00

SbII,4 20.95 Gı,2 0.25 Process

23.08 0.00 kr1 0.25

SbII,5 20.95 SbI,1 50.00 0.99

23.08 50.00 kr2 0.00

PIcontroller SbI,2 0.25 0.00

KI 0.25 0.00 kr3 0.25

0.99 SbI,3 50.00 0.99

KP 0.00 50.00

0.00 SbI,4 0.25

SmI,1 0.25 0.00

0.99 SbI,5 0.25

SmI,2 0.00 0.00

0.00 GD,1 50.00

SmI,3 0.25 50.00

0.99 GD,2 50.00

G_KP,1 0.00 50.00

0.00

G_KP,2 0.00

0.00

rangeof±10%fromthenominalvalues.Thecontrollerdisplaysa goodlevelofrobustperformancewithnostabilityissuesasaresult ofvaryingparameters.Thesimulationwasrepeatedwith pertur-bationsupto±100%(i.e.(1+)with=1)andnoinstabilitywas observed.

Despite the robustness analysis showing good performance with no stability issues, one notable result warranting further investigationis the presence of steady state error.It is known thatanycontrollerwithintegralaction shouldeliminate steady stateerrorsevenin thepresence ofmodeluncertaintyand this is not observed in our simulation. This is due to the effect of uncertaintyonthebiomolecularsubtractionandsummation oper-ators.From(22)–(26),toachieveexactsummationandsubtraction, theirassociatedreactionrates (Sb and Sm)arerequired tobe

identical.Thiscanbeseenbyconsideringthefollowing summa-tion,i.e.,dy/dt=1x1+2x2−3y.Thesummationisexactwhen

1=2=3,whereatsteadystate,y=x1+x2.Whentheparameters

areperturbed byuncertainty,however,this resultsin the sum-mationreactionratesnolongerbeingidentical(1 =/ 2 =/ 3).

Then,atsteadystate,y=(1/3)x1+(2/3)x2,andthesummation

isnolongerexact.Thus,thesenon-propersummationand subtrac-tionoperatorsarelikelytoresultinthecontrollerbeingunableto computethecorrectcontrolsignalinresponsetotheerror,leading totheobservedsteadystateerror.Toconﬁrmthis,weperformed parametersensitivity analysis,wherebyeach of theparameters showninTable1aremultipliedbyafactorthatrangesfrom0.5 to2.0withincrementsof0.1.Sensitivityisquantiﬁedby comput-ingtherelativesteadystateerrorattimes40,000s(step-up)and 80,000s(step-down)ormathematically,

Relativeess,U=

ˆ

y(40,000)−4 ˆ

y(40,000)

Relativeess,D=

ˆ

y(80,000)−1 ˆ

y(80,000)

(27)

where ˆyistheoutputresponsesubjectedtoparametersensitivity.

Table2showsthemaximumrelativesteadystateerrorsresulting

fromtrackingthestep-upandstep-downofthereferencesignal.

Theresultsofthesensitivityanalysisconﬁrmthatthesteady stateerrorislargelyattributabletotheSummationIIand Subtrac-tionIImodules.Thesetwooperatorsareresponsibleforcomputing the total control signal tothe process and the error to the PI controllerrespectively.Theparametersassociatedwiththe feed-forwardcontrollerarealsohighlysensitive.Thisisexpectedgiven thatthefeedforwardcontrolleroperatesbyinvertingtherelevant dynamicsoftheprocessatsteadystate.Thus,anychangestothe parameterswithinthefeedforwardcontrollercouldresultinthe computationoftheincorrectcontrolsignaltonegatetheprocess dynamic,whichsubsequentlyleadstolargesteadystateerror.Care isthusrequiredwhenspecifyingandimplementingthereaction ratesrelatedtothecontrollerandthesummationandsubtraction operators.

4.4. Simpliﬁedfeedforwardcontroller

Therealisationoftheinverse-feedforwardcontrollerinvolvesa totalof26abstractchemicalreactions.Whilethisproducesa reduc-tionofcircuitcomplexityby28%comparedtousingtheframework proposedinOishi and Klavins(2011),furtherreductionstothe numberofabstractchemicalreactionswouldbehighlydesirable. Thiscanbeachievedusinganalternativewaytorealisethis feed-forwardcontroller,asfollows.From(12),whichwerewritehere,

u= ˛r (ˇ−ır)

wenotethatrappearsinboththenumeratoranddenominator.This fractionalrepresentationcanoftenbeapproximatedbya polyno-mialtoobtainasimpliﬁedrepresentation.TakingtheTaylorseries expansionofuatr*₌_0,_we_have_the_following:

˛r (ˇ−ır) ≈

˛r∗ (ˇ−ır∗)+

˛ˇ

(ˇ−ır∗)2(r−r

∗₎

+ ˛ˇı

(ˇ−ır∗)3(r−r

∗₎2

(8)

Fig.6. Blockdiagramconﬁgurationwithsimpliﬁedfeedforwardcontroller.

Neglectingthecontributionofhigherordertermsandsubstituting r*=0to(28),weget

˛r (ˇ−ır)=

˛ ˇr+

˛ı ˇ2r

2 ₍₂₉₎

Asaremark,forr* ₌_/ _0,_we_can_always_perform_a_change_of_variables

toensurethattheequilibriumiszero.

With this approximation, we have reduced the inverse-feedforward controller to requiring two gain operators, one summation operatorand one polynomialoperator. Substituting therelevantvaluesof˛,ˇandı,whicharerelatedtotheprocess parameters(i.e.kr1,kr2andkr3),weobtainu≈0.029r+0.000002r2.

Giventhatthecoefficientofthesecondtermisverysmall,ourfinal approximationoftheinverse-feedforwardcontrolleristhengiven byu≈0.029r.Hence,wehavefurtherreducedthecomplexityof theinverse-feedforwardcontrollertorequiringonlyonegain oper-ator.Withthat,theblockdiagramconfigurationoftheresulting simplifiedfeedforwardcontrollerisshowninFig.6.

Usingthesamevariablesasinthefullfeedforwardcontroller case,theassociatedabstractchemicalreactionforthesimpliﬁed feedforwardcontrollerisgivenby

[Simpliﬁedfeedforwardcontroller:]

r→Gr+x4

x4

_G →∅

(30)

whereGisthegainreactionrateand=˛/ˇ.TheassociatedODE

isgivenby

dx4

dt =G(r−x4) (31)

Werepeatthereferencetrackingsimulationsasinthecaseof afullfeedforwardcontrollerandtheresultsareshowninFig.7. Theresultsshowthattheperformanceofthesimpliﬁed feedfor-wardcontrollerissimilartothecaseofafullfeedforwardcontroller. However,implementationofthesimpliﬁedfeedforwardcontroller requiresonly18chemicalreactions,areductionof31%and50% fromthefullfeedforwardcontrollerandframeworkproposedin

OishiandKlavins(2011)respectively.

We repeattherobustness analysiswiththe parameters ∈ {G,i,G_KP,i,Sm_I,j,Sb_II,k,Sm_II,j,KP,KI,kr1,kr2,kr3},wherei∈{1,

2},j∈{1,2,3}andk∈{1,2,3,4,5}.TheresultisshowninFig.8. Again,weobservesimilarperformancecomparedtothefull feed-forwardcontroller.Infacttheenvelopeofpossibleresponsesfor thesimpliﬁedfeedforwardcontrollerissmallercomparedtothe fullfeedforwardcontroller,sincethenumberofuncertain param-etersisreduced.Thesensitivityanalysisisalsorepeatedandthe resultsareshowninTable3.

0 1 2 3 4 5 6 7 8

x 104 0

2 4 6

time [s]

[a.u.]

0 1 2 3 4 5 6 7 8

x 104 0

0.05 0.1 0.15 0.2

time [s]

[a.u.]

control signal (simplfied FF) control signal (PI)

reference (set−point) output response

Fig.7.Systemresponsestothetrackingcontrollerusingsimpliﬁedfeedforward controller.Top:output,controlandreferencesignals.Bottom:controlsignalsfrom simpliﬁedfeedforward(FF)andPIcontrollers.

0 1 2 3 4 5 6 7 8

x 104 0

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

time [s]

[a.u.]

output with uncertainties reference (set−point) output response

Fig.8.Robustnessanalysisofthetrackingcontrollerusingthesimpliﬁed feedfor-wardcontroller.

As in the case of the full feedforward controller, the most sensitiveparametersareassociatedwiththesummationand sub-tractionoperatorsaswellasthegainofthesimpliﬁedfeedforward controller.

4.5. Designconsiderationsandlimitationsfordifferent biochemicalprocesses

Thedynamicsoftheenzymaticreactionprocessdependonkr1,

kr2andkr3.Fromtheexperimentalbiologyliterature,itisclearthat

thesethreeparameterscanvaryoverseveralordersofmagnitude -dependingontheparticularbiochemicalprocessinquestion val-uesofkr1,kr2andkr3intheliteraturerangefrom10−9to106,10−2

to102_and₁₀−5_to₁₀−3_respectively_(see_e.g._Faeder_et_al.,_2003;

Hatakeyamaetal.,2003;Charusantietal.,2004).Suchlarge

(9)

Table3

Parametersensitivityanalysisofthetrackingcontrollerusingthesimpliﬁed feed-forwardcontroller.Themaximumpercentageofrelativesteadystateerrorhastwo rows,wheretheupperandbottomrowsdenoteRelativeess,UandRelativeess,D respectively.

Parameters Rel.ess(%) Parameters Rel.ess(%)

SubtractionII Simpliﬁedfeedforwardcontroller

SbII,1 29.58 G,1 50.12

31.97 50.25

SbII,2 21.10 G,2 50.12

23.08 50.25

SbII,3 0.50 SummationII

0.99 SmII,1 50.12

SbII,4 21.10 50.25

23.08 SmII,2 0.50

SbII,5 21.10 0.99

23.08 SmII,1 50.12

PIcontroller 50.25

KI 0.50 Process

0.99 kr1 0.74

KP 0.50 0.99

0.99 kr2 0.74

SmI,1 0.50 0.99

0.99 kr3 0.74

SmI,2 0.50 0.99

0.99

SmI,3 0.50

0.99

G_KP,1 0.50

0.99

G_KP,2 0.50

0.99

theseprocessparametersontheperformanceofthetracking con-trollerusingboththefullandsimpliﬁedfeedforwardcontrollers. We retain all the original parameters apart from the process parameters, which we will vary in our analysis.For each pro-cessparameter,weusetheminimumandmaximumvaluesgiven aboveandconsidertheireffectindependentlyandjointly.Table4

summarisestheﬁndingsandthesimulationresultsareshownin

Figs.9and10.

InFigs.9and10,thesigns(+)and(−)representrespectively

themaximumandminimumvaluesofkr1tokr3oftheprocess.For

example,thenotation(+,−,−)meansthatweareanalysingthe sce-nariowherekr1takesthemaximumvalueofitsrange,i.e.1×106,

andkr2andkr3taketheminimumvalueoftheirrespectiverange,

i.e.1×10−2_and₁_×₁₀−5_._Likewise,_the_notation_(+,+,₋₎_represents

theanalysisofthescenariowherekr1andkr2taketheirmaximum

valueoftheirrespectiverangeandkr3isattheminimumvalueof

itsrange.Ineachofthesubﬁgures((A)–(H)),thetoprowrepresents thereferencetrackingcapabilityoftheoutputandthebottomrow representsthecontrolactiongivenbyboththefeedforwardand feedbackcontrol.

Theobtainedresultsshowthatthetrackingcontroller using both full and simpliﬁed feedforward controller has difﬁculty in tracking reference inputs properly if the degradation term of the process kr3 is very small (∼10−5). As shown in both

Figs. 9and 10(A)–(D),the controlaction of PI(cyan line)stays

at zero most of the time indicating that the subtraction is not working properly due to r<y. Now, with a small value of kr3, y will degrade slowly, meaning r<y for a longer time,

hence thepoortracking (redline).On the otherhand, ifkr3 is

large (∼10−3_), _the _time _for _which _r_<_y _is _short _as _y _degrades

muchfaster,thusleadingtoaccuratetrackingasshowninboth

Figs.9and10(E)–(H).

Forverysmallvaluesofkr1,toachievegoodreferencetracking

requiresthereactionratesintheinverse-feedforwardcontrollerto beincreased.ForimplementationusingDNA-basedchemistry,the abilitytoincreasereactionratescouldpotentiallybeconstrainedby

thephysicalbindingpropertyoftheDNA,thusthispointhastobe takenintoaccountwhenadjustingthosereactionrates.Foralarge valueofkr1,goodreferencetrackingcanbeachievedbydecreasing

thePIcontrollergainsKPandKI.

Ontheotherhand,whenthesimpliﬁedfeedforwardcontroller isused,agoodreferencetrackingcanbeachievedbyjust adjus-tingthePIcontrollergain.Finally,inthecasewhenthevalueof kr1issmallandthevaluesofkr2andkr3arelarge,noadjustment

isrequiredtothecontrollerparameters.InFigs.9and10(E)and (F),wenotethatdespitethetrackingcontrollerbeingabletotrack thereferencesignalproperly,potentiallylargecontrolsignalsare required,whosefeasibilityneedstobecheckedduringthedesign processwithexperimentalists.

4.6. Retroactivity

Here,weinvestigatehowretroactivityaffectstheoverall per-formanceofthetrackingcontrollers.Inprevioussections,wehave assumedperfectmodularityofthedifferentelementsinthe closed-loop feedback control systemshown in Figs. 2 and 6.In other words,thedynamic responsesarenotaffectedbythe intercon-nectionofthecomponents.Whilethisassumptioniswidelymade intheanalysisofchemical reactionbasedsystems,recentwork

(seee.g.DelVechhioetal.,2008)hasshownthatsuchan

assump-tiondoesnotholdfor manybiomolecularfeedback systems.As shown inDel Vechhioet al.(2008),theoccurrence of different modulessharingthesamemolecularspeciesiscommon,andthis sharingofspeciescanaffecttheoveralldynamicsoftheprocesses upon theirinterconnection. To quantifythe waythat intercon-nection of two modules alters theirdynamics with respect to theirbehaviourinisolation,theconceptofretroactivityhasbeen introduced(DelVecchioandJayanthi,2008;Jayanthietal.,2013). Fortheprocessconsideredhere,itshouldbenotedthatno retroac-tivityeffects arepresentdue totheinterconnectionofmodules involvingunimolecularreactions.Forexample,fortheclosed-loop feedbackcontrolsystemsinFigs.2and6,retroactivitydoesnot affecttheinterconnectionofthemoduleSubtractionIIwiththePI controller.

Ontheotherhand,aninterconnectionoftwomodules,where one module comprises unimolecular reactions while the other modulecomprisesbimolecularreactions,willfeaturea unidirect-ionalretroactivity.Again,usinganexampleinthecontextofthe closed-loopfeedbackcontrolsystemshowninbothFigs.2and6, retroactivityaffects theinterconnectionof themodule Summa-tionIIwiththeprocess(see(8) and(21)).Totakeintoaccount the effect of retroactivity, the ODE representation of Summa-tionIImustconsiderthechemicalreactionsdownstream ofthe process(see (25)).ThustheODErepresentation ofthe Summa-tion II module including the effect of retroactivity is given as follows:

dx5

dt =SmII(x4+x10−x5)− k

r1

x5x

e retroactivity

(32)

WerepeatthesimulationsinSection4.2usingboththefulland simpliﬁedfeedforwardcontrollertakingintoaccounttheeffects ofretroactivity,andtheresultsareshowninFigs.11and12.The resultsshowthatretroactivitydoesnotsigniﬁcantlyaffectthe per-formanceoftheoverallclosed-loopsystemotherthantointroduce averysmallsteadystateerror.Thereasonforthissteadystateerror canbeexplainedasthefollowing.Theretroactivityaffectstheinput totheprocess,x5.Atthesametime,x5isalsotheresultingcontrol

(10)

Table4

Effectofprocessparametersonreferencetrackingcapabilityandremarksonthecontroldesignguidelines.Forthereferencetrackingcapability,YdenotesYesandNdenotes No.

Referencetrackingcapability(Y/N) kr1 kr2 kr3 Designremarks

Inverse-feedforwardcontroller

N 1×10−9 ₁_×₁₀−2 ₁_×₁₀−5

N 1×10−9 ₁_×₁₀2 ₁_×₁₀−5

N 1×106 ₁_×₁₀−2 ₁_×₁₀−5

N 1×106 ₁_×₁₀2 ₁_×₁₀−5

Y 1×10−9 ₁_×₁₀−2 ₁_×₁₀−3 _Increase_reaction_rates

Y 1×10−9 ₁_×₁₀2 ₁_×₁₀−3 _Increase_reaction_rates

Y 1×106 ₁_×₁₀−2 ₁_×₁₀−3 _Decrease_K

PandKI

Y 1×106 ₁_×₁₀2 ₁_×₁₀−3 _Decrease_K

PandKI

Simpliﬁedfeedforwardcontroller

N 1×10−9 ₁_×₁₀−2 ₁_×₁₀−5

N 1×10−9 ₁_×₁₀2 ₁_×₁₀−5

N 1×106 ₁_×₁₀−2 ₁_×₁₀−5

N 1×106 ₁_×₁₀2 ₁_×₁₀−5

Y 1×10−9 ₁_×₁₀−2 ₁_×₁₀−3 _Increase_K

PandKI

Y 1×10−9 ₁_×₁₀2 ₁_×₁₀−3 _No_change_to_existing_parameters

Y 1×106 ₁_×₁₀−2 ₁_×₁₀−3 _Decrease_K

PandKI

Y 1×106 ₁_×₁₀2 ₁_×₁₀−3 _Decrease_K

PandKI

totheobservedsteadystateerror.Nevertheless,ourresultsshow thattheperformanceofthetrackingcontrollerisnotsigniﬁcantly affectedbyretroactivity,allowingustoundertakedesignsbasedon theassumptionofmodularity(asisthecasewithstandardcontrol applications).

4.7. ImplementationofCRNviaDNAstranddisplacement chemistry

Finally,we brieﬂyprovideanoverview ofthewayin which thetypeofdesignspresentedinthispapermaybeimplemented

0 1 2

x 105 0

2 4

[a.u]

(−, −, −)

0 1 2

x 105 0

5 10

[a.u]

0 1 2

x 105 0

2 4

(−, +, −)

0 1 2

x 105 0

5 10

0 1 2

x 105 0

5 10

(+, −, −)

0 1 2

x 105 0

2 4x 10

−10

0 1 2

x 105 0

5 10

(+, +, −)

0 1 2

x 105 0

2 4x 10

−10

0 1 2

x 105 0

2 4

[a.u]

(−, −, +)

0 1 2

x 105 0

5 10x 10

5

time [s]

[a.u]

0 1 2

x 105 0

2 4

(−, +. +)

0 1 2

x 105 0

5 10x 10

5

time [s]

0 1 2

x 105 0

2 4 6

(+, −, +)

0 1 2

x 105 0

0.5 1x 10

−9

time [s]

0 1 2

x 105 0

2 4 6

(+, +, +)

0 1 2

x 105 0

0.5 1x 10

−9

time [s]

x 103 x 103

(A)

(B)

(C)

(D)

(H)

(G)

(F)

(E)

(11)

0 1 2

x 105 0

2 4

(−, −, −)

[a.u.]

0 1 2

x 105 0

5 10

[a.u.]

0 1 2

x 105 0

2 4

(−, +, −)

0 1 2

x 105 0

5 10

0 1 2

x 105 0

5 10

(+, −, −)

0 1 2

x 105 0

2 4x 10

−10

0 1 2

x 105 0

5 10

(+, +, −)

0 1 2

x 105 0

2 4x 10

−10

0 1 2

x 105 0

2 4 6

(−, −, +)

[a.u]

0 1 2

x 105 0

5 10x 10

5

time [s]

[a.u]

0 1 2

x 105 0

2 4 6

(−, +, +)

0 1 2

x 105 0

5 10x 10

5

time [s]

0 1 2

x 105 0

2 4 6

(+, −, +)

0 1 2

x 105 0

0.5 1x 10

−9

time [s]

0 1 2

x 105 0

2 4 6

(+, +, +)

0 1 2

x 105 0

0.5 1x 10

−9

time [s]

x 103 x 103

(A)

(B)

(C)

(D)

(H)

(G)

(F)

(E)

Fig.10.Effectofvaryingprocessparametersonreferencetrackingusingsimplifiedfeedforwardcontroller.Thenotation‘+’and‘−’denotesrespectivelythemaximumand minimumvaluesoftheprocessparameter.Redline:outputresponse.Blackline:reference(set-point).Magentaline:controlsignalfromsimplifiedfeedforwardcontroller. Cyanline:controlsignalfromPIcontroller.(Forinterpretationofthereferencestocolorinthisfigurelegend,thereaderisreferredtothewebversionofthearticle.)

experimentally.Recently,therehavebeenanumberofstudiesthat describedhowsyntheticcircuitscomposedofabstractchemical reactionsmaybereadilyimplementedusing DNA-based chem-istry(seee.g.Seeligetal.,2006;Soloveichiketal.,2010;Qianand

Winfree,2011;Chenetal.,2013).InChenetal.(2013),theauthors

highlightedthatchemicalreactionscanserveasaprogramming languageforthedesignofDNA-basedchemistrysyntheticcircuits. Therefore,circuitcomponentssynthesisedusingDNAthatcanbe expressedmathematicallycanbederivedfrombiologically synthe-sisedplasmids.Consequently,inprinciplethisenablestheinvitro implementationofthosecircuits.Theadvantageofutilising DNA-basedchemistryinthedesignliesintheeaseofimplementation, asthedesigndependsonthechoiceofrelevantsequences follow-ingthestandardWatson-Crickpairing(i.e.adenine-thymineand guanine-cytosineorA-TandG-C).Asaresult,designframeworks and software tools that allow synthetic biomolecular circuitry describedusingCRN’stobereadilyimplementedusingDNAstrand displacement(DSD)chemistryhavebeendevelopedinrecentyears

(see e.g. Seelig et al., 2006; Soloveichiket al., 2010; Qian and

Winfree,2011;Chenetal.,2013).

In particular,it hasbeen shown in Soloveichiket al.(2010)

thatunimolecular(onereactantattheLHSofthechemical reac-tion)andbimolecular(tworeactantsattheLHSofthechemical reaction)chemicalreactionscanbecompiledinDSDchemistryto

accomplishtheintendedbehaviouroftheirdesigned biomolecu-larcircuit.Alldesignspresentedinthisworkareallmadeupof eitherunimolecularorbimolecularchemicalreaction,makingthis frameworksuitableforthedesignedcontrollertobeimplemented experimentally.Here,we brieﬂydescribetheirproposed frame-workandreferinterestedreaderstoSoloveichiketal.(2010)for details.

ConsiderthefollowingbimolecularDSDreaction,

X+Qkb

k_ubY+W (33)

wherekbisthebindingreactionrateandkubistheunbinding

reac-tionrateoftheDNAstrand.Thereactioncommences whenthe invaderstrandQbindsinastandardWatson–Crick complemen-tarymannertothetoe-holddomainofstrandX.Asthebinding occurs,portionsofthestrandofXaredisplacedwherebythis sep-arationresultsintheproductYandwasteW.Thispartiallydouble strandedproductYcanthenbindwithtothetoe-holddomainof otherDNAstrandsandcomplexesforsubsequentreactions.The rateoftheoverallreactioncanbecontrolledbyvaryingthe bind-ingand unbindingrates, kb andkub.Quite often,differentDNA

(12)

0 1 2 3 4 5 6 7 8

x 104 0

2 4 6

time [s]

[a.u.]

0 1 2 3 4 5 6 7 8

x 104 0

0.05 0.1 0.15 0.2

time [s]

[a.u.]

control signal (FF) control signal (PI)

20,000 40,000 3.8

4 4.2

Fig.11.Systemresponseswithretroactivityusinginverse-feedforwardcontroller. Top:outputandreferencesignals.Bottom:controlsignalsfrominverse-feedforward (FF)andPIcontrollers.

0 1 2 3 4 5 6 7 8

x 104 0

2 4 6

time [s]

[a.u.]

0 1 2 3 4 5 6 7 8

x 104 0

0.05 0.1 0.15 0.2

time [s]

[a.u.]

control signal (simplified FF) control signal (PI)

20,000 40,000 3.8

4 4.2

Fig.12.Systemresponsestoretroactivityusingsimpliﬁedfeedforwardcontroller. Top:output,controlandreferencesignals.Bottom:controlsignalsfromsimpliﬁed feedforward(FF)andPIcontrollers.

implementationfor the respectiveunimolecular (X→Y+Z) and bimolecular(X+Y→Z)reactionsaregivenby,

X+G→qO

O+Tq→maxY+Z

(34)

and

X+Lq

qmaxH+B

Y+Hq→maxO

O+Tq→maxZ

(35)

whereG,O,T,L,HandBaretheauxiliaryspecieswithappropriate initialconcentrationsCmax.Thepartialstranddisplacementrateis

givenbyq=kb/Cmaxandqmaxisthemaximumstranddisplacement

rate.

Sincetheabstractchemicalreactionsdescribingthetracking controllerpresentedherearemadeupexclusivelyof unimolec-ular and bimolecular reactions, the framework introduced in

Soloveichiketal.(2010)enablesourdesigntobeimplementedvia

DNAchemistry.Nevertheless,sincetheintroductionofauxiliary speciesfurtherincreasesthenumberofreactionneededforDSD implementations,itisimperativethatcircuitdesignsutiliseasfew reactionsaspossible.Hence,ourresultsshowingthefeasibilityof aradicallysimpliﬁedinverse-feedforwardtrackingcontrolleroffer strongpotentialforfuturewetlabimplementationsofembedded biomolecularcontrolsystems.

5. Conclusions

In this paper,we haveshown for thefirst time how a con-trollerarchitectureforimplementingreferencetracking,basedon theuseof aninverse-feedforwardcontroller,canbeadaptedto thespecificcontextofembeddedbiomolecularfeedbacksystems. Ourproposedapproachcircumventsafundamentallimitationof CRN-basedsystemsfromthepointofviewoftrackingcontrol,i.e. theinabilityofstandardCRNstoimplementatwo-sided subtrac-tionoperator.Theonlycurrentlyavailablesolutiontothisproblem proposedby Oishi and Klavins(2011) requires theadoption of anon-standardmodellinganddesignframeworkthatrepresents signals as the difference in the concentration of two chemical species,anapproach thatsignificantly increasesthecomplexity ofthedesignprocessanddoublesthetotalnumberofchemical reactionsneededtoimplementagivencircuit.

Thecomplexityofthecontrolproblemconsideredhereis sig-nificantlyhigher thanin previousstudies– almostall previous studiesconsidereitherastaticora verysimplefirstorder pro-cessthatavoidsthepossibilityofthereferencevaluebeinglarger thantheoutputvalue(Cosentinoetal.,2016)oremploying con-trolstrategiesthatcanreducethedurationofthereferencevalue beingsmallerthantheoutputvalue(Briatetal.,2016)orconsider onlyaverysimplestaticprocesswithnodynamics(Yordanovetal., 2014).Usingtheexistingstandardrealisationofone-sided subtrac-tion,ourapproach ofusinginversefeedforwardcombinedwith feedbackcontrolproduces highlyaccurateandrobustreference tracking.Byexploitingthebiochemicalstructure ofthe feedfor-wardcontroller,weareabletoreduceitscomplexityusingaTaylor seriesapproximation.Theresultingsimplifiedfeedforwardcontrol notonlyachievessimilarperformancetothefullfeedforward con-troller,butalsoutilisesfarfewerchemicalreactions.Thereduction inthenumberofchemicalreactionsisimportant,asitwill signifi-cantlyfacilitatetheexperimentalimplementationoftheproposed designinDNA-basedchemistryeitherinvitroorinvivo.

Conﬂictofinterest

Theauthorsdeclarethattheyhavenocompetinginterests.

Acknowledgements

ThisworkissupportedbyEPSRCandBBSRCviaresearchgrants BB/M017982/1andtheSchoolofEngineeringoftheUniversityof Warwick.

AppendixA. Supplementarydata

Supplementarydataassociatedwiththisarticlecanbefound, intheonlineversion,athttp://dx.doi.org/10.1016/j.compchemeng.

2017.01.027.

References

(13)

Comput.Chem.Eng.51,42–54,http://dx.doi.org/10.1016/j.compchemeng. 2012.05.016.

Bilotta,M.,Cosentino,C.,Bates,D.G.,Salerno,L.,Amato,F.,2015.Retroactivity analysisofachemicalreactionnetworkmoduleforthesubtractionof molecularﬂuxes.In:ProceedingsofIEEEConferenceonEngineeringin MedicineandBiologySociety,pp.941–944,http://dx.doi.org/10.1109/EMBC. 2015.7318518.

Bilotta,M.,Cosentino,C.,Merola,A.,Bates,D.G.,Amato,F.,2016.Zero-retroactivity subtractionmoduleforembeddedfeedbackcontrolofchemicalreaction networks.In:ProceedingsofIFACConferenceonFoundationsofSystems BiologyinEngineering.

Briat,C.,Zechner,C.,Khammash,M.,2016.Designofasyntheticintegralfeedback circuit:dynamicanalysisandDNAimplementation.ACSSynth.Biol.5, 1108–1116,http://dx.doi.org/10.1021/acssynbio.6b00014.

Buisman,H.J.,tenEikelder,H.M.M.,Hilbers,P.A.J.,Liekens,A.M.L.,2009.Computing algebraicfunctionswithbiochemicalreactionnetworks.Artif.Life15,1–15, http://dx.doi.org/10.1162/artl.2009.15.1.15101.

Charusanti,P.,Hu,X.,Chen,L.,Neuhauser,D.,DiStefanoIII,J.J.,2004.A mathematicalmodelofBCR-ABLautophosphorylation,signalingthroughthe CRKLpathwayandGleevecdynamicsinchronicmyeloidleukemia.Discrete Contin.Dyn.Syst.Ser.B4,99–114,http://dx.doi.org/10.3934/dcdsb.2004. 4.99.

Chen,Y.J.,Dalchau,N.,Srinivas,N.,Phillips,A.,Cardelli,L.,Soloveichik,D.,Seelig,G., 2013.ProgrammablechemicalcontrollersmakefromDNA.Nat.Nanotechnol. 8,755–762,http://dx.doi.org/10.1038/nnano.2013.189.

Cosentino,C.,Ambrosino,R.,Ariola,M.,Bilotta,M.,Pironti,A.,Amato,F.,2016.On therealizationofanembeddedsubtractormoduleforthecontrolofchemical reactionnetwork.IEEETrans.Autom.Control61,3638–3643,http://dx.doi.org/ 10.1109/TAC.2016.2523679.

Cosentino,C.,Bilotta,M.,Merola,A.,Amato,F.,2013.Asyntheticbiologyapproach totherealizationofembeddedfeedbackcontrollersforchemicalreaction networks.In:ProceedingsofIEEEConferenceonBioinformaticsand Bioengineering,http://dx.doi.org/10.1109/BIBE.2013.6701600. DelVecchio,D.,Jayanthi,S.,2008.Retroactivityattenuationintranscriptional

networks:designandanalysisofaninsulationdevice.In:ProceedingsofIEEE ConferenceonDecisionandControl,pp.774–780,http://dx.doi.org/10.1109/ CDC.2008.4739037.

DelVechhio,D.,Ninfa,A.,Sontag,E.,2008.Modularcellbiology:retroactivityand insulation.Mol.Syst.Biol.,4,http://dx.doi.org/10.1038/msb4100204. Devasia,S.,2002.Shouldmodel-basedinverseinputsbeusedasfeedforward

underplantuncertainty?IEEETrans.Autom.Control47,1865–1871,http://dx. doi.org/10.1109/TAC.2002.804478.

Faeder,J.R.,Hlavacek,W.S.,Reischl,I.,Blinov,M.L.,Metzger,H.,Redondo,A.,Wofsy, C.,Goldstein,B.,2003.Investigationofearlyeventsinfcri-mediatedsignaling usingadetailedmathematicalmodel.J.Immunol.170,3769–3781,http://dx. doi.org/10.4049/jimmunol.170.7.3769.

Feinberg,M.,1986.Chemicalreactionnetworkstructureandthestabilityof complexisothermalreactors–I.Thedeﬁciencyzeroanddeﬁciencyone theorems.Chem.Eng.Sci.42,2229–2268, http://dx.doi.org/10.1016/0009-2509(87)80099-4.

Feinberg,M.,1988.Chemicalreactionnetworkstructureandthestabilityof complexisothermalreactors–II.Multiplesteadystatesfornetworkdeﬁciency one.Chem.Eng.Sci.43,1–25, http://dx.doi.org/10.1016/0009-2509(88)87122-7.

Franklin,G.F.,Powell,J.D.,Emami-Naeini,A.,2014.FeedbackControlofDynamic Systems,7thed.Pearson.

Galazzo,J.L.,Bailey,J.E.,1990.Fermentationpathwaykineticsandmetabolicﬂux controlinsuspendedandimmobilizedS.cerevisiae.EnzymeMicrob.Technol. 12,162–172,http://dx.doi.org/10.1016/0141-0229(90)90033-M.

Gershon,E.,Shaked,U.,2008.H∞feedback-controltheoryinbiochemicalsystems. Int.J.RobustNonlinearControl18,14–50,http://dx.doi.org/10.1002/rnc.1195.

Harris,A.W.K.,Dolan,J.,Kelly,C.,Anderson,J.,Papachristodoulou,A.,2015. Designinggeneticfeedbackcontrollers.IEEETrans.Biomed.CircuitsSyst.9, 475–484,http://dx.doi.org/10.1109/TBCAS.2015.2458435.

Hatakeyama,M.,Kimura,S.,Naka,T.,Kawasaki,T.,Yumoto,N.,Ichikawa,M.,Kim, J.H.,Saito,K.,Saeki,M.,Shirouzu,M.,Yokoyama,S.,Konagaya,A.,2003.A computationalmodelonthemodulationofmitogen-activatedproteinkinase (MAPK)andAktpathwaysinheregulin-inducedErbBsignalling.Biochem.J. 373,451–463,http://dx.doi.org/10.1042/bj20021824.

Heinrich,R.,Neel,B.G.,Rapoport,T.A.,2002.Mathematicalmodelsofprotein kinasesignaltransduction.Mol.Cell9,957–970,http://dx.doi.org/10.1016/ S1097-2765(02)00528-2.

Henson,M.A.,2003.Dynamicmodelingandcontrolofyeastcellpopulationsin continuousbiochemicalreactors.Comput.Chem.Eng.27,1185–1199,http:// dx.doi.org/10.1016/S0098-1354(03)00046-2.

Hsiao,V.,delosSantos,E.L.C.,Whitaker,W.R.,Dueber,J.E.,Murray,R.M.,2015. Designandimplementationofabiomolecularconcentrationtracker.ACS Synth.Biol.4,150–161,http://dx.doi.org/10.1021/sb500024b.

Jayanthi,S.,Nilgiriwala,K.S.,DelVecchio,D.,2013.Retroactivitycontrolsthe temporaldynamicsofgenetranscription.ACSSynth.Biol.2,431–441,http:// dx.doi.org/10.1021/sb300098w.

Lin,H.Y.,Chen,J.Z.,Li,H.Y.,Yang,C.N.,2015.Asimplethree-inputDNA-based systemworksasafull-subtractor.Sci.Rep.,5,http://dx.doi.org/10.1038/ srep10686.

Menon,P.P.,Postlethwaite,I.,Bennani,S.,Marcos,A.,Bates,D.G.,2009.Robustness analysisofareusablelaunchvehicleﬂightcontrollaw.ControlEng.Pract.7, 751–765,http://dx.doi.org/10.1016/j.conengprac.2008.12.002.

Ogata,K.,2010.ModernControlEngineering,5thed.Prentice-Hall.

Oishi,K.,Klavins,E.,2011.BiomolecularimplementationoflinearI/Osystems.IET Syst.Biol.5,252–260,http://dx.doi.org/10.1049/iet-syb.2010.0056. Qian,L.,Winfree,E.,2011.AsimpleDNAgatemotifforsynthesizinglarge-scale

circuits.J.R.Soc.Interface,http://dx.doi.org/10.1098/rsif.2010.0729. Rivera,D.,Morari,M.,Skogestad,S.,1986.Internalmodelcontrol:PIDcontroller

design.Ind.Eng.Chem.ProcessDes.Dev.25,252–265,http://dx.doi.org/10. 1021/i200032a041.

Salehi,S.A.,Parhi,K.K.,Riedel,M.D.,2016.Chemicalreactionnetworksfor computingpolynomials.ACSSynth.Biol.6,76–83,http://dx.doi.org/10.1021/ acssynbio.5b00163.

Seelig,G.,Soloveichik,D.,Zhang,D.Y.,Winfree,E.,2006.Enzyme-freenucleicacid logiccircuits.Science314,1585–1588,http://dx.doi.org/10.1126/science. 1132493.

Segel,I.R.,1975.EnzymeKinetics–BehaviourandAnalysisofRapidEquilibrium andSteady-stateEnzymesSystems.Wiley.

Soloveichik,D.,Seelig,G.,Winfree,E.,2010.DNAasauniversalsubstratefor chemicalkinetics.Proc.Natl.Acad.Sci.U.S.A.107,5393–5398,http://dx.doi. org/10.1073/pnas.0909380107.

Song,T.,Garg,S.,Mokhtar,R.,Bui,H.,Reif,J.,2016.AnalogcomputationbyDNA stranddisplacementcircuits.ACSSynth.Biol.5,898–912,http://dx.doi.org/10. 1021/acssynbio.6b00144.

Taylor,D.J.,Green,N.P.O.,Stout,G.W.,Soper,R.,2008.BiologicalScience,3rded. CambridgeUniversityPress.

Vidyasagar,M.,1998.Statisticallearningtheoryandrandomisedalgorithmfor control.IEEEControlSyst.18,69–85,http://dx.doi.org/10.1109/37.736014. Williams,P.S.,2001.AMonteCarlodispersionanalysisofthex-33simulation

software.In:ProceedingsofAIAAConferenceonGuidance,Navigationand Control,pp.69–85,http://dx.doi.org/10.2514/6.2001-4067.

Xu,S.,Li,H.,Miao,Y.,Liu,Y.,Wang,E.,2013.Implementationofhalfadderandhalf subtractorwithasimpleanduniversalDNA-basedplatform.NPGAsiaMater. 5,e76,http://dx.doi.org/10.1038/am.2013.66.