Patron: Her Majesty The Queen
Rothamsted Research
Harpenden, Herts, AL5 2JQ
Telephone: +44 (0)1582 763133
Web: http://www.rothamsted.ac.uk/
Rothamsted Research is a Company Limited by Guarantee Registered Office: as above. Registered in England No. 2393175. Registered Charity No. 802038. VAT No. 197 4201 51. Founded in 1843 by John Bennet Lawes.
Rothamsted Repository Download
A - Papers appearing in refereed journals
Crout, N. M. J., Craigon, J., Cox, G. M., Jao, Y., Tarsitano, D., Wood, A.
T. A. and Semenov, M. A. 2014. An objective approach to model
reduction: application to the Sirius wheat model. Agricultural and Forest
Meteorology. 189-190, pp. 211-219.
The publisher's version can be accessed at:
•
https://dx.doi.org/10.1016/j.agrformet.2014.01.010
The output can be accessed at:
https://repository.rothamsted.ac.uk/item/8qy21/an-objective-approach-to-model-reduction-application-to-the-sirius-wheat-model.
© 2014. This manuscript version is made available under the CC-BY-NC-ND 4.0 license
http://creativecommons.org/licenses/by-nc-nd/4.0/
ContentslistsavailableatScienceDirect
Agricultural
and
Forest
Meteorology
jo u rn al h om ep a g e :w w w . e l s e v i e r . c o m / l o c a t e / a g r f o r m e t
An
objective
approach
to
model
reduction:
Application
to
the
Sirius
wheat
model
N.M.J.
Crout
a,∗,
J.
Craigon
a,
G.M.
Cox
a,
Y.
Jao
a,
D.
Tarsitano
a,
A.T.A.
Wood
b,
M.
Semenov
c aSchoolofBiosciences,UniversityofNottingham,Loughborough,LE125RD,UKbSchoolofMathematicalScience,UniversityofNottingham,NottinghamNG72RD,UK
cComputationalandSystemsBiologyDepartment,RothamstedResearch,Harpenden,HertsAL52JQ,UK
a
r
t
i
c
l
e
i
n
f
o
Articlehistory: Received15April2013 Receivedinrevisedform 21November2013 Accepted13January2014
Keywords: Cropmodel Wheat Evaluation Complexity Modelreduction Parsimony
a
b
s
t
r
a
c
t
Anexistingsimulationmodelofwheatgrowthanddevelopment,Sirius,wasevaluatedthrougha
system-aticmodelreductionprocedure.Themodelwasautomaticallymanipulatedundersoftwarecontrolto
replacevariableswithinthemodelstructurewithconstants,individuallyandincombination.Predictions
oftheresultantmodelswerecomparedtogrowthanalysisobservationsoftotalbiomass,grainyield,and
canopyleafareaderivedfrom9trialsconductedintheUKandNewZealandunderoptimal,nitrogen
limitinganddroughtconditions.Modelperformanceinpredictingtheseobservationswascomparedin
ordertoevaluatewhetherindividualmodelvariablescontributedpositivelytotheoverallprediction.
Ofthe111modelvariablesconsidered16wereidentifiedaspotentiallyredundant.Areasofthemodel
wheretherewasevidenceofredundancywere:(a)translocationofbiomasscarbontograin;(b)nitrogen
physiology;(c)adjustmentofairtemperatureforvariousmodelledprocesses;(d)allowancefordiurnal
variationintemperature;(e)vernalisation(f)soilnitrogenmineralisation(g)soilsurfaceevaporation.
Itisnotsuggestedthatthesearenotimportantprocessesinrealcrops,rather,thattheirrepresentation
inthemodelcannotbejustifiedinthecontextoftheanalysis.Theapproachdescribedisanalogousto
adetailedmodelinter-comparisonalthoughitwouldbebetterdescribedasamodelintra-comparison
asitisbasedonthecomparisonofmanysimplifiedformsofthesamemodel.Theapproachprovides
automationtoincreasetheefficiencyoftheevaluationandasystematicmeansofincreasingtherigour
oftheevaluation.
©2014TheAuthors.PublishedbyElsevierB.V.
1. Introduction
Simulationmodelsthatpredicttheyieldofagriculturalcrops fromweather,soilandmanagementdatahaveprovidedafocus for cropphysiological researchover thelast threedecades and havecontributedtocurrentunderstandingofcrop-environment interactions.Manysuchmodelshavebeendevelopedforawide rangeof crops,for example,STICS (Brisson etal.,2003), APSIM (Keatingetal.,2003),andDSSAT(Jonesetal.,2003).Thegrowth and developmentof agriculturalcropsin thefield is theresult of non-linear and inter-related processes, and as a result crop modelsarenecessarilycomplex.Evenwhenamodelapproximates individualprocessesbyrelativelysimplerelationships,therearea
∗Correspondingauthorat:SchoolofBiosciences,UniversityofNottingham, Sut-tonBonington,LE125RD,UK.Tel.:+441158516.
E-mailaddress:[email protected](N.M.J.Crout).
largenumberofinter-actingprocessesthathavetobeconsidered. Thereforethecomplexityofcropmodelstypicallyarisesfromthe inter-relationships between modelled mechanisms rather than thesophisticationofindividualprocessrepresentation.Thelevel of detail, the number of processes considered, and the means wherebytheyinteractareallchoicestobemadeinthedesignof themodelleadingtoaverydiverserangeofcropmodeldesigns and,asaresult,aneedforeffectivemethodsofmodelevaluation. Thepurposeof a modelis vitalin defining theapproachtaken toitsdesignandevaluation(Jakemanetal.,2006).ForJamieson et al. (1998b) the explicitaim of a crop model was improved understanding of thecrop’s response toits environment. They describedacropmodelasa‘...collectionoftestablehypotheses...’ and viewedmodel inter-comparisonasamethodoftesting the hypothesesembeddedinthemodelsthroughanexaminationof theirabilitytopredictdetailed withinseasonmeasurementsof thecropsgrowthanddevelopment.ForexampleJamiesonetal. (1998b)presenteda comparisonoftheperformanceof 5wheat modelswithrespecttocropandenvironmentaldatafromwithin the growing season. Their conclusions were directed towards themechanistic basis of thedifferent models.For example the
0168-1923 ©2014TheAuthors.PublishedbyElsevierB.V.
http://dx.doi.org/10.1016/j.agrformet.2014.01.010
Open access under CC BY license.
212 N.M.J.Croutetal./AgriculturalandForestMeteorology189–190(2014)211–219
assumptionsabouttheroleofrootdistributiononwateruptake and the influence of water stress on canopy expansion were highlightedasimportantdifferencesinthemodelsconsidered.
Manyresearchers(Croutetal.,2009;Coxetal.,2006;Anderson, 2005;Kimminsetal.,2008;Bernhardt,2008;Smithetal.,2010) haveemphasisedtheneedforasystematic evaluationofmodel structure. Crout et al. (2009) proposed a conceptually simple methodforundertakinganevaluationofmodelstructureby reduc-ingamodelthroughthereplacementofvariablequantitieswith constants.Byiterativelyreplacingdifferentvariablesin combina-tionwithoneanotherasetofalternativemodelformulationswere created.Theabilityofthereducedmodelstopredictobservations wasthencomparedwiththeoriginalmodelinordertotestthe importanceofthereplacedvariablesinthemodel.Todate pub-lishedworkwiththisapproachhasconsideredrelativelysimple models(e.g.Tarsitano etal., 2011).In thisworkwe extendthe approachtothemorechallengingcase ofafullcropsimulation model withtheaimof explicitly testing the hypothesesof the model.Theusefulnessoftheanalysisisdependentonthereliability andcomprehensivenessoftheobservationaldataused.Inevitably thedataavailablearepartial,andthereforeanymodelanalysisis limitedtosomeextent.Neverthelesswearguethatthisapproach providesgreatersupportforthemodeldesignthansimply com-paringthepredictionsofthefullmodelwithobservations.
2. Methods
2.1. Siriusmodel
A typical example of a process-based wheat model is Sir-ius.Thismodelcalculates biomassproduction fromintercepted photosyntheticallyactive radiationand graingrowthfrom sim-plepartitioningrules (Jamiesonet al.,1998b)utilisingnitrogen responseandphenologicaldevelopmentsub-modelsdescribedby
Jamieson and Semenov (2000). Sirius is an actively developing modelcurrentlybeingappliedatanumberoflevels,frombasic researchtoon-farmdecisionsupport(e.g.SemenovandShewry, 2011;Semenovetal.,2009).
2.2. Observationaldataandmodelcalibration
The reduction approach requires a quantitative comparison between model predictions and observations. In principle this canbebasedonanyrelevantdataseriesavailable.Ourpurpose wasto mechanisticallyevaluate theperformance of the model inreproducingthepatternofgrowthanddevelopmentwithina growingseasonaswellasbetweensitesandseasons(i.e.testing thedescriptionof a growingcropnot just the final yield).We thereforeselecteddatafromtrialswheredetailedgrowthanalysis had been conducted including cases where the major abiotic stressesofnitrogenandwaterlimitationwerepresent.
Datafrom9trialshavebeenusedfortheanalysis(Table1):(i) aspringwheatstudyatLincolnNewZealandwithfourlevelsof watersupply(ii)winterwheattrialsatthreesitesintheUKwith highnitrogenapplicationratesand(iii)afurtherUKwinterwheat trialwithtwolevelsofnitrogenapplication.
Typicallycropmodelsarecalibratedforusewithparticular culti-varsandrequiresitespecificinputsforweatherandsoilconditions. SiriushadbeenpreviouslyappliedtotheNewZealandfielddataby
JamiesonandSemenov(2000)andtheircultivarandsite param-eterswereemployedforthis work.TheUKfieldtrials usedthe cultivarMercia,for whichSiriushad beencalibratedpreviously usingfieldexperimentsintheUK(Wolfetal.,1996;Ewertetal., 2002;Lawlessetal.,2005).Soilandweathercharacteristicsused wereasreportedbyGillettetal.(1999).
2.3. Overviewofreductionprocedure
Theapproachwastocomparethepredictiveperformance(skill) ofalargenumberofalternativemodelformulations.Thesewere basedontheoriginalfullmodelbutwithspecificvariablesreplaced byconstantvalues.Inthiscontextmodelvariablesweredefined as internalquantities calculated usingan assumed relationship expressedintermsofthemodel’sparameters,inputvariablesand othermodelvariables.Thisdefinitionofmodelvariableswas par-tiallysubjectivebecauseintermediatestepsinamodelcalculation couldbedefinedasindividualmodelvariables,orcombinedinto alargerrelationshipasasinglemodelvariable.Suchchoicesoften dependupontherequirementsofspecificcomputer implementa-tion.However,weregardedeachmodelvariableashavingaspecific mechanisticroleinthemodelanddefinedavariableasaspecific modelcomponentwhosevaluewasallowedtochangeduringthe runofthemodel(Coxetal.,2006).Wethereforetestedtheeffectof fixinga modelvariabletoaconstantonthemodel’sskill.Ifthe variablewasimportant for modelprediction onewould expect replacingitwithaconstant tohave adetrimentaleffectonthe comparisonbetweenobservationsandpredictions.
As the assessment of model skillwas based ona compari-sonofobservationsandmodelpredictions(describedbelow),the approachwasexplicitlyreliantonobservationaldata.Theutilityof theanalysiswasthereforedirectlydependentonthequalityand scopeofthedataused.
Thekeystepsintheanalysiswere:
1.Implementation. The analysis was undertaken using a soft-ware environment which manages the variable replace-ment on the fly under programmatic control (OpenModel;
www.nottingham.ac.uk/environmental-modelling.htm). There-fore the first step was to implement the model within this environmentandcheckitsbehaviourtoensureitisthesame astheoriginalsourcemodel.Thiswasaccomplishedthrough comparisonstotheoriginalhardcodedSiriusmodel.
Table1
Summaryoftheexperimentaltrialsusedfortheanalysis.
Trial Sites Cultivar Treatments Year Measurementsmade(figuresin
parenthesisarethenumberof observationaldataavailable)
NZwaterstress(Jamieson andSemenov,2000; Jamiesonetal.,1998a)
Lincoln Battenspring wheat
4levelsofrain shelter
1991–2 Timeseriesoftotalbiomass(48), grainbiomass(35),LAI(59)
UKintensivemanagement (Gillettetal.,1999)
SuttonBonington, Boxworth, Gleadthorpe
Mercia Notreatments, maximuminputs
1992–3 Timeseriesoftotalbiomass(72), grainbiomass,LAI(41),leaf number(74)andanthesisdate(3) UKnitrogenstress
(Gillettetal.,1999)
SuttonBonington Mercia Appliednitrogen levelsof0and 90kgha−1
2.Evaluationofmodelstructuretoenabletheidentificationof can-didatevariablesforreductionanalysis.Notallthevariablesinthe modelmeritedinvestigation.Forexample,manyvariablesinthe modelcodewerefordiagnosticoroutputpurposesandnotpart ofthemodelsfunctionalstructure.Useofthereductionsoftware facilitatedasyntacticalanalysisofthemodelstructureto iden-tifyvariableswhichhadatechnicaloroperationalfunctionin themodelcode.Theremaining111variables,representingthe modelledprocesses,wereconsideredinthescreeninganalysis. 3.Screening analysis. The computational requirements of the reductionprocedureincreasewiththenumberofvariables con-sidered.Thereforeascreeninganalysiswasconductedtoassist inrefiningthelistofcandidatevariables.Thisinvolvedreplacing variablesindividually(oneatatime)ratherthanincombination. 4.Multi-factorial reductionanalysis.In this stagethecandidate model variables were replaced in combination in order to explorethesetofpossiblemodelreplacementsasfullyas possi-ble.
2.4. Estimationofmodelperformance
Weightedresidualsums ofsquares,RSS,werecalculated for eachmodelpermutationconsidered.Theweightsusedwerethe estimatedstandarderrorsoftheobservations.Theseweretakenas 10,10and20%oftheobservedvalueforthebiomass,grainweight andLAIvaluesrespectively.Thesefractionalstandarderrorvalues wereselectedonthebasisofexperiencewithcropanalysisdata sets.Overallmodelperformancewassummarisedbycalculating coefficientsofdetermination(alsoknownasNash–Sutcliffemodel efficiency(NashandSutcliffe,1970)),NS,
NS=1−
(Oj−Mj) sj
2(Oj−O¯) sj
2 (1)whereOjandMjwerethejthobservationandmodelprediction
respectively,sjistheestimatedstandarderrorofthejth
observa-tion,and ¯Owasthemeanoftheobservations.NSisanalogoustor2
inregressionanalysis,themethodofcalculationisidentical, how-ever,theMivaluesareforthemodelconsidered,notnecessarily
thebestfittingmodel.Therefore,unliker2,NScanbenegative.
Inpreviouswork(Croutetal.,2009;Tarsitanoetal.,2011)we usedestimatesofthelikelihoodintegratedoverthemodels param-eterspace(KassandRafferty,1995)asmeasuresoftheprediction skillof eachreduced model combination. However,in thecase ofSirius,modelparametersandreplacementconstantswerenot fittedforeachreducedmodelcombination.Thereforetheuseof integratedmodellikelihoodwasnotappropriateandweemployed aninformalapproachrelatingbeliefinthemodeltotheweighted residualsumsofthesquares.Thisissimilartotheuseofinformal likelihoodsasdiscussedbyBeven(2009).Apseudo-likelihood(Q) wascalculatedforeachreducedmodelcombinationusingan infor-malrelationshipbetweentheresidualsumsofthesquaresofthe reducedandfullmodels
Qi=Aexp
−ln(0.5) (RSSfull−RSSi)
˛
(2)
whereAisanormalisationconstantsuchthat
Qi=1overthemodelsconsidered;RSSfullandRSSiaretheweightedresidualsums
ofsquaresforthefullmodelandithreducedmodelrespectively and˛isaconstantwhosevaluecontrolshowchangesinRSSaffect beliefinthemodel(asmeasuredbyQi).Forexample,ifRSSiexceeds
RSSfullby˛,QiwillbehalfthevalueofQfull.
TheuseoftheinformalQivaluesaspseudo-likelihoodsrequired
cautiousinterpretation.Howeverouraimwastoassessthe mecha-nisticbasisofthemodel,notobtainabsoluteprobabilityvalues.The
effectofchangingthevalueof˛ontheinterpretationwas inves-tigatedbycalculatingtheQiswith˛setat2.5,5.0and10%ofthe
valueofRSSfull.
2.5. Screeninganalysis
Inthescreeninganalysiseach variablewasreplaced individ-ually. Thevalue ofthereplacement constant wasestimatedby minimisingthemodelresidualsumofsquares,subjecttothe con-straintthatthevaluemustbewithintherangethevariabletakes intherunofthefull(unreduced)model.Thesereplacementvalues wereusedthroughoutthesubsequentanalysis.
Variableswhosereplacementhadlittleornodetrimentaleffect onmodelperformancewereconsideredforinclusioninthe multi-factorialanalysis.Whereanumberofthepotentialvariablesrelated tothesameprocess,switchvariableswereintroducedtothecode whichenabledtheeffectofreplacingtheresultofwholeprocess byaconstanttobeconsidered.Forexample,theadjustmentofsoil maximumtemperaturefromobservedmaximumairtemperature wasmodelledusingarelationshipinvolvingtwomodelvariables (ENAVandTADJ)calculatedelsewhereinthemodel.Ratherthan replacetheseindividuallyitwasmoreconvenienttomodifythe modelcodeandintroduceaswitchvariabletoreducethe mod-elled valueof soil maximum temperatureto themaximum air temperature,thereby testingtherole of ENAVand TADJ simul-taneously.Switchvariablesofthistypewereintroducedforsoil minimum temperature,soilmaximum temperatureand adjust-mentofcanopytemperaturefromairtemperature.Ineachcase replacing the switch variable with zero reduced the tempera-turevariabletotheappropriateairtemperature.Furtherswitch variableswereintroducedtoreducethediurnalvariationin tem-peratureusedformanytemperaturedependentprocessesinthe modeltoadailymeantemperatureandtoswitchoffthe vernali-sationsub-model.
2.6. Multi-factorialanalysis
Thereare2N−1differentpossiblecombinationsofreplacements
forNcandidatevariables.Searchingthisreplacementspaceisnot possibleexhaustivelyforevenmoderatevaluesofN.Thereforea stochasticsearchbasedontheMetropolis–Hastingprinciplewas used(e.g.VanOijenetal.,2005;Hastings,1970).Thestateofeach candidatevariablewaseithernormalorreplaced.Theprocedure movedthroughthesearchspacebychangingthesestates.Foreach iterationa stepwasmadetoa newmodel bychangingagiven numberofthevariablestates(eitherfromnormaltoreplacedor viceversa).Thenumberofstatestochangeisadjustedoperationally toachievearandomwalkthroughthesearchspaceratherthanjust arandomsample.InthecaseofSirius,wefoundthatallowingjust onestatetochangeperiterationgavethemostefficientsearch. Theresultsoftheanalysiswereverysimilarwhenuptotwoor threestatechangesperiterationwereallowedbuttheprocedure requiredalargernumberofiterationstoconverge.
At each step the model predictions were compared to the observedvaluestocalculatethepseudo-likelihood(Q;Eq.(2)).The stepwasacceptedif
Qtrial
Qcurrent >r
(3)
whereandwhereQcurrentandQtrialwerethepseudo-likelihoods
ofthecurrentlyacceptedandtrialreducedmodelcombinations respectivelyandrwasarandomvaluebetween0and1.
214 N.M.J.Croutetal./AgriculturalandForestMeteorology189–190(2014)211–219
0 5000 10000 15000 20000 25000 30000
g/m
2
DAS
NZ Natural Rain
0 5000 10000 15000 20000 25000 30000
g/
m
2
DAS
NZ Partial Rain
0 5000 10000 15000 20000 25000 30000
g/
m
2
DAS
NZ Partial Rain
0 5000 10000 15000 20000 25000 30000
g/
m2
DAS
NZ No Rain
0 5000 10000 15000 20000 25000 30000
g/
m
2
DAS
SB Maximum Management
0 5000 10000 15000 20000 25000 30000
g/m
2
DAS
GL Maximum Management
0 5000 10000 15000 20000 25000 30000
g/
m
2
DAS
BX Maximum Management
0 5000 10000 15000 20000 25000 30000
g/
m
2
DAS
SB No Nitrogen
0 5000 10000 15000 20000 25000 30000
50 100 150 200 250
120 170 220 270 320
g/
m2
DAS
SB Low N
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 120 170 220 270 320
120 170 220 270 320 120 170 220 270 320 120 170 220 270 320
Fig.1.Biomassandgrainweightmodelpredictionscomparedtoobservations.Inallcasesgrainweightvaluesareinthebottomrightofthegraph.Thefullmodelisthe dashedline,reducedmodelisthesolidline.
handsideofEq.(3)andforeachiterationthiswascomparedtothe randomdrawr.Theeffectisthatmoveswithasmalldetrimental impactonthemodelfitwillbeacceptedquiteoften,whereasmoves whichseriouslyworsenthefitareunlikelytobeaccepted.Although thewalkthroughthesearchspacemayreturntoapreviously eval-uatedmodelthisdoesnotadverselyaffectthesearchefficiencyin ourcaseasthepreviouslycalculatedpseudo-likelihoodvaluewas usedavoidingtheneedtore-evaluatethemodel.
Thepseudo-likelihoodswerenormalisedtounityoverthe mod-elsconsidered.Areplacementprobabilitywascalculatedforeach individualvariablebysummingthenormalisedpseudo-likelihoods forthemodelswherethegivenvariablewasreplaced.Theresults presentedwerebasedon10,000uniquemodelevaluations.The replacementprobabilitieswerereportedat suitableintervalsto ensuretheyhadstabilisedafterthisnumberofiterations(bysimple inspection).Thereplacementconstantsusedwerethoseobtained foreachofthecandidatevariablesinthescreeninganalysis.
3. Resultsanddiscussion
3.1. Fullmodel
Predictedand observed total biomass,grainweightand leaf areawerecomparedforthefullmodel(Figs.1and2respectively) andsummarisedasNash–Sutcliffestatistics(Table2).Thetrends inbiomassandgrainweightwerewellreproduced,althoughleaf arealesssatisfactory.Theoveralltimingofthecanopywaswell describedinthemodelsimulationsalthoughthecanopysizewas oftenover-predicted.Inpracticeoverpredictionofleafareatends tobedisconnectedfromthepredictionofbiomassandgrainaslight
interceptiondoesnotincreaselinearlywithcanopysize.For exam-ple,over-predictionofleafareaindexfromfourtofiveincreases fractionalinterceptionbyonly5%andthereforehaslittleeffecton predictedcropproduction.Under-predictionofleafareawouldbe expectedtohavedetrimentaleffectsonbiomassandgrainyield prediction.
3.2. Screeninganalysis
Thebehaviourofthereducedmodelsinthescreeninganalysis wassummarisedusingtheratioofRSSforthereducedmodelto thatofthefullmodel.Thedistributionofthesevaluesforthe111 variablesconsideredisshowninFig.3.Theindividualreplacement of29oftheconsideredvariablesbyaconstantincreasedtheratioof RSSforthereducedmodeltothatofthefullmodeltogreaterthan >1.1.Oftheremaining82replacements60resultedinasmallerRSS thantheoriginalfullmodel;theremaining22hadasmall detrimen-taleffect(<10%increase).These82variableswereconsideredas potentialcandidatesinthefactorialanalysis.ThereductionofRSS
Table2
Nash–Sutcliffe(NashandSutcliffe,1970)valuesforthefullandminimumreduced modelsforthepredictionoftotalbiomass,grainweightandleafareaoverallthe observationsconsidered.InthiscaseNash–Sutclifferepresentstheproportionof theweightedvariationaccountedforbythemodel,inordertobeconsistentwith themeasuresusedtosummarisemodelperformanceinthereplacementanalysis.
Fullmodel Minimummodel
Totalbiomass 0.971 0.972
Grainweight 0.845 0.868
0 2 4 6 8 10
LA
I
DAS NZ Natural Rain
0 2 4 6 8 10
LA
I
DAS NZ Partial Rain
0 2 4 6 8 10
LA
I
DAS NZ Partial Rain
0 2 4 6 8 10
LA
I
DAS NZ No Rain
0 2 4 6 8 10
LA
I
DAS SB Maximum Management
0 2 4 6 8 10
LA
I
DAS GL Maximum Management
0 2 4 6 8 10
LA
I
DAS BX Maximum Management
0 2 4 6 8 10
LA
I
DAS SB No Nitrogen
0 2 4 6 8 10
50 100 150 200 250
120 170 220 270 320
LA
I
DAS SB Low N
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 120 170 220 270 320
120 170 220 270 320
120 170 220 270 320
120 170 220 270 320
Fig.2. Modelpredictedcanopyleafareaindex(leafareaperunitgroundarea)comparedtoobservations.Thefullmodelisthedashedline,reducedmodelisthesolidline.
inthisscreeninganalysiswasexpectedasthevaluesofthe replace-mentconstantforeachvariablewereselectedbyfittingthemtothe observeddata.Howeverthisdidsuggestedthatthesevariablesmay notbeimportantforthepredictiveperformanceofthemodel.
Uptothispointtheanalysiswasentirelyautomatic.However atthisstagemechanisticinterpretationofeachofthe82potential variablesroleinthemodelwasrequiredtoensurethatthe replace-mentofspecificvariableswasmeaningful.Forexamplesomeofthe variablesareintermediatestepsinacalculationwhosereduction
0 10 20 30 40 50 60
<0.8 0.8-0.9 0.9-1.0 1.0-1.1 1.1-1.2 1.2-1.3 1.3-1.4 1.4-1.5 >1.5
Fr
equ
en
cy
Rao of reduced model RSS to full model RSS
Fig.3.FrequencyoftheratioofreducedmodelRSStofullmodelRSSforthe111 variablesconsideredinthescreeninganalysis.
wouldbebestaccomplishedthroughthereplacementofthefinal endpointvariable.Thereplacementofsomevariableswouldbreak themassbalanceofthemodel,forexampleallowingittocreate nitrogenordrymattertotranslocatetothegrainirrespectiveofthe statusofthecrop.Onthisbasis54variableswereeliminatedfrom theanalysis.Theremaining28modelvariables,togetherwith5 switchvariableswereidentifiedforinclusioninthemulti-factorial analysis.
3.3. Multi-factorialanalysis
The evolution of the estimated replacement probability for selectedvariablesisshowninFig.4toillustratethegradual con-vergenceofthecomputationalanalysis.
Themodelscomprisingtheuppermost75%ofthemodel prob-abilitydistributionareshowninrankorderinFig.5.Thisshowsa smallnumberofrelativelybetterperformingmodels,followedby agradualdeclineinmodelperformance.Incomparisonto previ-ouslypublishedworkusingthereplacementmethod(Croutetal., 2009;Tarsitanoetal.,2011)theresultsherearenotableforthe largenumberofmodelswithrelativelysimilarperformance.
The replacement probabilities are shown in Table3.Values tendingtounityimplythatmodelperformanceimprovedwhen thevariablewasreplaced(anoisevariable).Valuesof0.5imply that modelperformancewasunaffectedwhen thevariablewas replacedbyaconstant(aredundantvariable).Valuestendingto zeroimpliedthatmodelperformancewasworsewhenthevariable wasreplaced.
216 N.M.J.Croutetal./AgriculturalandForestMeteorology189–190(2014)211–219
Table3
Symbolsandfunctionofthemodelvariablesconsideredinthemulti-factorialanalysistogetherwiththerangeofthevariableinsimulationsofthefullmodelandreplacement probabilitiescalculatedusingthreevaluesof˛(Eq.(2)andfurtherdescribedinthemaintext).
Modelvariable Function Fullmodel
range
Replacement constant
Replacementprobability
Temperatureadjustments ˛=2.50% ˛=5% ˛=10%
S CTEMP Switchtosetcanopytemperatureto airtemperature
n/a 0 0.55 0.54 0.53
SSOILMAX Switchtosettheestimateofmaximum soiltemperaturetomaximumair temperature.Maximumsoil temperatureisusedintheearlystages ofgrowthtoestimatethetemperature controllingplantprocesses.
n/a 0 0.47 0.46 0.46
SSOILMIN Switchtosettheestimateofminimum soiltemperaturetominimumair temperature.Minimumsoil temperatureisusedintheearlystages ofgrowthtoestimatethetemperature controllingplantprocesses.
n/a 0 0.49 0.49 0.50
SHTEMP Switchtoremovethediurnal temperatureadjustmentssothatthe modelusesdailymeantemperature.
n/a 0 0.52 0.53 0.54
ENAV Asoilheatphysicscalculationfeeding intothecalculationofHCROPand maximumsoiltemperature.
0.027–19.7 0.027 0.42 0.46 0.47
HCROP Numeratorinthecorrectionappliedto airtemperaturetoestimatecanopy temperature.
−3.62–7.67 0.76 0.48 0.46 0.47
CONDUC Denominatorinthecorrectionapplied toairtemperaturetoestimatecanopy temperatureinthecanopy
temperaturecorrection
0.014–0.115 0.022 0.45 0.50 0.50
TADJ Atemperaturecorrectionbasedon meanairtempwhichfeedsinto maximumsoiltemperature
0–6.13 0.25 0.57 0.53 0.52
Nitrogenuptake MAXSTEMDE-MAND
Maximumdailystemnitrogenuptake, calculatedfrommaximumstemN
concentration
0–88.8 5.44 0.48 0.47 0.47
LEAFDEMAND Dailyleafnitrogendemandcalculated fromleafexpansionandleafnitrogen requirements.
−4.28–5.73 5.32 0.18 0.29 0.36
MINSTEMDEMAND Differencebetweenstemnitrogen concentrationandtheminimumstem nitrogen(i.e.thestemnitrogendeficit) onwholecropbasis.Settingthis variabletoremovesplantcontrolon nitrogenuptakesothatuptakeis limitedonlybysoilsupply.
0–58.5 0 0.41 0.43 0.44
Grainfilling
BIOANTH Biomassatanthesis;usedtodetermine themaximumbiomassavailablefor translocationduringgrainfilling
0–12602 9259 0.61 0.57 0.54
Nitrogen mineralisation
FQQ Factorrepresentingtheinfluenceof soilmoistureonnitrogen mineralisation
0–1 0.39 0.55 0.54 0.53
TA 7-daymovingaverageairtemperature; usedtoestimatetheinfluenceof temperatureonnitrogen mineralisation
−2–19.8 5.58 0.49 0.48 0.48
Leafexpansion
GAKILR Factorusedtorepresenttheeffectof droughtonthecanopysenescence
0–23.4 5.25 0.23 0.27 0.29
DRFACLAI Factorusedtorepresenttheeffectof droughtoncanopyexpansion
−0.438–1 1.0 0.15 0.20 0.26
Vernalisation
POTLFNO Potentialleafnumber,usedinthe calculationofvernalisationeffecton cropdevelopment
0–23.94 9.77 0.44 0.46 0.47
PRIMORDNO Primordianumber,usedinthe calculationofvernalisationeffecton cropdevelopment
Table3(Continued)
Modelvariable Function Fullmodel
range
Replacement constant
Replacementprobability
Temperatureadjustments ˛=2.50% ˛=5% ˛=10%
SVERNALISATION Switchtoremovetheinfluenceof vernalisationoncropdevelopment
n/a 0 0.76 0.71 0.68
Soilsurfaceevaporation
ALPHA Factorrepresentingtheeffectof canopyshadingonsoilsurface evaporation
1–1.35 1.34 0.54 0.52 0.51
PTSOIL Intermediatevariableusedinthe calculationofsoilsurfaceevaporation.
0.014–7.68 0.33 0.58 0.57 0.56
SLOSL Intermediatevariableusedinthe calculationofsoilsurfaceevaporation.
0–409.7 8.36 0.43 0.43 0.43
Penman
EW Intermediatevariableusedinthe calculationofvapourpressuredeficit, inturnusedforthecalculationof evaporation
4.46–28.8 14.879 0.39 0.40 0.42
HSLOPtmean Intermediatevariableusedinthe calculationofPriestly–Taylor evaporation
0.34–1.69 0.66942 0.45 0.45 0.44
WND Windspeed,usedinthecalculationof Penmanevaporation(from
Priestly–Taylorevaporation)
0–9.3 6.8452 0.21 0.26 0.31
minimumform ofthe Siriusmodel whose overall performance couldbecomparedtothefullmodel.Drymatterforgrainfilling was derived from a combination of photosynthesis during the grainfillingperiodandtranslocationofstemandleafbiomass.The modelrecordedbiomassatanthesis(BIOANTH)andthiswasused todefinetherateatwhichstemandleafbiomasscouldbe translo-cated to the filling grain. In effect translocation potential was relatedtobiomassatanthesis.Thereduction analysissuggested biomassatanthesis(BIOANTH)isredundantandthattranslocation potentialcouldbeaconstantacrosssitesandtreatments.Inthe potentialand droughttreatments(wherethedatashowedlittle differenceinbiomass atthetime ofanthesis) theeffectof this replacementwastoreduce thecontributionof translocationto grainyield. Inthecase ofNlimitedtreatments,where anthesis biomasswasrelativelylow,theeffectwastoincreasetherelative contributionoftranslocationtograinyield.
Themodelallowedfortheexpansionofthecanopytobereduced underwaterstressthroughthevariableDrFACLAIand similarly therateofcanopysenescenceincreasedthroughthevariable GAK-ILR.Thesevariablesgenerallyhadlowreplacementprobabilities implyingthattheycontributedtothemodel’spredictiveskill.
Themodelusedseveralinterestingtemperatureadjustments toaccountfor thedifferencesbetweenairtemperatureandsoil
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 2000 4000 6000 8000 10000
Re
p
la
ce
m
e
n
t P
ro
b
ab
ilit
y
Iter
aon
Root Length S-SoilMin S-CTemp S-Vernalisaon
Fig.4.Evolutionofthereplacementprobabilityforaselectionofthecandidate variablesoverthecourseoftheanalysis(variablesymbolsaredefinedinTable3).
minimumandmaximumandcanopytemperatures.Threemodel variables(ENAV,HCROP,TADJ)relatedtotheseadjustmentswere identifiedasredundantandasoutlinedearliertheiroverall impor-tancewasassessedthroughtheinclusionofthreeswitchvariables (SwSOILMAX,SwSOILMIN,andSwCMAX)whichhadtheeffectof replacingeachofthesetemperatureswiththeappropriateair tem-perature.Allthreewerefoundtoberedundant.Anotherinteresting featureofSiriusisthatthediurnalvariationintemperatureisused toestimatetheratesof progressofavarietyofplantprocesses ratherthansimplyusingthemeantemperature.Thisfeaturewas foundtoberedundantwithreplacementprobabilitiesof approxi-mately0.5.
Siriususedplantnitrogenstatustoinfluencenitrogenuptake and to drivenitrogentranslocation betweenthestem and leaf (JamiesonandSemenov, 2000).ThevariableMINSTEMDEMAND wascalculatedforeachdayandrepresentedtheminimumnitrogen demandwhichmustbesupportedifgrowthwastooccur.Ifthis nitrogenwasnotavailablefromuptakeitwasobtainedthrough translocationofnitrogenfromtheleaftothestem. MAXSTEMDE-MANDprovidedanupperlimitoncropnitrogenuptakeincases wherestemnitrogenwashigh,causingnitrogenuptaketocease.
218 N.M.J.Croutetal./AgriculturalandForestMeteorology189–190(2014)211–219
RelatedtothesevariablesLEAFDEMANDcalculatesthenitrogen requiredfortheexpectedleafexpansionandusedthistocalculate appropriatetransportof nitrogenbetweenleafand stemifthat wasrequiredtosustainleafexpansion.MAXSTEMDEMANDwas redundantintheanalysisandMINSTEMDEMANDwasborderline redundant. Given the values of the replacement constants the effectofthesewastoremovetheinfluenceofplantnitrogenstatus onnitrogenuptake,thecropsimplyremovedwhatevernitrogen was available to it. The replacement probabilities for LEAFDE-MANDvariedbetweenthelikelihoodmethodsbutoverallwereall <0.4.Thereforetheminimummodel simplifiednitrogenuptake, ignoringtheeffectofplantnitrogenstatusonuptake,butretaining theuseofleafdemandtodriveinternalnitrogenallocation.These variablesdonotrelatetothelinkbetweencropnitrogenstatus andgrowthwhichremainedunchangedintheminimummodel.
Twovariablesassociatedwiththepredictionofnitrogen min-eralisation were identified as redundant. These related to the influenceofsoilmoisture(FQQ)andtemperature(TA)ontherate ofmineralisation.Bothwerereplacedatthelowendoftheirrange withtheeffectofreducingnitrogenmineralisationtoalow con-stantvalue.
TherepresentationofcropdevelopmentinSiriuscombinedthe effectof temperature,including theeffect oflow temperatures (vernalisation),and daylength throughsimulation ofthe crop’s leafnumber.The approachis fullydescribed byJamiesonet al. (1998a)andisonlybrieflysummarisedhere.Leavesareproduced ataconstantthermaltimeinterval(phyllochron)whichisa cul-tivarspecificparameter.Duringthecourseofthegrowingseason themodelsetsafinalleafnumberdependingonthevernalisation anddaylengthexperiencedbythegrowingcropaccordingtothe parametersdefinedforthecultivar.Anthesisoccurredthree phyl-lochronsafterthepointwhenthecropsimulatedleafnumberis equaltothefinalleafnumber.Forcultivarswithavernalisation requirementapotentialfinalleafnumberwascalculatedinthe ver-nalisationprocedure.Ifthecultivarwasdaylengthsensitivethis potentialfinalleafnumberwasfurthermodifiedbyadaylength functiontosetthefinalleafnumber.
Twovariablesrelatedtothevernalisationsubmodel(POTLFNO and PRIMORDNO) were found to be redundant, moreover the switchvariablethatturnsoffvernalisationentirelyhada replace-ment values of c. 0.7 indicating that model predictions were improvedwhenthisvariableisreplaced.
Severalvariables related tosoil surface evaporation(EVsoil) wereredundant(ALPHA,CONDUC,PTSOIL,SLOSL)suchthatEVsoil iseffectivelyreplacedbyaconstantlowvalueof0.32mmday−1.
However ignoring soil surface evaporation completely (i.e. EVsoil=0)hadadetrimentaleffectonmodelperformance,withthe implicationthatalthoughthemodelmaybeover-estimating evap-oration,itdidneedtobeconsidered.Inthemodelsoilevaporation influencesboththecalculationofsoilmaximumtemperatureand thesoilwaterbudget.Replacingsoilevaporationwithaconstant continuedtohaveanadvantageformodelperformanceevenwhen soilmaximumtemperaturewassetequaltothemaximumair tem-peratureimplyingthatthechangestowaterbudgetarebeneficial tomodelperformance.
Inadditiontothereplacementprobabilitiesforindividual vari-ables shown in Table 3 joint probabilities were calculated to indicatewhethertherewerecaseswherethereplacementofone variablewasdependentonwhetheranothervariablewas,orwas not replaced.These results (notpresented) showedno notable interactionsfortheredundantvariables.
Theanalysisdescribedispartialastheobservationaldataused do notprovide a testfor allaspects of themodel,nor dothey represent a fully comprehensive range of site conditions. The interpretationoftheresultsoftheanalysisneedstoreflectthese limitationsifusefulinsightsintothemodeldesignaretobegained.
For example, although setting translocation potentialtoa con-stantimprovedmodelpredictionsinouranalysisthereplacement wouldbeproblematicifmodelledanthesisbiomasswaslowerthan theproposedconstantforsettingthepotentialfortranslocation (9258gm−2),asmightbethecaseunderextremestress.This
find-ingmayimplythattranslocationpotentialisnotlinearlyrelatedto biomassandratherthansimplysetthevariabletoaconstantitmay bemoreproductivetoconsiderthisfeatureofthecropsbehaviour morecarefullyinfuturemodeldevelopment.
Similarargumentsapplytotheredundancyofvariablesrelated tovernalisation.Wearenotsuggestingthatthereisnosuch pro-cessasvernalisation,ratherthatthemodelledmodificationofleaf numbertoaccountforvernalisationgivesaworseresultthanthat obtainedbyignoringtheprocessinthemodel.Thismayprovide anindicationthattherepresentationoftheprocessinthemodel isinappropriate.Howevercautionisrequired,thefindingswere dictatedbytheresponseofthemodeltotherangeofconditions experiencedoverthe3UKsitesasNewZealandtrialsusedaspring wheatcultivar.
Redundancyinthevariablesrelatedtonitrogenmineralisation alsoillustratestheeffectofpartialdataontheanalysis.These vari-ableshadlittleeffectonmodelperformanceasthecropsnitrogen supply washighrelative totherateof nitrogenmineralisation. Thisistrueeveninthetreatmentwithnonitrogenfertiliser addi-tionswherethenitrogensupplyisdominatedbytheresidualsoil inorganicnitrogenatthetimeofplanting.
3.4. Minimummodel
OnthebasisoftheaboveanalysisavariantofSiriuswas devel-opedinwhichalltheidentifiedredundantmodelvariableswere reducedwiththeaimofcomparingtheresultantpredictionswith thoseofthefullmodel.Thedifferenceswerethat(a)canopyandsoil temperaturesweretakenasequaltotheairtemperature;(b)there wasnoallowancefordiurnalvariationintemperatureon develop-mentorotherprocesses;(c)nitrogenuptakeissimplifiedsuchthat theplantsimplyremovesallnitrogenavailabletoitineachtime step;(d)theeffectofvernalisationisignored,withdevelopment beingdrivensolelybytemperatureandphotoperiod;(e)soil nitro-genmineralisationandsoilsurfaceevaporationwereignored;(f) translocationpotentialwasconsideredaconstant.
Theresultingmodelpredictionsarecomparedtothefullmodel inFig.1andthesummaryNash–Sutcliffestatisticsarepresented inTable2.Notwithstandingthesesimplificationsofthemodelthe performancewasalmostidenticaltothefullmodel,withslightly improvedperformanceforleafareaandgrainweights.
3.5. Conclusions
Inpreviousapplicationsofthevariablereplacementapproach tomodelreductionallthemodelsinvestigatedwerefoundtohave redundantvariables(Croutetal.,2009;Gibbonsetal.,2010;Cox etal.,2006;Tarsitanoetal.,2011).SimilarlySiriuswasfoundto con-tainvariableswhoseusewasredundantforpredictingthedatawe haveusedinthisanalysis.However,mostofthemodel’svariables couldnotbereducedtoaconstant;ofthe111variablesconsidered 16wereultimatelyidentifiedaspotentiallyredundant.
it indicates that the model’s predictive performance was not improvedthroughtheirrepresentationinthemodel.
Theoutcomesoftheworkwehavedescribeddependedonour choiceofcomparisondata.Inourcasethiswaswithinseason mea-surementsofmultiplecomponentsofthecropover arelatively smallnumberof trials.Wefocussedonchallengingthe mecha-nismswithinthemodelatarelativelydetailedlevelinorderto evaluatewhichofthemodelledprocessesarecontributingtothe overallpredictionofgrowthanddevelopmentoverthegrowing season.Thereforetheapproachisanalogoustothetypeofdetailed modelinter-comparisondescribedbyJamiesonetal.(1998b). How-everourworkcouldbedescribedasamodelintra-comparisonasit wasbasedonthecomparisonofmanysimplifiedformsofthesame model.Theapproachprovidesautomationtoincreasetheefficiency oftheevaluationandisasystematicmeansofincreasingtherigour oftheevaluation.Howeverthereis,asyet,nowaytoavoidtheneed formechanisticmodelunderstandingandinterpretationifmodel performanceistobecriticallyevaluated.
Theanalysisisdependentontheobservationaldataused. Sub-jecttothis limitationit providesatest of whethera particular formulationofmodelvariablescontributestothemodels predic-tiveperformance.Theaimshouldnotbetosimplyfindasimpler modelanduseit,buttousetheidentificationofredundant vari-ablesasameanstochallengeandimprovetheformulationusedin themodel.
Acknowledgment
WewouldliketothanktheBiotechnologyandBiological Sci-encesResearchCouncilforfinanciallysupportingthiswork(grant referenceBBS/B/05672).Rothamsted Researchreceivesstrategic fundingfromtheBiotechnologyandBiologicalSciencesResearch CounciloftheUK.
References
Anderson,T.R.,2005.Planktonfunctionaltypemodelling:runningbeforewecan walk?J.PlanktonRes.27,1073–1081.
Bernhardt,K.,2008.Findingalternativesandreducedformulationsfor process-basedmodels.Evol.Comput.16,63–88.
Beven,K.,2009.EnvironmentalModelling:AnUncertainFuture.Routledge,London.
Brisson,N.,Gary,C.,Justes,E.,Roche,R.,Mary,B.,Ripoche,D.,Zimmer,D.,Sierra, J.,Bertuzzi,P.,Burger,P.,Bussiere,F.,Cabidoche,Y.M.,Cellier,P.,Debaeke,P., Gaudillere,J.P.,Henault,C.,Maraux,F.,Seguin,B.,Sinoquet,H.,2003.Anoverview ofthecropmodelSTICS.Eur.J.Agron.18,309–332.
Cox,G.M.,Gibbons,J.M.,Wood,A.T.A.,Craigon,J.,Ramsden,S.J.,Crout,N.M.J.,2006.
Towardsthesystematicsimplificationofmechanisticmodels.Ecol.Model.198, 240–246.
Crout,N.M.J.,Tarsitano,D.,Wood,A.T.A.,2009.Ismymodeltoocomplex?Evaluating modelformulationusingmodelreduction.Environ.Model.Software24,1–7.
Ewert,F.,Rodriguez,D.,Jamieson,P.,Semenov,M.A.,Mitchell,R.A.C.,Goudriann, J.,Porter,J.R.,Kimball,B.A.,Pinter,P.J.,Manderscheid,R.,Weigal,H.J., Famg-meier,A.,Fereres,E.,Villalobos,F.,2002.EffectsofelevatedCO2anddroughton
wheat:testingcropsimulationmodelsfordifferentexperimentalandclimatic conditions.Agric.Ecosyst.Environ.93,249–266.
Gibbons,J.M.,Wood,A.T.A.,Craigon,J.,Ramsden,S.J.,Crout,N.M.J.,2010. Semi-automatic reduction and upscaling oflarge models: afarm management example.Ecol.Model.221,590–598.
Gillett,A.G.,Crout,N.M.J.,Stokes,D.T.,Sylvester-Bradley,R.,Scott,R.K.,1999.Simple winterwheatgreenareaindexmodelunderUKconditions.J.Agric.Sci.132, 263–271.
Hastings,W.K.,1970.MonteCarlosamplingmethodsusingMarkovChainsandtheir applications.Biometrika57(1),97–109.
Jakeman,A.J.,Letcher,R.A.,Norton,J.P.,2006.Teniterativestepsindevelopmentand evaluationofenvironmentalmodels.Environ.Model.Software21,602–614.
Jamieson,P.D.,Brooking,I.R., Semenov,M.A.,Porter,J.R.,1998a.Makingsense of wheat development: a critique of methodology. Field Crops Res. 55, 117–127.
Jamieson,P.D.,Semenov,M.A.,Brooking,I.R.,Francis,G.S.,1998b.Sirius:a mech-anisticmodelofwheatresponsetoenvironmentalvariation.Eur.J.Agron.8, 161–179.
Jamieson,P.D.,Semenov,M.A.,2000.Modellingnitrogenuptakeandredistribution inwheat.FieldCropRes.68,21–29.
Jones,J.W.,Hoogenboom,G.,Porter,C.H.,Boote,K.J.,Batchelor,W.D.,Hunt,L.A., Wilkens,P.W.,Singh,U.,Gijsman,A.J.,Ritchie,J.T.,2003.DSSATcroppingsystem model.Eur.J.Agron.18,235–265.
Kass,R.E.,Rafferty,A.E.,1995.Bayesfactors.J.Am.Stat.Assoc.90,773–795.
Keating,B.A.,Carberry,P.S.,Hammer,G.L.,Probert,M.E.,Robertson,M.J.,Holzworth, D.,Huth,N.I.,Hargreaves,J.N.G.,Meinke,H.,Hochman,Z.,McLean,G.,Verburg, K.,Snow,V.,Dimes,J.P.,Silburn,M.,Wang,E.,Brown,S.,Bristow,K.L.,Asseng, S.,Chapman,S.,McCown,R.L.,Freebairn,D.M.,Smith,C.J.,2003.Anoverview ofAPSIM,amodeldesignedforfarmingsystemssimulation.Eur.J.Agron.18, 267–288.
Kimmins,J.P.,Blanco,J.A.,Seely,B.,Welham,C.,Scoullar,K.,2008.Complexityin modellingforestecosystems:howmuchisenough?For.Ecol.Manage.256, 16461658.
Lawless,C.,Semenov,M.A.,Jamieson,P.D.,2005.Awheatcanopymodellinkingleaf areaandphenology.Eur.J.Agron.22(1),19–32.
Nash,J.E.,Sutcliffe,J.V.,1970.Riverflowforecastingthroughconceptualmodels. Part1eadiscussionofprinciples.J.Hydrol.27,282–290.
SemenovM.A.,ShewryP.R.,2011.ModellingPredictsthatHeatStress,NotDrought, willIncreaseVulnerabilityofWheatinEurope.ScientificReports1,Article66, NaturePublishingGroup,London.doi:10.1038/srep00066Itisavailableonline at:http://www.nature.com/srep/2011/110818/srep00066/full/srep00066.html
Semenov,M.A.,Martre,P.,Jamieson,P.D.,2009.Quantifyingeffectsofsimplewheat traitsonyieldinwater-limitedenvironmentsusingamodellingapproach.Agric. For.Meteorol.149,1095–1104.
Smith, J.T.,Clarke,R.T.,Bowes,M.J.,2010. Aregroundwaternitrate concentra-tionsreaching aturning point in someaquifers? Sci. TotalEnviron. 408, 4722–4732.
Tarsitano,D.,Young,S.D.,Crout,N.M.J.,2011.Evaluatingandreducingamodelof radiocaesiumsoil-plantuptake.J.Environ.Radioactiv.102,262–269.
Wolf,J.,Evans,L.G.,Semenov,M.A.,Eckersten,H.,Iglesias,A.,1996.Comparison ofwheatsimulationmodelsunderclimatechange.1.Modelcalibrationand sensitivityanalyses.Clim.Res.7,253–270.
