ContentslistsavailableatScienceDirect
European
Journal
of
Agronomy
jo u r n al hom e p a g e :w w w . e l s e v i e r . c o m / l o c a t e / e j a
Quantifying
model-structure-
and
parameter-driven
uncertainties
in
spring
wheat
phenology
prediction
with
Bayesian
analysis
Phillip
D.
Alderman
a,b,∗,
Bryan
Stanfill
c,daDepartmentofPlantandSoilSciences,371AgriculturalHall,Stillwater,OK,USA
bInternationalMaizeandWheatImprovementCenter,Apdo.Postal6-641,06600Mexico,D.F.,Mexico cPacificNorthwestNationalLaboratory,P.O.Box999,Richland,WA99352,USA
dCommonwealthScientificandIndustrialResearchOrganisation,41BoggoRoad,DuttonPark,QLD,Australia
a
r
t
i
c
l
e
i
n
f
o
Articlehistory:Received1September2015 Receivedinrevisedform14June2016 Accepted21September2016 Availableonlinexxx Keywords:
Bayesianparameterestimation Predictionuncertainty Cropmodeling
Agriculturalsystemsmodeling Wheatphenology
a
b
s
t
r
a
c
t
Recentinternationaleffortshavebroughtrenewedemphasisonthecomparisonofdifferentagricultural systemsmodels.Thusfar,analysisofmodel-ensemblesimulatedresultshasnotclearlydifferentiated betweenensemblepredictionuncertaintiesduetomodelstructuraldifferencesperseandthosedue toparametervalueuncertainties.Additionally,despiteincreasinguseofBayesianparameterestimation approacheswithfield-scalecropmodels,inadequateattentionhasbeengiventothefullposterior distri-butionsforestimatedparameters.Theobjectivesofthisstudyweretoquantifytheimpactofparameter valueuncertaintyonpredictionuncertaintyformodelingspringwheatphenologyusingBayesiananalysis andtoassesstherelativecontributionsofmodel-structure-drivenandparameter-value-driven uncer-taintytooverallpredictionuncertainty.ThisstudyusedarandomwalkMetropolisalgorithmtoestimate parametersfor30springwheatgenotypesusingninephenologymodelsbasedonmulti-locationtrial datafordaystoheadinganddaystomaturity.Acrossallcases,parameter-drivenuncertaintyaccounted forbetween19and52%ofpredictiveuncertainty,whilemodel-structure-drivenuncertaintyaccounted forbetween12and64%.Thisstudydemonstratedtheimportanceofquantifyingboth model-structure-andparameter-value-drivenuncertaintywhenassessingoverall predictionuncertaintyinmodeling springwheatphenology.Moregenerally,Bayesianparameterestimationprovidedausefulframework forquantifyingandanalyzingsourcesofpredictionuncertainty.
©2016TheAuthor(s).PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCC BY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
Recent international efforts, such as the Agricultural Model IntercomparisonandImprovementProject(AgMIP;http://www. agmip.org;Rosenzweigetal.,2013)andModellingEuropean Agri-culturewithClimateChangeforFoodSecurity(MACSUR;http:// www.macsur.eu),have broughtrenewed emphasisonthe com-parisonofdifferentagriculturalsystemsmodels.Inparticular,the AgMIP wheatmodelingteam hasproduced a numberof recent publications documentingthe utilityof multi-modelensembles asameansofimprovingclimateimpactassessmentsforwheat (Alderman etal., 2013;Assenget al.,2013,2015;Martre etal., 2015).Aswithpreviousefforts(e.g.Jamiesonetal.,1998;Porter etal.,1993),theserecentmodelcomparisonshavestronglyfocused
∗ Correspondingauthorat:DepartmentofPlantandSoilSciences,371Agricultural Hall,Stillwater,OK,USA.
E-mailaddress:[email protected](P.D.Alderman).
onmodelstructure.Nevertheless,theanalysisofsimulatedresults ofmodelensembleshasnotyetbeenabletodifferentiateprediction uncertaintiesinsimulationsduetodifferencesinmodelstructure perseandthoseduetoparametervalueuncertainty,despite spe-cificsimulationprotocols(Assengetal.,2013;Martreetal.,2015). Furthermore,evenifamodel’sstructureisperfectlyspecified(i.e. itisthecorrectmodel),itisnotguaranteedtopredictaccurately iftheuncertaintiesaboutparametervaluesarenotaccountedfor (Wallach,2011).
Whiletheimportanceofselectingappropriateparametersfor dynamic cropmodelshaslong beenacknowledged,approaches to setting parameter values have ranged from manual calibra-tion(Booteetal.,2002;Whiteetal.,2008;Jamiesonetal.,2007) toautomated,objectiveapproaches(Aldermanetal.,2015;Hunt et al., 1993; Zheng et al., 2013). Previously, the goal of these approacheshasbeentodetermineasingleacceptablevalueforeach parameterwhichwillpermitcorrespondencebetweensimulated resultsandmeasureddata.Recently,theimportanceofquantifying theuncertaintiesassociatedwithparameterestimateshasgained http://dx.doi.org/10.1016/j.eja.2016.09.016
1161-0301/©2016TheAuthor(s).PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/ 4.0/).
recognition(Confalonierietal.,2009;Wallach,2011),whichhasled togreaterinterestinBayesianparameterestimationapproaches thatpermitsuchquantification,e.g.GeneralizedLikelihood Uncer-taintyEstimation(GLUE)andMarkovChainMonteCarlo(MCMC) methods(Heetal.,2010;Makowskietal.,2002;Archontoulisetal., 2014;Iizumietal.,2009;Dzotsi etal.,2015).However,theuse ofsuchmethodswithfield-scalecropmodelshasnotresultedin aconcomitantemphasisontheimplicationsofsuchuncertainty. Indeed,theendgoalofparameterestimationinthiscontexthas largelyremainedunchanged,namely,todetermineasingle accept-ablevalueforeachparameter.Thishasledtoanover-emphasison parameterposteriormeanswithoutanaccompanyinganalysisof thefullposteriordistribution.TobefullyconsistentwithBayesian theory,theposteriordistributionsforparametersshouldgiverise todistributionsinsimulatedoutput.Thatis,theuncertaintiesone hasaboutparametervalues shouldbereflectedin uncertainties inthe simulatedoutputbased onthose parametervalues. Fur-thermore,when beingapplied toout-of-sampleprediction, e.g. climatechangeimpactassessment,thecontributionofparameter valueuncertaintiesonpredictionuncertaintiesmustbeadequately accountedfor(Wallach,2011).
Thus, frameworks are needed which can combine analysis ofparameter-value-andmodel-structure-drivenuncertaintyand relatetheseuncertaintiestoout-of-sampleprediction uncertain-ties.Theobjectivesofthisstudyweretoquantifytheimpactof parametervalueuncertaintyonpredictionuncertaintyfor mod-eling spring wheat phenology using Bayesian analysis and to assess therelative contributions of model-structure-driven and parameter-value-drivenuncertaintytooverallprediction uncer-tainty.
2. Methods 2.1. Datasets
Thisstudymadeuseoftwospringwheatphenologydatasets. Thefirstdatasetcamefromamulti-locationtrialestablishedacross 27locationswithin14countriesincollaborationwiththe Interna-tionalMaizeandWheatImprovementCenter(CIMMYT)through the International Wheat Improvement Network (Chavez et al., 2013).Sixtyelitebreedinglinesandhistoricalspringwheat geno-typesfromwithintheCIMMYTbreedingprogram,knownasthe CIMMYTCore Germplasm(CIMCOG) collection(Reynoldset al., 2011),weregrownundernon-yield-limitingnutrient,water,and pestmanagement overtheyears 2010and 2011(Chavezetal., 2013).Dataondaystoheadinganddaystomaturitywerescreened foranomalies(e.g.unusuallyhighvariationwithingenotypeacross replications)andmissingvalues,andasubsetof13locationswere identifiedforuseinthisstudy.Thesubsetoflocationsincludedsites across8countriesandranginginlatitudefrom34.58◦Sto43.77◦N. The second wheat phenology dataset includedheading and maturitydatacollectedonasubsetof30CIMCOGgenotypesover athree-yearperiodfrom2010to2013attheCIMMYTNormanE. BorlaugExperimentStationnearCiudadObregón,Sonora,Mexico (Moleroetal.,2015).Twosowingdateswereusedtocreate con-trasting thermal conditions:one temperate (Decembersowing) andonehot(Februarysowing).Otherthansupra-optimal tempera-turestress,fieldsweremanagedtoensurenon-limitingconditions. Forthisstudy,onlydatafromthe30genotypescommontoboth datasetswereusedthusresultinginacombineddatasetof30 geno-typesacross19site×year×sowingdatecombinations.
Dailymaximumandminimumtemperaturedataforsimulating themulti-locationdatasetwereextractedfromtheNASA/POWER database(Stackhouse,2015)ateachlocation.Thesedatahave esti-matedbiasesof−1.83and0.24◦Cformonthly-averagedmaximum
and minimum temperature, respectively. For the CIMMYT-Mexicodataset,theprimarytemperaturedataweremeasuredon station. Gaps in these data were first filled using temperature data fromthe nearby Agroson weather station (http://agroson. org.mx) and remaining gaps were filled from NASA/POWER data.
2.2. Wheatphenologymodels
Threewheatphenologymodelsweredevelopedforthisstudy basedonthephenologymodulesof24wheatmodelsparticipating inAgMIP (Aldermanet al.,2013;Asseng etal., 2015). Underly-ingtheAgMIPmodelsandthethreedevelopedforthisstudyis theconceptofdailyaccumulationofdevelopmentunitsasa func-tionoftemperatureandphotoperiod.Untilsimulatedheadingdate, thedevelopmentunitswerecalculatedonadailytimestepasthe productofthermaltimeaccumulatedonagivenday(TT)anda pho-toperiodfactor(Fp)whichaccountedfortheeffectofphotoperiod
ontherateofdevelopment.Afterheadingdate,thephotoperiod factorwasexcludedfromcalculations, resultingina purely TT-driven simulation between headingand maturity. Because this studyfocusedonspring wheat(whichhasgenerallysmall ver-nalization response; Ottman et al., 2013; Zheng et al., 2013), vernalizationresponsewasnotmodeled.Thenumberofdaysto headingormaturityweresimulatedbyaccumulatingdevelopment unitsuntilgenotype-specificdevelopment-unitrequirementswere satisfied.Oncethedevelopment-unitrequirementsforphasesfrom emergencetoheading(tth)orfromheadingtomaturity(tthm)were met,thedaystoheadingormaturity werecalculated.A recent collectionof modeldocumentation(Alderman et al.,2013)was reviewedanditwasfoundthatthedifferencesbetweenthe phen-ologymodulesformostAgMIPwheatmodelsliesintheshapeof thefunctionusedtocalculatedailythermaltime.
2.2.1. Thermaltime
For this study, we selected three thermal time functions, namely,atriangular-shapedpiece-wiselinearfunction (triangu-lar),atrapezoidal-shapedpiece-wiselinearfunction(trapezoidal), and a non-linearfunction (Wang–Engel).Thetriangular-shaped piece-wiselinearfunctionwasgivenby:
TT=
⎧
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎩
Topt Tavg−Tbase Topt−Tbaseif Tbase<Tavg<Topt
Topt
Tmax−Tavg
Tmax−Topt
if Topt≤Tavg≤Tmax
0 otherwise
(1)
whereTTisdailythermaltime,Tavgisthemidwaypointbetween
dailymaximum and minimumtemperature, Topt isthe optimal
temperatureatwhichphenologicaldevelopmentrateisatits max-imum,Tbaseisthebasetemperaturebelowwhichnodevelopment
occurs,and Tmax is themaximumtemperature abovewhich no
development occurs. Similarly, thetrapezoidal-shaped function wasgivenby:
TT=
⎧
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎩
0 if Tavg≤Tbase Topt Tavg−Tbase Topt−Tbaseif Tbase<Tavg<Topt
Topt if Tavg≥Topt
0 10 20 0 10 20 30 40 Temperature
(
°C)
De v elopment Units Trapezoidal Triangular Wang−EngelFig.1.Wheatdevelopmentunitsascalculatedacrossarangeoftemperaturesusing atriangular-shapedpiecewiselinearfunction(triangular;seeEq.(1)),a trapezoidal-shapedpiecewiselinearfunction(trapezoidal;seeEq.(2)),andanon-linearfunction (Wang–Engel;seeEq.(3))assumingabasetemperatureof0,anoptimaltemperature of26,andamaximumtemperatureof34.
whereTT,Tavg,Topt,Tbaseareasdefinedpreviously.Thenon-linear
function used in this study was the Wang–Engel temperature model(WangandEngel,1998)andisgivenas:
TT=
⎧
⎪
⎨
⎪
⎩
Topt2(Tavg−Tbase)˛(Topt−Tbase)˛−(Tavg−Tbase)2˛
(Topt−Tbase)2˛
if Tbase≤Tavg≤Tmax
0 otherwise
(3)
whereTT,Tavg,Topt,Tbaseareasdefinedpreviouslyand˛isa
param-eterderivedby:
˛= ln(2) ln
Tmax−Tbase Topt−Tbase.
Avisualrepresentationofthethreethermaltimefunctionsacross atemperaturerangeof0–40◦CisprovidedinFig.1andisbasedon aTbaseof0,aToptof26,andaTmaxof34.Notethatthetrapezoidal
functionoverlapsexactlywiththetriangularfunctionbelowTopt.
AboveTopt,thetrapezoidalfunctioncalculatesthesamethermal
timeasatToptdespiteanincreaseintemperature,whilethe
ther-maltimecalculatedwiththetriangularfunctiondeclineslinearly as thetemperature approaches Tmax.The Wang–Engel function
calculateslowerthermaltimethantheothertwofunctionsat tem-peraturesbelowapproximately20◦C,whereasbetween20◦Cand Topt theWang–Engel functioncalculates slightlyhigher thermal
time.BetweenToptandTmax,theWang–Engelfunctioncalculates
higherthermaltimethanthetriangularfunctionandlowerthermal timethanthetrapezoidalfunction.
2.2.2. Photoperiod
For all photoperiod response models,the daily photoperiod (P;daylength pluscivil twilight)wasapproximatedby a setof
equationsusedtheintheDSSAT-CSM(Hoogenboometal.,2013), namely: Ls=sin(0.01745Lat) Lc=cos(0.01745Lat) dec=0.4093sin(0.0172[Dyr−82.2]) dlv=max
−0.87,−Lssin(dec)−0.1047 Lccos(dec) )
P=7.639arccos(dlv)
where Lat is the latitude of the simulated location in decimal degrees,andDyristhesimulateddayofyear.Thisphotoperiodwas
thenusedtocalculateaphotoperiodfactorusingoneofthree pho-toperiodresponsemodels.Thefirstisaquadratic functionused withintheDSSAT-CSM-CERESmodel(Hoogenboometal.,2013) andisgivenas:
Fp=
⎧
⎪
⎪
⎨
⎪
⎪
⎩
0 if ppsen 100 (20−P)2 (20−ppmin)2 >1 1−ppsen100 (20−P) 2 (20−ppmin)2 if ppsen 100 (20−P)2 (20−ppmin)2 <1 (4)whereFpisthedailyphotoperiodfactor,ppsenisaphotoperiod
sen-sitivityparameter,andPisthedailyphotoperiodasdefinedabove. Thesecondphotoperiodresponsemodelisapiece-wiselinear func-tionadaptedfromthemodeldescribedbyChewetal.(2012)and isgivenas: Fp=
⎧
⎪
⎪
⎨
⎪
⎪
⎩
sddev if P<ppminsddev+(P−ppmin)(1−sddev)
ppmax−ppmin if ppmin<P<ppmax
1 if P>ppmax
(5)
where Fp, P,and ppmin are as defined previously,sddev is the
minimum development rate, and ppmax is the photoperiod at which developmentrateisatitsmaximum.Thethird photope-riodresponsemodelisanasymptoticexponentialfunctionadapted fromthemodeldescribedbyHasegawaetal.(2008)andisgiven by: Fp=
⎧
⎨
⎩
0 P<ppmin 1−eppslope(ppmin−P) if P≥ppmin (6) whereFp,P,andppminareasdefinedpreviouslyandppslopeisa parameter controlling theslope of thephotoperiod response curve.
Fig.2visuallyillustratestheshapeofeachofthephotoperiod responsefunctionsoverarangeofphotoperiods.Allthreefunctions saturateatamaximumvalueof1.TheCERESandHasegawa func-tionshaveaminimumvalueof0belowthephotoperioddefinedby ppmin.ThevalueoftheChewfunctionbelowppminisdetermined bysddev,whichissetto0.25forFig.2.Theshapeofresponsefor theCERESmodelisdeterminedbyppsen,highervaluesofwhich resultinasharperincreaseaboveppmin.FortheHasegawa func-tion,theparameterppslopecontrolstheshapeofthecurve.Low valuesofppsloperesultinashapethatroughlyapproximatesthe shapeoftheCERESfunction.Highvaluesofppsloperesultinasharp
0.00 0.25 0.50 0.75 1.00 0 4 8 12 16 20 24 Photoperiod Fp CERES Chew Hasegawa
Fig.2. Wheatphotoperiodfactor(FP)ascalculatedacrossarangeofphotoperiods
usingaquadraticfunctionfromtheDSSAT-CSM-CERES-Wheatmodel(CERES;see Eq.(4)),apiecewiselinearfunction(Chew;seeEq.(5)),andanasymptotic expo-nentialfunction(Hasegawa;seeEq.(6))assumingparametervaluesofppmin=10, ppsen=100,sddev=0.25,ppslope=1,andppmax=15.
increaseaboveppmin.TheshapeoftheChewfunctionis deter-minedbytheproximityoftheparametersppminandppmax.When thetwoparametersareclosertogether,theincreaseaboveppmin issharp.Whenthetwoparametersarefurtherapart,theresponse tophotoperiodismoregradual.
2.2.3. Modelparametersestimated
Severalparameterscontrollingresponsetophotoperiod(ppsen, ppmin,sddev,ppmax,andppslope),optimumcardinaltemperature (Topt),andthenumberofdevelopmentunitsfromemergenceto
heading(tth)andfromheadingtomaturity(tthm)wereestimated foreachofthe30genotypes.Minimum(Tbase)andmaximum(Tmax)
cardinaltemperatureparametersforallthreetemperature mod-elswereassumed to befixedat 0 and 34◦C, values consistent witha reviewofpreviousresearchonwheatcardinal tempera-tures(Porterand Gawith,1999).While someuncertaintyexists astothevaluesfor Tbase andTmax,initialattempts atincluding
theseparameterswereunsuccessfulduetothelimited tempera-turerangerepresentedinthedataset.Consequently,theanalysis focusedontheotherparameters,forwhichsufficientinformation wasavailable.Fromapracticalstandpoint,cardinaltemperature parametersarelessfrequentlyestimatedincropmodelsthanthe otherparameters affectingphenological development,thus,the resultsoftheanalysisshouldstillhavesufficientlybroad applica-bility.
2.3. Bayesiananalysis
ToemployBayesiananalysis,amodelfortheparametersprior toincorporatingdata(i.e.aparameterpriordistribution)mustbe specifiedaswellasamodelforthedata(i.e.adatalikelihood).The priordistributionforeachoftheparametersisgiveninTable1.The positivenormaldistributionisatruncatednormaldistributionthat givesprobabilityzerotoanyvaluelessthan0.Thetruncatednormal distributionisanormaldistributionthatgivesprobabilityofzero toanyvalueoutsideofagivenupperorlowerbound.Thesepriors
werechosenbasedonrecentresearchonmodelingspringwheat phenology(Ottmanetal.,2013;Whiteetal.,2011;Zhengetal., 2013;PorterandGawith,1999).Abriefsensitivityanalysisrevealed thatthechoiceofpriordistributiondidnothaveanoticeableeffect upontheresults(resultsomitted).
Foreachoftheninemodels(3thermaltimefunctions×3 pho-toperiodfunctions), we assumed theobserved days toheading valueswereindependentandnormallydistributedaboutthemodel predictionfordaystoheadingwithanunknown,model-specific variance.Thedatameanandvariancewasassumedtovaryforeach modelandgenotypebutnotlocation.Othervariancestructures, suchaslocationspecificvariances,wereconsideredbutultimately discardedbecausenoevidenceofarelationshipbetweenthe loca-tionanduncertaintyinmodelpredictionswasevident.
Letl=1,...,njindicatetheobservationnumberforgenotype
j=1,...,30,wherenjisthetotalnumberofgenotypej
observa-tionsacrosslocationsandi∈{1,...,9}indicatesoneofthenine modelsresultingfromapair-wise combinationofthermal time functions(triangular,trapezoidalandWang–Engel)and photope-riodresponse functions(CERES,Chew, andHasegawa).Thenlet yjl representthelthobserved daystoheadingfor genonotypej
andfi(ij)istheestimateddaystoheadinggivenparameter
vec-torij=(ttoptij,tthij,thmij,...)usingphenologymodelfi.Thenwe
assume
yjl∼N(fi(ij),2h,ij)
where 2
h,ij is the days to headingresidual variance for model
i, genotype j.Therefore thelikelihood of theparameter vector ˚ij=[ij,h,ij]giventhedatavectory=
y11,...,y30,n30 isgiven by L(˚ij|y)= 30 j=1 nj l=1 1 22 h,ij exp −yjl−fi(ij) 2 22 h,ij .Thedaystomaturitylikelihoodcanbewrittensimilarly.An ini-tialinspectionofthedatasupportedtheassumptionthatdaysto headingandmaturityareindependentandnormallydistributed, butfurtherexaminationofthebivariatealternativeisofinterest.
Togetsamplesfromtheposteriordistributionsofthe parame-tersgiventhedata,arandomwalkMetropolisalgorithmwasused. TherandomwalkMetropolisalgorithmcanbesummarizedas fol-lows.Let˚ij bethevectorofparametersofinterestformodeli,
genotypejandKbethetotalnumberofdrawstobeselected.
1Randomlyselectinitialvalues˚0
ijfromtheassumedprior
distri-bution.
2Fortimek=1,...,K
(a)Proposeacandidateparametervector˚∗ijfromthesymmetric
transitionkernel g(˚k−1
ij ,)where is amatrixof tuning
parametersandcovariancesusedtocontrolthedispersionof thetransitionkernel.
(b) Giventhedatavectory,calculatetheacceptanceratio r= (˚ ∗ ij|y) (˚k−1 ij |y) = P(˚ ∗ ij)L(˚∗ij|y) P(˚k−1 ij )L(˚ k−1 ij |y)
whereP(˚ij)isthepriordistributionandL(˚ij|y)isthe
like-lihoodoftheparametervector˚ijgiventhedatavectory.
(c) RandomlygenerateavalueUfromtheuniformdistribution ontherange[0,1].
(d)Ifr>Uthenset˚k
ij=˚∗ij,otherwiseset˚kij=˚k−1ij .
Forthisapplicationwechosethemultivariatenormal distribu-tionforthetransitionkernelg(·,·)andthetuningparameters werechosensuchthattheacceptanceratewasbetween20and30%
Table1
Priordistributionsassumedfortheparametersofinterest.
Parameter(abbrev.) Priordistribution Priorparameters
a b
Photoperiodsensitivity(ppsen) Positivenormal(,) 60 20
Minimumphotoperiod(ppmin) Truncatednormal(,,a,b) 10 1 0 24
Maximumphotoperiod(ppmax) Truncatednormal(,,a,b) 15 1 0 24
Slopeofphotoperiodresponse(ppslope) Positivenormal(,) 0.28 2.72
Short-daydevelopmentrate(sddev) Truncatednormal(,,a,b) 0.4 0.4 0 1
Optimalcardinaltemperature(topt) Normal(,) 22.7 2.4
Developmentunitstoheading(tth) Positivenormal(,) 800 200
Developmentunitsfromheadingtomaturity(tthm) Positivenormal(,) 800 200
Residualstandarddeviation−daystoheading(h) Positivenormal(,) 21 10
Residualstandarddeviation−daystomaturity(m) Positivenormal(,) 20 10
(Robertsetal.,1997).Typically,thefirstseveralparametervectors generatedinthisfashionarediscardedasburn-in.Theresultant chainswerecheckedforconvergence;seeGelmanetal.(2014)for moreMCMCdiagnostics.
Toassessthepredictiveaccuracyofeachmodel,aleave-one-out cross-validationapproachwasused.Theposteriordistributionof themodelparameterswasestimatedusingallofthedataexcept foroneyear-by-sowingdatecombination.Theaveragephenology outcomesfortheomittedlocationarethenpredictedusingthe esti-mateposteriordistribution.Thedistributionofthepredicteddays toheadingandmaturitythatresultsrepresentsthepredictedmean responseforthatyear-by-sowingdate.
Forexample,theMetropolisalgorithmdetailedinSection2was usedtogetdrawsfromtheposteriordistributionforthe photope-riodsensitivity, thermal timeand variance parametersfor each modelandgenotypeatallsitesexcepttheDecembersowingin Obregón,Mexicoduring2013.Eachdrawfromtheposterior distri-bution, ˆk
ij,wasusedtopredicttheaveragedaystoheadingforthe
DecembersowingatObregónin2013,fi(ˆkij).Thepredictedmean
daystoheadingwasthencomparedtotheobservedaveragedays toheadingforthatsamelocationgenotypeduringthesametime period,whichisgivenby
Ll=1yjl/L.Wecomparedpredictedandobservedmeansbecausethecropmodelsunderinvestigationhere arenotintendedtopredictsingleobservations,rathertheaverage responseforagivengenotype,locationandsowing.Thefollowing discussiononpredictionaccuracyisreferringtothepredictionof anaverageresponseasjustdescribed.
Themeansquareerrorofprediction(MSEP)isusedtoquantify predictionuncertainty.MSEPcanbedecomposedintotwoparts: squaredbiasandpredictionvarianceorpredictionuncertainty.The squaredbias accountsforthe predictiveaccuracy (thedistance betweenpredictionand truth)whiletheprediction uncertainty quantifiesthepredictionprecision(thespreadinpredictions).
Recallthatyjlrepresentsthelthobserveddaystoheadingfor
genotypejandletfi(ˆijk)representthepredictedmeandaysto
head-ingforgenotypejusingmodeliandparametervectork.Thesquared biasforeachyearandsowingdateisestimatedby
1 30 30
j=1⎡
⎣
1 L L l=1 yjl− 1 IK I i=1 K k=1 fi(ˆijk) 2⎤
⎦
. (7)The prediction uncertainty can be further decomposed into sources:genotype,modelformulationandparameters.Itis possi-blefortheuncertaintyassociatedwithgenotypetobedifferentfor eachmodel,thereforetheinteractionbetweenmodelandgenotype isalsoincluded.Becausedifferentparametervectorsareestimated foreachmodel andgenotypecombination,parameters are con-sideredanestedvariableanddonotinteractwithgenotypesor models.
UsingnotationfromWallachetal.(2016),theprediction uncer-taintycanbedecomposedas
Var(ˆyk
ij)=f2+X2+Xf2 +2 (8)
where2
Xisthevarianceduetodifferencesingenotype,f2isthe
varianceduetomodelformulation,2
Xf istheinteractionbetween
genotypesandmodelformulationand2
isthevariance dueto
parameteruncertainty.
ThevariancecomponentsinEq.(8)areestimatedusinga ran-domeffectsmodelwheretheresponseistheparametervectors drawnfromtheposteriordistributionsforthedaystoheading mod-elsandthecovariatesareindicatorvariablesforeachgenotype, modelformulationandtheproductofthoseindicatorvariables.To illustrate,considerthetrianglemodelfortheDecembersowingat Obregónin2013.Foreachgenotypeandmodel,25,000parameter vectorsaredrawnfromthejointposteriordistributionforij
result-ingin25,000predictionsfordaystoheadinganddaystomaturity foreachgenotype.Sincethereareninemodels,30different geno-typesand25,000parametervectors,eachvariancecomponentin (8)isestimable.Becausewearetreatingthepredictionsas mea-suredwithouterror,theparameteruncertainty2
isestimatedby
whatiscommonlyreferredtoastheresidualvariance.
All simulationsand calculations werecompleted within the Rstatisticalsoftwareenvironment(RCoreTeam, 2015).Figures weregeneratedusingtheggplot2(Wickham,2009)andstandard graphics packages. Two-dimensionalkernal density estimates were computed using the MASS package (Venables and Ripley, 2002).Datatablesweregeneratedusingthextablepackage(Dahl, 2016).
3. Resultsanddiscussion
Foreachmodelandgenotype,theposteriordistributionforthe parameterswasestimatedusingtheMetropolisalgorithmdetailed inSection2.Parameterinferenceisbasedonthreeindependent chainsoflength25,000afteraninitialburn-inof25,000eachwith randomstartingvalues.Inallcasestheindependentchainsreached thesameposteriordistributionaccordingtovisualinspectionofthe traceplotsandcalculationofthepotentialscalereductionstatistic (GelmanandRubin,1992).
3.1. Posteriorpredictionaccuracy
Thedrawsfromtheparameterposteriordistributionswerefed intothephenologymodelstopredicttheheadingandmaturity datesfortheyear-sowingdatecombinationthatwasomittedfrom theparameterestimationprocess.Theoutputfromthese simula-tionswasusedtocalculateaveragesquaredposteriorpredictive biasesforeachyearandsowingdateusingEq.(7),whichare pre-sentedinTable2.Acrossallmodels,daystoheadingwasgenerally
Table2
Estimatesofmeansquaredbiasforsimulateddaystoheading(DTH)anddaystomaturity(DTM)forwheatgrownatCiudadObregón,Mexicofornormal(Temperate)and hightemperature(Hot)sowingdatesoverthreeyearsforpairwisecombinationsoffunctionsforthermaltime(TTFunction)andphotoperiodresponse(PPDFunction).The thermaltimefunctionsincludedatrapezoidalpiecewiselinearfunction(Trap.;SeeEq.(2)),atriangularpiecewiselinearfunction(Tri.;SeeEq.(1)),andanon-linearfunction (Wang;SeeEq.(3)).PhotoperiodresponsefunctionsincludedaquadraticfunctionfromtheDSSAT-CSM-CERES-Wheatmodel(CERES;seeEq.(4)),apiecewiselinearfunction (Chew;seeEq.(5)),andanasymptoticexponentialfunction(Hasegawa;seeEq.(6)).
Resp. TTFunction PPDFunction Temperate Hot
2011 2012 2013 2011 2012 2013 DTH Trap. CERES 21.72 72.72 247.95 3.25 48.44 3.97 Chew 22.44 78.20 249.23 2.86 46.40 21.03 Hasegawa 6.05 22.93 181.07 17.71 18.84 14.46 Tri. CERES 55.10 138.26 309.58 23.76 12.97 5.90 Chew 56.72 141.04 310.11 21.43 14.39 6.26 Hasegawa 42.46 114.25 293.76 33.75 8.63 16.02 Wang CERES 53.88 166.60 315.25 9.37 21.32 2.70 Chew 56.68 170.15 318.37 7.29 25.81 2.69 Hasegawa 35.89 120.69 285.56 7.50 26.19 2.17 DTM Trap. CERES 6.20 9.75 19.77 24.27 4.06 10.81 Chew 6.75 8.58 20.18 22.87 4.19 23.41 Hasegawa 3.62 44.39 9.71 91.74 28.21 85.10 Tri. CERES 11.79 5.83 16.61 9.15 16.15 8.77 Chew 13.26 5.78 16.62 7.85 18.63 10.29 Hasegawa 7.17 14.48 12.51 18.74 14.70 10.00 Wang CERES 3.49 9.43 11.28 9.21 29.72 25.98 Chew 3.67 8.95 10.34 11.14 34.36 29.86 Hasegawa 7.19 27.92 10.13 14.45 35.63 25.84
predictedbetterforthehightemperaturesowingsthanforthe
tem-peratesowings.Theonlyexceptiontothatgeneraltrendwasthe
combinedTrapezoidal-Hasegawamodel,whichpredictedthe
tem-perate2011daystoheadingbetterthananyotheryear-sowingdate
combination.Thismodelcombinationalsohadthelowestaverage
squaredbiasacrossallyearsandsowingdatesfordaystoheading.
Fordaystomaturity,modelsgenerallyperformedbetteroverall.
Inthiscase,thecombinedTriangular-CERESmodelhadthelowest
averagesquaredbias.Unlikedaystoheading,predictionofdaysto
maturitywasgenerallybetterforthetemperatesowingthanforthe
hightemperaturesowing.Theseresultssuggestadegreeof
com-pensationbetweentheaccuracyofpredictionsfordaystoheading
anddaystomaturity.
Withinthetemperatesowings,theTrapezoidalfunction
per-formedbestoverallfordaystoheading,followedbytheTriangular,
withtheWang-Engelfunctionlast.Alsowithindaystoheading
forthetemperatesowings,theHasegawafunctionperformedbest
withtheCERESfunctionperformingapproximatelyequaltothe
Chewfunction.Withinthehightemperaturesowings,theeffectof
photoperiodfunctionwasdiminishedsubstantiallywiththeCERES
functionnarrowlyperformingbetterthantheothertwo.Forthe
thermaltimefunctions,theWang-Engelfunctionperformedbest
followedbytheTriangularfunction,withtheTrapezoidalfunction
last.Fordaystomaturitywithinthetemperatesowings,the
Wang-EngelfunctionperformedbestfollowedcloselybytheTriangular
functionwhiletheTrapezoidalfunctionperformedleastwell.For
thehightemperaturesowings,theTriangularfunctionpredicted
daystomaturitybestwiththeWang-EngelandTrapezoidal
func-tionsperformingsecondandthird,respectively.
Giventhelongerphotoperiodforthehightemperature
sow-ings, it is not surprising that differences between photoperiod
functionswouldbediminishedfordaystoheadingsinceallthree
photoperiodfunctionsproducesimilarphotoperiodfactorvalues
forlongphotoperiods.Thus,theperformanceofthesefunctionsat
thetemperatesowingsmightbemoreindicativeoftheirpredictive
capacity.Similarly,thehightemperaturesowingmaygiveabetter
indicationoftheperformanceofthethermaltimefunctionsdueto
theincreasedrangeintemperaturesoverwhichthefunctionswere
beingapplied.Whenconsideredacrossdaystoheadinganddaysto
maturity,theTriangularfunctionseemedtoperformbestoverall
forthermaltime.AlthoughtheHasegawafunctionperformedbest
fordaystoheadingfortemperatesowingdates,nophotoperiod
functionwasconsistentlybetterforthehightemperaturesowing
dates.Furthermore,theconsistentlypoorpredictiveaccuracyfor
daystoheadingacrossallmodelsfortemperatesowingdatesin
2012and2013pointstoaneedforimprovingthesemodels.
Prelim-inaryanalysisofweatherdatasuggeststhattheparticularlypoor
predictionofdaystoheadinginthetemperatesowingof2013may
bedue tolowerlight intensity,aneffectwhichispresentlynot
includedinanyofthemodelsusedinthisstudyorinwheat
phen-ologymodelinggenerally(Aldermanetal.,2013;Harrisonetal.,
2012;Jamiesonetal.,2007;Ottmanetal.,2013;Zhengetal.,2013).
3.2. Posteriorparametervalueuncertainty
Theparameterposteriordistributionsfordevelopmentunitsto heading(tth)forgenotype775areplottedinFig.3alongwiththe priordistributionasspecifiedinTable1.Similarresultswereseen fortheotherparametersandgenotypesandarethereforeomitted. Overall,theeffectofpriordistributionchoiceontheseresultswas minimalaccordingtotheresultsofaBayesiansensitivityanalysis (resultsomitted).Comparedtothepriordistributions,the poste-riordistributionsforeachparametershowadecreaseduncertainty (narrowingofthedistribution)andalocationshift.
Itisimportanttonotethatseveralparametersshowed corre-lations.Mostnotably,tthwascorrelatedwithsomeparameters controllingtheshapeofthephotoperiodresponse(ppsen,ppmin, ppmax,andppslope).Thus,theapparentuncertaintyshowninFig.3 mayoverestimate theactualuncertainty.Thatis, for anygiven valueofppsen,ppmin,ppmax,orppslopetheuncertaintyabouttth wasnarrowerthanindicatedbythemarginaldensityestimates. Fig.4illustratesthisconcept.Fig.4Ashowsthemarginaldensity plotoftheposteriordrawsfor estimatingtthfor GID775across thefulldatasetusingtheTrapezoidalfunctionforthermal time andtheChewfunctionforphotoperiodresponse.Fig.4Bshows athree-dimensionalplotoftwo-dimensionalkerneldensity esti-matesfortheparameterstthandppmax.ComparingFig.4AandB, thepeakindicatingthejointposteriormodeismorepronouncedin thetwo-dimensionaldensityplotthaninthemarginaldensityplot. Further,whenFig.4Bisrotatedabouttheverticalaxissuchthatthe viewpointislookingalongthelineofcorrelationbetweentthand ppmax(Fig.4C),onecanseethatthedistributionoftthwithina
Fig.3.Priorandposteriordistributionsforthedevelopmentunitstoheading(tth)forspringwheatgenotype775forcombinationsoftheTriangular-shaped(Triangular), Trapezoidal-shaped(Trapezoidal),andWang–Engel(WangEngel)functionsforthermaltimeandthequadratic(CERES),piece-wiselinear(Chew),andasymptoticexponential (Hasegawa)functionsforphotoperiodresponseusingallyearsandsowingdates.
Fig.4.Plotsofone-(A)andtwo-dimensional(BandC)posteriorkerneldensityestimatesforthetthandppmaxparametersusingtheTrapezoidal(Eq.(2))functionfor thermaltimeandtheChew(Eq.(5))functionforphotoperiodresponse.PanelCisaplotofthesamedatafromPanelBrotated45◦abouttheverticalaxis.
givenvalueofppmaxismuchnarrowerthanthemarginaldensity showninFig.4A.Thus,cautionshouldbetakenininterpretingthe resultsofBayesianparameterestimationwhenappliedto mod-elsthathavestronginteractionsbetweenparameters.Giventhe strongfeedbacksandinteractionsbetweenparametersandstate variablestypicalofmanydynamiccropmodelsanddynamics sys-temsmodelsgenerally,summarizingtheposteriordistributionin termsofthemarginalposteriormeanandstandarddeviationfor eachparameterwouldlikelygivebiasedparameterestimates.In
thiscontext,carefulexaminationofthejointposteriorparameter distributionisessentialforensuringaccurateandreliableresults.
3.3. Posteriorpredictionuncertainty
Table3showsthedecompositionofthemeansquarederrorof prediction(MSEP)intothesquaredbiasandthecomponentsof posteriorpredictionuncertainty.Fordaystoheading,thesquared biasaccounted forthemajority ofMSEP whenaveraged across
Table3
Estimatesofsquaredbias(Sq.Bias)andcomponentsofthepredictionuncertaintyduetomodel(M),genotype(G),model×genotypeinteraction(M×G),andparameter values(P).TheitalicizedsquaredbiasvaluesindicatecaseswherethesquaredbiasaccountsforamajorityoftheMSEP.Theboldvaluesarethelargestsourcesofprediction uncertainty.
Resp. Sowing Year Sq.Bias M(2
f) G( 2 X) M×G(fX2) P( 2 ) DTH Temp. 2011 34.90 4.33 9.76 0.31 3.80 2012 107.12 7.28 10.58 0.36 4.27 2013 276.65 2.23 9.58 0.40 5.32 Hot 2011 10.99 2.87 8.62 0.60 6.67 2012 22.35 2.20 9.54 0.56 6.69 2013 3.86 2.77 7.64 2.31 6.80 DTM Temp. 2011 3.64 3.23 4.57 0.59 6.54 2012 11.75 3.00 4.57 0.66 7.92 2013 11.38 2.31 3.15 0.75 6.63 Hot 2011 9.99 13.49 3.23 1.47 7.63 2012 8.55 12.33 3.22 1.18 6.97 2013 5.67 20.54 2.29 1.91 7.28
allmodels for allthree years of the temperate sowing and for
2012ofthehightemperaturesowing.Withintheposterior
predic-tionuncertainty,thelargestcomponentfordaystoheadingacross
yearsandsowingswasgenotype.Thisresultindicatesthatthere
weredifferencesbetweengenotypesinthepredictionuncertainty
andthegenerallysmallvalues ofthegenotype×model
interac-tionindicatesthatthiseffectwasconsistentacrossmodels.When
comparingmodel-structure-andparameter-drivenuncertaintyfor
daystoheading,theparameter-drivenuncertainty(P)washigher
forthetemperatesowingin2013andforallthreeyearsofthehigh
temperaturesowing.However,forthetemperatesowingin2011
and2012model-structure-drivenuncertaintywashigher,though
onlymarginallyin 2011.For daystomaturity, posterior
predic-tionuncertaintyaccountedforthemajorityofMSEPforallyears
andsowings(Table3).Parameter-drivenuncertaintyaccountedfor
thehighestproportionofpredictionuncertaintyforallyearsofthe temperatesowing,whilemodel-structure-drivenuncertaintywas higherforthehightemperaturesowing.Thelowergenotype com-ponent(G)fordaystomaturity,ascomparedtodaystoheading, indicatesthattherewaslessvariationbetweengenotypesinthe predictionuncertaintyfordaystomaturity.
Overallfordaystoheading,theparameter-drivenuncertainty accounted for between19 and 36% of the posterior prediction uncertaintywhilethemodel-structure-drivenuncertaintyranged from12to32%ofthetotalposteriorpredictionuncertainty. Simi-larly,parameter-drivenuncertaintywasbetween23and52%ofthe predictionuncertaintyfordaystomaturity,whilemodel-structure drivenuncertainty ranged from18 to64%. Thus, for both days toheadinganddaystomaturity,thecontributionof parameter-andmodel-structure-drivenuncertaintyeach contributed signif-icantlytopredictionuncertaintydependingonyearandsowing date. That differences in model-structure would contribute to uncertaintyinmodelensemblepredictionsisintuitiveand,thus, hasbeenthefocusofpreviouseffortsinmodel-intercomparisons (Jamiesonetal.,1998,2007;Porteretal.,1993).However,these findingshighlighttheimportanceofalsoconsideringparameter valueuncertainty whenworkingwithcropmodelsapoint sup-portedbyotherrecentresearch(Confalonierietal.,2009;Wallach, 2011).
4. Summaryandconclusions
Thisstudydemonstrated theimportanceofquantifying both model-structure-andparameter-value-drivenuncertainty when assessingoverallpredictionuncertaintyinmodelingspringwheat phenology.Eithersourceof uncertaintycouldrepresent alarge portion(upto52or64%)oftotalpredictionuncertainty depend-ingonthepredictedvariableandyearofanalysis.Thisstudyalso
showedthe limitedability of current wheatphenology models topredictacrossawiderangeofconditionsandhighlightedthe needforcontinuedmodelimprovement.Moregenerally,wehave demonstratedthatBayesianparameterestimationcanprovidea usefulframeworkforquantifyingandanalyzingsourcesof predic-tionuncertainty.However,caremustbetakenwhenusingsuch methodswithcropmodelswhichhavestronginteractionsbetween parameters.AppropriateapplicationofBayesianparameter esti-mationinthesecasesrequiresthatsuchcorrelationsbeaccounted forboth intheestimationprocessand intheanalysisof poste-riordistributions.Althoughtheparticularcaseanalyzedherewas limitedtospring wheatphenology models,inprinciple,similar effortscouldbeundertakenwithmorecomplexmodels.Indeed, furtherworkwithmorecomplexmodelsshouldbepursuedto con-firmtheresultsofthisstudy.Apotentiallyfruitfulextensionofthis workwouldbetoundertakeasimilarstudyanalyzingwinterwheat phenologywheremorephenologicalstagesanddifferentmodels forvernalizationcouldbeincluded.
Acknowledgements
PhillipD.AldermanwasfundedbytheCGIARResearchProgram onClimateChange,AgricultureandFoodSecurity(CCAFS)andby theNationalScienceFoundationunder GrantNo. OIA-1301789. BryanStanfillistherecipientofaJohnStockerPostdoctoral Fellow-shipfromtheScienceandIndustryResearchFund.Thecomputing forthisprojectwasperformedattheOSUHighPerformance Com-puting Center at Oklahoma State University supported in part throughtheNationalScienceFoundationgrantOCI1126330.
References
Alderman,P.D.,Quilligan,E.,Asseng,S.,Ewert,F.,Reynolds,M.(Eds.),2013. ProceedingsoftheWorkshoponModelingWheatResponsetoHigh Temperature.ElBatan,Texcoco,Mexico;19–21June2013,Mexico,DF,Mexico, June2013,CIMMYThttp://repository.cimmyt.org/xmlui/handle/10883/3211. Alderman,P.D.,Boote,K.J.,Jones,J.W.,Bhatia,V.S.,2015.Adaptingthe
CSM-CROPGROmodelforpigeonpeausingsequentialparameterestimation. FieldCropsRes.181,1–15.
Archontoulis,S.V.,Miguez,F.E.,Moore,K.J.,2014.Amethodologyandan optimizationtooltocalibratephenologyofshort-dayspeciesincludedinthe APSIMPLANTmodel.Applicationtosoybean.Environ.Model.Softw.62, 465–477.
Asseng,S.,Ewert,F.,Rosenzweig,C.,Jones,J.W.,Hatfield,J.L.,Ruane,A.C.,Boote, K.J.,Thorburn,P.J.,Rötter,R.P.,Cammarano,D.,Brisson,N.,Basso,B.,Martre,P., Aggarwal,P.K.,Angulo,C.,Bertuzzi,P.,Biernath,C.,Challinor,A.J.,Doltra,J., Gayler,S.,Goldberg,R.,Grant,R.,Heng,L.,Hooker,J.,Hunt,L.A.,Ingwersen,J., Izaurralde,R.C.,Kersebaum,K.C.,Müller,C.,Kumar,S.N.,Nendel,C.,O’Leary,G., Olesen,J.E.,Osborne,T.M.,Palosuo,T.,Priesack,E.,Ripoche,D.,Semenov,M.A., Shcherbak,I.,Steduto,P.,Stöckle,C.,Stratonovitch,P.,Streck,T.,Supit,I.,Tao, F.,Travasso,M.,Waha,K.,Wallach,D.,White,J.W.,Williams,J.R.,Wolf,J.,2013. Uncertaintyinsimulatingwheatyieldsunderclimatechange.Nat.Clim. Change3,827–832,http://dx.doi.org/10.1038/nclimate1916.
Asseng,S.,Ewert,F.,Martre,P.,Rötter,R.P.,Lobell,D.B.,Cammarano,D.,Kimball, B.A.,Ottman,M.J.,Wall,G.W.,White,J.W.,Reynolds,M.P.,Alderman,P.D., Prasad,P.V.V.,Aggarwal,P.K.,Anothai,J.,Basso,B.,Biernath,C.,Challinor,A.J., DeSanctis,G.,Doltra,J.,Fereres,E.,Garcia-Vila,M.,Gayler,S.,Hoogenboom,G., Hunt,L.A.,Izaurralde,R.C.,Jabloun,M.,Jones,C.D.,Kersebaum,K.C.,Koehler, A.-K.,Müller,C.,Kumar,S.N.,Nendel,C.,O’Leary,G.,Olesen,J.E.,Palosuo,T., Priesack,E.,Rezaei,E.E.,Ruane,A.C.,Semenov,M.A.,Shcherbak,I.,Stöckle,C., Stratonovitch,P.,Streck,T.,Supit,I.,Tao,F.,Thorburn,P.,Waha,K.,Wang,E., Wallach,D.,Wolf,J.,Zhao,Z.,Zhu,Y.,2015.Risingtemperaturesreduceglobal wheatproduction.Nat.Clim.Change5(2),143–147.
Boote,K.J.,Mínguez,M.I.,Sau,F.,2002.AdaptingtheCROPGROlegumemodelto simulategrowthoffababean.Agron.J.94(4),743–756.
Chavez,P.,Moya,E.S.,Molero,G.,Reynolds,M.,2013.CIMCOG:Mexicoand InternationalDataSummary.CIMMYT,Mexico,DF,Mexico,pp.63–69http:// repository.cimmyt.org/xmlui/bitstream/handle/10883/3210/98023.pdf. Chew,Y.H.,Wilczek,A.M.,Williams,M.,Welch,S.M.,Schmitt,J.,Halliday,K.J.,2012.
AnaugmentedArabidopsisphenologymodelrevealsseasonaltemperature controloffloweringtime.NewPhytol.194(3),654–665.
Confalonieri,R.,Acutis,M.,Bellocchi,G.,Donatelli,M.,2009.Multi-metric evaluationofthemodelsWARM,CropSyst,andWOFOSTforrice.Ecol.Model. 220(11),1395–1410.
Dahl,D.B.,2016.xtable:ExportTablestoLaTeXorHTML.RPackageVersion1.8-2. https://CRAN.R-project.org/package=xtable.
Dzotsi,K.,Basso,B.,Jones,J.,2015.Parameteranduncertaintyestimationfor maize,peanutandcottonusingtheSALUScropmodel.Agric.Syst.135,31–47. Gelman,A.,Rubin,D.B.,1992.Inferencefromiterativesimulationusingmultiple
sequences.Stat.Sci.7(4),457–472.
Gelman,A.,Carlin,J.B.,Stern,H.S.,Rubin,D.B.,2014.BayesianDataAnalysis.Taylor &Francis.
Harrison,M.T.,Evans,J.R.,Moore,A.D.,2012.Usingamathematicalframeworkto examinephysiologicalchangesinwinterwheatafterlivestockgrazing:1. Modelderivationandcoefficientcalibration.FieldCropsRes.136,116–126. Hasegawa,T.,Sawano,S.,Goto,S.,Konghakote,P.,Polthanee,A.,Ishigooka,Y.,
Kuwagata,T.,Toritani,H.,Furuya,J.,2008.Amodeldrivenbycropwateruse andnitrogensupplyforsimulatingchangesintheregionalyieldofrain-fed lowlandriceinnortheastThailand.PaddyWaterEnviron.6(1),73–82. He,J.,Jones,J.W.,Graham,W.D.,Dukes,M.D.,2010.Influenceoflikelihoodfunction
choiceforestimatingcropmodelparametersusingthegeneralizedlikelihood uncertaintyestimationmethod.Agric.Syst.103(5),256–264.
Hoogenboom,G.,Jones,J.W.,Wilkens,P.W.,Porter,C.H.,Boote,K.J.,Hunt,L.A., Singh,U.,Lizaso,J.L.,White,J.W.,Uryasev,O.,Royce,F.S.,Ogoshi,R.,Gijsman, A.J.,Tsuji,G.Y.,Koo,J.,2013.DecisionSupportSystemforAgrotechnology Transfer(DSSAT)Version4.6.0.016.
Hunt,L.,Pararajasingham,S.,Jones,J.,Hoogenboom,G.,Imamura,D.,Ogoshi,R., 1993.Gencalc:softwaretofacilitatetheuseofcropmodelsforanalyzingfield experiments.Agron.J.85(5),1090–1094.
Iizumi,T.,Yokozawa,M.,Nishimori,M.,2009.Parameterestimationand uncertaintyanalysisofalarge-scalecropmodelforpaddyrice:applicationofa Bayesianapproach.Agric.ForestMeteorol.149(2),333–348.
Jamieson,P.,Porter,J.,Goudriaan,J.,Ritchie,J.,VanKeulen,H.,Stol,W.,1998.A comparisonofthemodelsAFRCWHEAT2,CERES-Wheat,Sirius,SUCROS2and SWHE,withmeasurementsfromwheatgrownunderdrought.FieldCropsRes. 55(1),23–44.
Jamieson,P.,Brooking,I.,Semenov,M.,McMaster,G.,White,J.,Porter,J.R.,2007. Reconcilingalternativemodelsofphenologicaldevelopmentinwinterwheat. FieldCropsRes.103(1),36–41.
Makowski,D.,Wallach,D.,Tremblay,M.,2002.UsingaBayesianapproachto parameterestimation;comparisonoftheGLUEandMCMCmethods. Agronomie22(2),191–203.
Martre,P.,Wallach,D.,Asseng,S.,Ewert,F.,Jones,J.W.,Rötter,R.P.,Boote,K.J., Ruane,A.C.,Thorburn,P.J.,Cammarano,D.,etal.,2015.Multimodelensembles ofwheatgrowth:manymodelsarebetterthanone.Glob.ChangeBiol.21(2), 911–925.
Molero,G.,Rievera-Amado,A.C.,Pinera-Chavez,F.J.,Trujillo-Negrellos,E., Quintero,A.,Gonzalez,O.E.,Slafer,G.A.,Calderini,D.F.,Foulkes,M.J.,Berry, P.M.,Griffiths,S.,neda,C.L.-C.,Singh,R.,Reynolds,M.P.,2015March.Dissection ofYieldPotentialRelatedTraits:WhatShallWeFocusOn?CIMMYT,Mexico, DF,Mexico,pp.43–46http://repository.cimmyt.org/xmlui/bitstream/handle/ 10883/4244/99501.pdf.
Ottman,M.J.,AnthonyHunt,L.,White,J.W.,2013.Photoperiodandvernalization effectonanthesisdateinwinter-sownspringwheatregions.Agron.J.105(4), 1017–1025.
Porter,J.,Jamieson,P.,Wilson,D.,1993.Comparisonofthewheatsimulation modelsAFRCWHEAT2,CERES-WheatandSWHE,fornon-limitingconditionsof cropgrowth.FieldCropsRes.33(1),131–157.
Porter,J.R.,Gawith,M.,1999.Temperaturesandthegrowthanddevelopmentof wheat:areview.Eur.J.Agron.10(1),23–36.
RCoreTeam,2015.R:ALanguageandEnvironmentforStatisticalComputing.R FoundationforStatisticalComputing,Vienna,Austriahttps://www.R-project. org/.
Reynolds,M.,Huerta,J.,Payne,T.,2011March.Sp3.4:GermplasmEvaluationand Delivery–EvaluationofCIMCOGasanInternationalNursery.CIMMYT, Mexico,DF,Mexico,pp.37–38http://repository.cimmyt.org/xmlui/bitstream/ handle/10883/1276/94761.pdf.
Roberts,G.O.,Gelman,A.,Gilks,W.R.,1997.Weakconvergenceandoptimalscaling ofrandomwalkmetropolisalgorithms.Ann.Appl.Probab.7(1),110–120. Rosenzweig,C.,Jones,J.W.,Hatfield,J.L.,Ruane,A.C.,Boote,K.J.,Thorburn,P.,Antle,
J.M.,Nelson,G.C.,Porter,C.,Janssen,S.,Asseng,S.,Basso,B.,Ewert,F.,Wallach, D.,Baigorria,G.,Winter,J.M.,2013.TheAgriculturalModelIntercomparison andImprovementProject(AgMIP):protocolsandpilotstudies.Agric.For. Meteorol.170,166–182,http://dx.doi.org/10.1016/j.agrformet.2012.09.011. Stackhouse,P.W.,2015.PredictionofWorldwideEnergyResources.NASA,
Washington,DChttp://power.larc.nasa.gov/.
Venables,W.N.,Ripley,B.D.,2002.ModernAppliedStatisticswithS,4thed. Springer,NewYork,ISBN0-387-95457-0,http://www.stats.ox.ac.uk/pub/ MASS4.
Wallach,D.,2011.Cropmodelcalibration:astatisticalperspective.Agron.J.103 (4),1144–1151.
Wallach,D.,Thorburn,P.,Asseng,S.,Challinor,A.J.,Ewert,F.,Jones,J.W.,Rötter,R., Ruane,A.,2016.Estimatingmodelpredictionerror:Shouldyoutreat predictionsasfixedorrandom?Environ.Model.Softw.84,529–539. Wang,E.,Engel,T.,1998.Simulationofphenologicaldevelopmentofwheatcrops.
Agric.Syst.58(1),1–24.
White,J.W.,Herndl,M.,Hunt,L.,Payne,T.S.,Hoogenboom,G.,2008.
Simulation-basedanalysisofeffectsofandlocionfloweringinwheat.CropSci. 48(2),678–687.
White,J.W.,Kimball,B.A.,Wall,G.W.,Ottman,M.J.,Hunt,L.,2011.Responsesof timeofanthesisandmaturitytosowingdatesandinfraredwarminginspring wheat.FieldCropsRes.124(2),213–222.
Wickham,H.,2009.ggplot2:ElegantGraphicsforDataAnalysis.Springer,New York,ISBN978-0-387-98140-6,http://had.co.nz/ggplot2/book.
Zheng,B.,Biddulph,B.,Li,D.,Kuchel,H.,Chapman,S.,2013.Quantificationofthe effectsofVRN1andPpd-D1topredictspringwheat(Triticumaestivum) headingtimeacrossdiverseenvironments.J.Exp.Bot.64(12),3747–3761.