Performance of the modified TRISS for evaluating trauma care in subpopulations: A cohort study

Performance

of

the

modi

fi

ed

TRISS

for

evaluating

trauma

care

in

subpopulations:

A

cohort

study

Leonie

de

Munter

*,

Suzanne

Polinder

,

Daan

Nieboer

,

Koen

W.W.

Lansink

,

Ewout

W.

Steyerberg

,

Mariska

A.C.

de

Jongh

a_{DepartmentTraumaTopCare,Elisabeth-TweeStedenHospital,Tilburg,TheNetherlands} b

DepartmentofPublicHealth,ErasmusMedicalCentre,Rotterdam,TheNetherlands

c

BrabantTraumaRegistry,NetworkEmergencyCareBrabant,Tilburg,TheNetherlands

d

DepartmentofSurgery,Elisabeth-TweeStedenHospital,Tilburg,TheNetherlands

e

DepartmentofBiomedicalDataSciences,LeidenUniversityMedicalCenter,Leiden,TheNetherlands

ARTICLE INFO Articlehistory: Accepted29March2018 Keywords: Predictionmodel Performance TRISS Trauma Subpopulations ABSTRACT

Introduction:Previousresearchshowedthatthereisnoagreementonapracticallyapplicablemodelto

useintheevaluationoftraumacare.AmodificationoftheTraumaandInjurySeverityScore(modified

TRISS)isusedtoevaluatetraumacareintheNetherlands.Theaimofthisstudywastoevaluatethe

prognosticabilityofthemodifiedTRISSandtodeterminewherethismodelneedsimprovementfor

bettersurvivalpredictions.

Methods:PatientswereincludediftheywereregisteredintheBrabantTraumaRegistryfrom2010

through2015.Missingvalueswereimputedaccordingtomultipleimputation.Subsetswerecreated

basedonage,lengthofstay,typeofinjuryandinjuryseverity.Probabilityofsurvivalwascalculatedwith

themodifiedTRISS.DiscriminationwasassessedwiththeAreaUndertheReceiverOperatingCurve

(AUROC).Calibrationwasstudiedgraphically.

Results:TheAUROCwas0.84(95%CI:0.83,0.85)forthetotalcohort(N=69747)butonly0.53(95%CI:

0.51,0.56)forelderlypatientswithhipfracture.Overall,calibrationofthemodifiedTRISSwasadequate

forthetotalcohort,withanoverestimationforelderlypatientsandanunderestimationforpatients

withoutbraininjury.

Conclusions:OutcomecomparisonconductedwithTRISS-basedpredictionsshouldbeinterpretedwith

care.Ifpossible,futureresearchshoulddevelopasimplepredictionmodelthathasaccuratesurvival

predictionintheagingoveralltraumapopulation(preferablewithpatientswithhipfracture),with

readilyavailablepredictors.

©2018ElsevierLtd.Allrightsreserved.

Introduction

Predictionmodelsthatadequatelypredictsurvivalarerequired todeterminethequalityofcareintraumapatients.Traumaisa majorcauseofmortalityinyoungadultsworldwide[1].In2014, almost 84 000 patients were admitted due to injuries in the Netherlandsandthe30-daymortalityratewas2.1%[2].Scoring systemsandpredictionmodelsareimportanttoolstoquantifythe

probabilityofsurvivalandtoevaluateandimprovethequalityof careforthelargenumberofinjuredpatients[3].

TheTraumaandInjurySeverityScore(TRISS)was developed fromtheMajorTraumaOutcomeStudy(MTOS)in1987toevaluate thequalityoftraumacarebycomparingoutcomeswithanorm score[4].TheTRISSisaweightedscorebasedontheInjurySeverity Score(ISS),age,andthecodedRevisedTraumascore(RTS).TheRTS combinestheGlasgowComaScore(GCS),SystolicBloodPressure (SBP) and RespiratoryRate (RR). TheMTOSwas a retrospective studyconductedinNorthAmericafrom1982through1987and wasofgreatvalueforthedevelopmentofTRISS.Ithasbeenshown previouslythattheTRISShasseverallimitations[5,6].Theuseof TRISSin anexternalpopulationraisesconcerns,because differ-encesbetweencohortsaredistinct[7,8].

Previous researchin the Dutchpopulationdemonstrated an adequate performance of the TRISSwith coef_ficients from the * Correspondingauthorat:Elisabeth-TweeStedenHospital,DepartmentTrauma

TopCare,Hilvarenbeekseweg60,5000LCTilburg,TheNetherlands.

E-mailaddresses:[email protected](L.deMunter),[email protected]

(S.Polinder),[email protected](D.Nieboer),[email protected]

(K.W.W. Lansink),[email protected](E.W. Steyerberg),

[email protected](M.A.C.deJongh).

https://doi.org/10.1016/j.injury.2018.03.036

0020-1383/©2018ElsevierLtd.Allrightsreserved.

–

ContentslistsavailableatScienceDirect

Injury

MTOSorfromtheNationalTraumaDataBank(NTDB)inthetotal population,butdemonstrateda poorreflectionof themortality riskofelderlypatients(withhipfractures)[9–12]. Furthermore, Frankema et al. [13] suggested developing and using a more accuratemodel for theevaluation of traumacarein the Dutch trauma population. A recent review showed that there is no agreementonabetterandpracticallyapplicablemodeltousein theevaluationoftraumacare[14].

In 2015 theDutchTraumaRegistrydeveloped a newmodel basedsolelyonthevariablesintheTRISSmodelaccordingtotheir traumapopulation,includingtheelderlypatientswithhipfracture [15].ThismodelisusedtocomparequalityofcarebetweenDutch hospitals,buthasneverbeenvalidatedinsubsets.Theaimofthis studywastodeterminetheperformanceofthemodifiedTRISSin

subpopulations and to determine where this model needs

improvementforbettersurvivalpredictionsintheDutchtrauma population.

Methods

Studypopulationanddatacollection

Atpresent,theNetherlandsconsistsofeleventraumaregions, all including a coordinating trauma level I center. The region Noord-Brabant is representative of the total Dutch trauma population.Itcovers16%ofalladmittedtraumapatientsinthe Netherlandsandincludesurbanaswellasruralpopulations[2]. ElevenhospitalsintheregionNoord-Brabantcontributedtothe BrabantTraumaRegistry(BTR),includingonelevelIhospitaland tenlevelIIorIIIhospitals.Theregistrydatabasecontainsdataofall traumapatientsinNoord-Brabantthatwereadmittedaftervisiting the Emergency Department (ED) within 48h after a trauma, independentofinjuryseverityorinjurytype.Secondaryreferrals andpatientswhodieintheEDwerealsoregistered.Patientswho weredeadonarrivalwereexcludedfromtheregistry.

Atotalof72411patientswereregisteredintheBTRfrom2010 through 2015. Variables collected in this registry included demographics,SBP, RR, GCS,ISS, trauma mechanism (blunt vs. penetrating) and Abbreviated Injury Scale (AIS)-codes (version 1998)[16].In-hospitalmortalitywas consideredastheprimary outcomemeasure.

Model

TheProbabilityofsurvival(Ps)wascalculatedusingthenatural logarithm(Logit)ofthemodifiedTRISS:

Logit (Ps)=

a

b

b

b

b

b

Agewasequalto0ifthepatientwas_<55yearsandequalto1if thepatientwas55years of age.The coefficients werederived fromtheDutchTraumaRegistryin2015(Table1)[15].

Subsets

Analyseswereperformedonthetotalcohort(withandwithout elderlypatientswithhipfractures)andonseveralsubsets.Elderly patientswithhipfractures(patients65yearsand an ISS13, sufferinganisolatedfractureoftheproximalfemur[definedas: AIS1998codes851808.3,851810.3and851812.3])wereexcluded in the subsets, because it was previouslysuggested that those patientsshouldbeexcludedfrompredictionmodeling[17,18].Each subsetrepresentsdifferentchallengestothetraumacenters,and werebasedon:

Age:

-Elderly,includingonlypatients>75years -Children,includingonlypatients15years

Typeof injury: Traumatic Brain Injury (TBI), defined as

AIS-head3

Injuryseverity:Majortrauma,definedasISS>15

LOS>2,includingpatientswho dieduring thefirst2daysof admission.

Traumacenter (including onlypatientsadmitted toa trauma center level I) and non-trauma center (including patients admittedtoatraumacenterlevelIIorIII).

Statisticalanalysis

PrehospitalcodedvaluesfortheVcomponentofGCSandRR were selected in patients that were sedated and/or intubated, insteadoftheregisteredhospitalvalues.Also,prehospitalvalues fortheEandMcomponentofGCSwereselectedinpatientsthat weresedated.Datawerescreenedformissingvalues.ISS,RTSand ageforpatientswithmissingoutcomevaluesandwithunknown mechanism of injury were compared to patients with known outcome values.Missing valuepatternswereanalyzedfor GCS, SBP,RRandISS.ThecomponentsoftheGCSweretransformedinto dummyvariablesandRRwaslog-transformedintheimputation process.WeassumedthatmissingvalueswereMissingAtRandom (MAR) [19] and imputed missing values using the following variables:mortality,mechanismofinjury,ISS,eye/motor/verbal(E, MandV)componentofGCS,SBP,RR,age,with45imputationsand 10iterations. Sensitivityanalysiswas performed in which only completecaseswereincluded.

Theperformancemeasuresthatwereusedinthisstudywere discrimination and calibration. Discrimination was calculated using the Area Under the Receiver Operating Curve (AUROC), includinga95%confidenceinterval(95%CI).Differencesbetween AUROCwereconsideredsignificantifthe95%CIdidnotoverlap. Calibration was assessed graphically using calibration plots. In calibrationplotstheagreementbetweenobservedproportionof survivalandthepredictedprobabilityisvisualizedusingrestricted cubicsplines.

The performance of modified TRISS for the total group of patientswascomparedwiththespeci_ficsubsets,todeterminethe effect of inclusion and exclusion of the subsets. The statistical programsIBMSPSSversion24(Chicago,USA)andRversion3.4.0 (R Foundation for Statistical Computing, Vienna, Austria) were usedfortheanalyses.

Results

Baselinecharacteristics

Atotalof72411patientswereincludedintheBTR.Patientswith penetratinginjury(N=2539)wereexcludedbecausethenumberof non-survivors (N=32) was too low to interpret the results of

Table1

Theestimatedcoefficients(bn)ofthemodifiedTRISSasusedinthisstudy. Coefficientsa

MechanismofInjury Blunt

Intercept 1.5090 RR 0.2372 SBP 0.6460 GCS 0.4008 ISS 0.1087 Age 2.2091

Listofabbreviations:GCS,GlasgowComaScale;ISS,InjurySeverityScore;RR, RespiratoryRate;SBP,SystolicBloodPressure.

a

All coefficients are for blunt injuries. Penetrating injuries havedifferent coefficients(datanotshown).

discrimination and calibration. Patients with unknown survival outcome (N=125) were excluded from analysis. The excluded patients didnotdifferinage,RTS,orISSfromthetotalcohort(datanotshown). Therewere69747patientsusedforfurtheranalysis,including 11514 elderlypatients with hip fracture. The eye component, motorcomponent and verbalcomponentof theGlasgowComa Scaleweremissingin 5995(8.6%), 6024(8.6%)and6162(8.8%) observations,respectively.SBPwasmissingin11777observations (16.9%),RRin30083observations(43.1%)andISSwasmissingin 158patients(0.2%).

Themortalityratedecreased(1.7%vs.2.1%)inthetotalcohort afterexclusionofelderlypatientswithhipfractures(Table2).Next, theexclusionofchildrenfromtheBTRresultedinanincreasein mortalityrate(2.1%).However,theISS(median[IQR])remained thesame.ExcludingtheelderlyfromtheBTRresultedinahigher percentage of men (60.7%) and a lower mortality rate (0.9%), comparedtothetotalcohort.

Theexclusionofnon-TBIpatientsandminorinjuryresultedin older age(55.3(SD:23.9)and60.6 (SD:22.3)respectively) and higher mortalityrates(12.1%and12.8%respectively).PatientswithLOS>2 hadamean(SD)ageof60.6(22.9)andamortalityrateof3.4%.

PatientsinthelevelItraumacenterwereyounger(45.4[SD: 27.6]),hadahighermortalityrate(4.5%),andhadahighermedian ISS(9[IQR:4–11])comparedtothelevelIIandIIItraumacenters (mortalityrate:1.2%,age:47.9[SD:27.9]andISS[IQR]:5[4–9]). Discrimination

Nodifferenceswerefoundbetweencompletecaseanalysisand the imputed dataset for discrimination (data not shown). The AUROCofthetotalcohortwas0.84(95%CI:0.83,0.85)andforthe elderlypatientswithhipfracture0.53(95%CI:0.51,0.56)(Fig.1). AfterexclusionoftheelderlypatientswithhipfracturetheAUROC ofthedifferentsubsetsrangedfrom0.72(95%CI:0.70,0.74)forthe subsetolderthan75yearsto0.93(95%CI:0.91,0.94)forthesubset levelItraumacenter.Thesubsetwithexclusionofchildren,the subset with elderly patients, the selections based on ISS, LOS>2days and the subset with only level II and III trauma centersdecreasedthediscriminativeabilitysignificantly. Discrim-inationwassignificantlyhigherintheLevelItraumacenterthanin thelevelIIandIIItraumacenters.

Calibration

CalibrationcurvesforthemodifiedTRISSwereshowninFig.2. Therewerenoapparent differencesbetween calibrationof the

total cohort and the complete cases. The total cohort showed predictions close to the identity line, thus equal observed proportion and predicted probabilities. After exclusion of the elderly patients with hip fracture, the cohort showed higher observed proportion of survival compared to the predicted probabilitiesinthehighestPsbins(0.8–1.0).

Thesubset75yearsoldshowedahigherobservedproportion ofsurvivalcomparedtothetotalcohort,especiallyinthehighestPs bins (0.5–1.0). The subset of elderly showed a significant overestimationofthepredictedsurvivalrate.

ThecalibrationcurveoftheTBIsubsetshowedan underesti-mationofthepredictedprobabilitiesoftheTRISSinthelowerPs bins(0.0–0.3)andanoverestimationofthehigherPsbins(0.3–1.0). Thenon-TBIpatientsshowedasignificantunderestimationofthe predictedprobabilityofsurvivalinallprobabilitybins.Calibration ofthesubsetwithanISS>15showedasimilarcurveastheTBI subset.ThesubsetLOS>2showedancalibrationcurveclosetothe identity line. The level I trauma center had higher observed proportion survivors in the lower Ps bins (0.0–0.7) but lower observedproportioninthehigherPsbins(0.7–1.0)comparedto thepredictedprobabilities.Thecalibrationcurveofthesubsetwith onlylevelIIandlevelIIItraumacenterswasclosetotheidentity line.

Discussion

Predictionmodelsneedtobereliableifusedinevaluatingthe qualityoftraumacare.Althoughdiscriminationofthemodified TRISS in the total trauma cohort was adequate, the model performedmuchbetterwhenexcludingelderly,withorwithout hip fractures. Overall, calibration of the modified TRISS was adequateforthetotalcohort.However,themodeloverestimates the survival for the elderly and underestimates survival for patientswithoutTBI.

Discriminationof amodel isdependentonthe distributionof prognosticfactors.Discriminationintheelderly(withhipfractures) couldbelowbecausetheheterogeneityofthecase-mixdecreased.In contrasttodiscrimination,calibrationofthecohortexcludingelderly withhipfracturesshowednodifferencescomparedtocalibrationof thetotalcohort.Thelackofdifferencesinthecalibrationplotcouldbe explainedbythehighnumberofpatientsremaininginthehighest predictedprobability groupafter excludingtherelativelownumberof elderlywithhipfractures.TheNTDBencouragesresearcherstouse inclusionandexclusioncriteriainthedataregistrytocreateamore homogeneousgroup;forexample,hipfracturesshouldbeexcludedor analyzedseparately,whichisalsoconfirmedbyGomezetal.[17,18].In

Table2

StudycharacteristicsoftheBrabantTraumaRegistrywithpatientsubsetsfrom2010through2015. BrabantTraumaRegistry

Includingelderlywithhip fracturesa

Excludingelderlywithhipfracturesa

Totalcohort Hipfracturea

Totalcohort Age(years) Traumaseverity Traumacenter

>15 75 TBIb

Majortraumac

LOS>2 LevelI LevelIIandIII

N 69747 11514 58233 47565 46150 3531 3932 28883 8443 49790

Age(mean,SD) 53.2(28.6) 82.0(7.7) 47.5(27.8) 56.6(22.2) 38.0(23.0) 55.3(23.9) 55.7(22.3) 60.6(22.9) 45.4(27.6) 47.9(27.9)

Male(%) 49.7 27.9 53.9 53.0 60.7 62.3 65.7 47.6 58.5 53.2

In-hospitalmortality(%,N) 2.1(1490) 4.4(502) 1.7(988) 2.1(982) 0.9(406) 12.1(428) 12.8(503) 3.4(975) 4.5(381) 1.2(605) RTS(mean,SD) 7.7(0.4) 7.8(0.3) 7.7(0.5) 7.7(0.5) 7.7(0.5) 7.1(1.3) 7.2(1.3) 7.7(0.6) 7.5(0.9) 7.7(0.4) ISS(median,IQR) 9(4–9) 9(9–9) 5(4–9) 5(4–9) 5(4–9) 17(11–25) 20(17–26) 9(4–9) 9(4–11) 5(4–9) ISS(mean,SD) 7.5(5.2) 9.0(0.2) 7.2(5.7) 7.4(6.1) 7.2(5.7) 19.2(9.5) 22.9(8.6) 8.7(6.7) 10.2(9.4) 6.7(4.6) Listofabbreviations:AIS,AbbreviatedInjuryScale;ISS,InjurySeverityScore;IQR,Interquartilerange;LOS,LengthofStay;N,number;SD,StandardDeviation;RTS,Revised TraumaScore;TBI,TraumaticBrainInjury.

a

Patientswithhipfracturewereconsideredequaltoorolderthan65years,ISS<13andoneofthefollowingAIS1998codes851808.3,851810.3or851812.3. b

PatientswithTBIweredefinedasAIS-head3. c

MajortraumawasdefinedasISS>15.

contrast,othersarguethatelderlywithisolatedhipfractureshouldbe included in the trauma registry [20]. Elderly with hip fracture comprisecurrently17%ofthetotalDutchtraumapopulation[2].Due totheagingpopulationandthehighincidenceoffallswithinthese oftenfrailpatients,thenumberofelderlyinthetraumaregistrieswith hipfractureswillincreasethefollowingdecades.However,thisstudy supportsthefactthatelderlywithhipfractureshouldbeexcludedfor generaltraumacenterbenchmarkingwhenthemodifiedTRISS isused andshouldbeanalyzedseparatelyforbenchmarkingpurposesusinga morespecificpredictionmodel.Nevertheless,ifitcouldbeachievedto develop a model with accurate predictions in all subsets, it is preferable to include elderly with hip fractures forevaluation of quality care,tocoveralltraumarelatedinjuries.

Anotherexplanationofthemoderatediscriminationinelderly couldbeexplainedbythelackofmeasuresforfrailtyinthemodel. Next to frailty, dichotomization of age leads to a loss of information,andcouldbeoneofthemain reasonsforthepoor performance of themodi_fiedTRISSin the elderly[21,22]. Also, elderlyoftensufferfromcomorbidities,whichcouldbeimportant predictorsofmortalityintheagingpopulation[17,23–25].Someof theseissueswerealready incorporatedinpreviouslydeveloped models[26–28].However,comorbiditymeasuresandfrailtyare notincorporatedintheDutchTraumaRegistryandcouldtherefore notbeusedinbenchmarkingtraumacare.

The TBI subset had accurate predictions among the lower survival probabilities intervals in the TBI subset. However, the higher intervals showed an overestimation of the survival

predictions. Previous research suggested an inability of ISS to accountenoughformultipleinjuriestothesamebodyregion[29]. WhilemortalityisoftenattributedtoTBI,severeTBIisoftennot entirelycapturedbyameasuresuchastheISS.Championetal.[30] suggested to include all anatomic injuries for more accurate survivalpredictioninpatientswithTBI.

Thereare somelimitationstothis study. First, themodified TRISSisdevelopedintheDutchtraumapopulation,includingthe BTR.Resultscoulddifferifthismodelisvalidatedinanexternal cohort(i.e.acohortthatisnotincorporatedintheDutchtrauma registry).Thestudymaynotbegeneralizabletoothersettingswith different patient composition. Second, missing values are a common problem in trauma registries. Ignoring them could influencetheresultsanddecreasethestatisticalpower.Multiple imputationisanincreasinglychosensolutiontominimizebiasand increaseprecision[31–33].Theimputationmodelappliedinthis studywasbasedonliterature[34].Therefore,weassumethatno major bias occurred, as con_firmed by the similar results in a completecaseanalysis.However,itcouldalsobearguedtoexclude predictorswithahighproportionofmissingvalues(e.g.RRinthe modifiedTRISS)formoreoptimalpredictions.Next,patientswith

unknown mechanism of injury (N=1172) were included as

patients with blunt injuries. It is unlikely this influenced the results, since most trauma patients have blunt injury [2]. In addition,theuseofAIS98isconsideredashortcoming.TheBTR includedAIS08codesfrom2015onwards,butconversionofAIS98 toAIS08showedtobeunreliable[35].However,toobtainpowerto

Fig.1.DiscriminationofthemodifiedTRISSforthetotalcohortandsubpopulations.

Listofabbreviations:AUROC,AreaUnderReceiverOperatingCurve;CI,ConfidenceInterval;ISS,InjurySeverityScore;LOS,LengthofStay;TBI,TraumaticBrainInjury. *Patientswithhipfracturewereconsideredequaltoorolderthan65years,ISS<13andoneofthefollowingAIS1998codes851808.3,851810.3or851812.3.

assess performancesamong subsets of the registrymore years witholdAIS98codeswereused,insteadoftwoyearswithnewer AIS08codes.Last,wenotethattheoutcomemeasure,in-hospital mortality,isapoormeasureofoutcome.Abetteroutcomewould be30-daymortality.However,thisoutcomeisnotreportedinthe traumaregistry.

The current overallmortalityrate in the Netherlandsof the acutehospitalizedtraumapopulationisonly2%.Therefore,itcould besuggestedthatmortalityisnotthemostimportantoutcometo evaluatetraumacare.Withthedecreaseinmortalityratesinthe developedcountries,traumacarecouldalsobeassessedwith non-fataloutcomemeasures.

Fig.2.CalibrationcurvesforthemodifiedTRISSinthetotalcohortandindifferentsubsetsofpatients. Listofabbreviations:ISS,InjurySeverityScore;LOS,LengthofStay;TBI,TraumaticBrainInjury.

*Cohort,excludingpatientswithhipfracture(patientswithhipfracturewereconsideredequaltoorolderthan65years,ISS<13andoneofthefollowingAIS1998codes 851808.3,851810.3or851812.3).

Outcome comparison conducted with the modified TRISS shouldbeinterpretedwithcare,becausetheperformanceofthe modelishighlydependentonthecasemixofthepatientsincluded intheregistry. Thequalityof careintheelderlyshouldnotbe evaluatedwhenthemodifiedTRISSisused.Ifitcouldbeachieved inthefuturetodevelopamodelwithaccuratepredictionsinall subsets,itispreferabletoincludeelderlyforevaluationofquality care to cover all trauma related injuries. Predictors should be readilyavailableandeasytocollectinthecurrenttraumaregistry. Conflictsofinterest

Theauthorsdeclarenoconflictofinterest. Acknowledgement

ThisworkwassupportedbyagrantoftheDutchorganization forhealthresearchandcareinnovation(ZonMW)sectionTopCare projects(grantnumber:80-84200-98-14226).ZonMwhadnorole inthedesign,analysesorinterpretationofthisstudy.

References

[1]WorldHealthOrganization.Injuriesandviolence:thefacts.2014WO700 (August).

[2]Landelijke Netwerk Acute Zorg. Traumazorg in beeld: Landelijke traumaregistratie2012-2016.rapportagenederland;2017.

[3]RutledgeR,OslerT,EmeryS,Kromhout-SchiroS.Theendoftheinjuryseverity score(ISS)andthetraumaandinjuryseverityscore(TRISS):ICISS,an internationalclassificationofdiseases,ninthrevision-basedpredictiontool, outperformsbothISSandTRISSaspredictorsoftraumapatientsurvival, hospitalcharges,andhospitallengthofstay.JTrauma1998;44(1):41–9. [4]ChampionHR,SaccoWJ,HuntTK.Traumaseverityscoringtopredictmortality.

WorldJSurg1983;7(1):4–11.

[5]ChampionHR,CopesWS,SaccoWJ,LawnickMM,KeastSL,FreyCF.Themajor traumaoutcomestudy:establishingnationalnormsfortraumacare.JTrauma 1990;30(11):1356–65.

[6]RogersFB,OslerT,KrasneM,RogersA,BradburnEH,LeeJC,etal.HasTRISS becomeananachronism?Acomparisonofmortalitybetweenthenational traumadatabankandmajortraumaoutcomestudydatabases.JTrauma 2012;73(2):326–31.

[7]DemetriadesD,ChanLS,VelmahosG,BerneTV,Cornwell3rdEE,BelzbergH, etal.TRISSmethodologyintrauma:theneedforalternatives.BrJSurg1998;85 (3):379–84.

[8]CaytenC,StahlW,MurphyJ,AgarwalN,ByrneD.LimitationsoftheTRISS methodforinterhospitalcomparisons:amultihospitalstudy.JTrauma 1991;31(4):471–82.

[9]ChampionH,SaccoW,Copes W.Injuryseverityscoring again.JTrauma 1995;38(1):94–5.

[10]SchluterPJ,NathensA,NealML,GobleS,CameronCM,DaveyTM,etal.Trauma andinjuryseverityscore(TRISS)coefficients2009revision.JTrauma2010;68 (4):761–70.

[11]DeJonghMAC,VerhofstadMHJ,LeenenLPH.Accuracyofdifferentsurvival predictionmodelsinatraumapopulation.BrJSurg2010;97(12):1805–13. [12]JoosseP,SchepNW,GoslingsJC,RegionalTraumaNetworkTraumaNetAMC

Collaborators.Injuryprofilesrelatedtomortalityinpatientswithalowinjury severityscore:acase-mixissue?JTrauma2012;73(1):179–85.

[13]FrankemaSPG,EdwardsMJR,SteyerbergEW,VanVugtAB.Predictingsurvival aftertrauma:acomparisonofTRISSandASCOTinthenetherlands.EurJ Trauma2002;28(6):355–64.

[14]deMunterL,PolinderS,LansinkK,CnossenM,SteyerbergE,deJonghMAC. Mortalitypredictionmodelsinthegeneraltraumapopulation:asystematic review.Injury2017;48(2):221–9.

[15]WARLTR. Landelijketraumaregistratie–toelichting uitkomstevaluatie2015. TRISSmodelNederlandsecoëfficiëntenenSMR.resultaten2015regio: Netwerkacutezorgbrabant.2016.

[16]AssociationfortheAdvancementofAutomotiveMedicine.Abbreviatedinjury scale;1990revision:update98.AAAM;1998.

[17]GomezD,HaasB,HemmilaM,PasqualeM,GobleS,NealM,etal.Hipscanlie: impactofexcludingisolatedhipfracturesonexternalbenchmarkingoftrauma centerperformance.JTrauma2010;69(5):1037–41.

[18]AmericanCollegeofSurgeons.Nationaltraumadatabank:NTDBresearchdata setadmissionyear2008.Usermanual.2010February.

[19]LittleR, Rubin D. Statisticalanalysis with missing data. Wiley series in probabilityandmathematicalstatistics.JohnWiley&Sons;2014. [20]BergeronE,LavoieA,BelcaidA,RatteS,ClasD.Shouldpatientswithisolated

hipfracturesbeincludedintraumaregistries?JTrauma2005;58(4):793–7. [21]SteyerbergEW.Chapter9:codingofcategoricalandcontinuouspredictors.

Clinicalpredictionmodels:apracticalapproachtodevelopment,validation, andupdating.SpringerScience&BusinessMedia;2010.p.159–73. [22]RoystonP,AltmanDG,SauerbreiW.Dichotomizingcontinuouspredictorsin

multipleregression:abadidea.StatMed2006;25(1):127–41.

[23]QuailJM,LixLM,OsmanBA,TeareGF.Comparingcomorbiditymeasuresfor predictingmortalityandhospitalizationinthreepopulation-basedcohorts. BMCHealthServRes2011;11(1):1.

[24]GlanceLG, DickAW, Mukamel DB,Meredith W,OslerTM. Theeffectof preexistingconditionsonhospitalqualitymeasurementforinjuredpatients. AnnSurg2010;251(4):728–34.

[25]ThompsonHJ,RivaraFP,NathensA,WangJ, JurkovichGJ, MacKenzieEJ. Developmentandvalidationofthemortalityriskfortraumacomorbidity index.AnnSurg2010;252(2):370–5.

[26]Jones JM, Skaga NO,Søvik SO,Lossius HM, EkenT. Norwegian survival predictionmodelintrauma:modellingeffectsofanatomicinjury,acute physiology,age,andco-morbidity.ActaAnaesthesiolScand2014;58(3):303–

15.

[27]BouamraO,WrotchfordA,HollisS,VailA,WoodfordM,LeckyF.Anew approachtooutcomepredictionintrauma:acomparisonwiththetrissmodel. JTrauma2006;61(3):701–10.

[28]BergeronE,RossignolM,OslerT,ClasD.ImprovingtheTRISSmethodologyby restructuringagecategoriesandaddingcomorbidities.JTrauma2004;56 (4):760–7.

[29]MoonJ,SeoB,JangJ,LeeJ,MoonH.Evaluationofprobabilityofsurvivalusing traumaandinjuryseverityscoremethodinsevereneurotraumapatients.J KoreanNeurosurgSoc2013;54(1):42–6.

[30]ChampionHR,CopesWS,SaccoWJ,Frey CF,HolcroftJW,HoytDB, etal. Improvedpredictionsfromaseveritycharacterizationoftrauma(ASCOT)over traumaandinjuryseverityscore(TRISS):resultsofanindependentevaluation. JTrauma1996;40(1):42–9.

[31]MooreL,LavoieA,LeSageN,LibermanM,SampalisJS,BergeronE,etal. Multipleimputationoftheglasgowcomascore.JTrauma2005;59(3):698–

704.

[32]O’ReillyGM,JolleyDJ,CameronPA,GabbeB.Missinginaction:acasestudyof theapplicationofmethodsfordealingwithmissingdatatotraumasystem benchmarking.AcadEmergMed2010;17(10):1122–9.

[33]NewgardCD.Thevalidityofusingmultipleimputationformissing out-of-hospitaldatainastatetraumaregistry.AcadEmergMed2006;13(3):314–24. [34]deMunterL,terBogtNC,HesselinkDD,deJonghMA.Imputationstrategiesin

thetraumaregistration.JTrauma2017;83(5):828–36.

[35]PalmerCS,FranklynM.Assessmentoftheeffectsandlimitationsofthe1998to 2008abbreviatedinjuryscalemapusingalargepopulation-baseddataset. ScandJTraumaResuscEmergMed2011;19(1):1.