Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views

Contents lists available at ScienceDirect

Medical

Image

Analysis

journal homepage: www.elsevier.com/locate/media

Slic-Seg:

A

minimally

interactive

segmentation

of

the

placenta

from

sparse

and

motion-corrupted

fetal

MRI

in

multiple

views

Guotai Wang

a ,∗

_{, Maria A. Zuluaga}

a

_{, Rosalind Pratt}

a ,b

_{, Michael Aertsen}

c

_{, Tom Doel}

a

_,

Maria Klusmann

d

, Anna L. David

b

, Jan Deprest

e

, Tom Vercauteren

a

, Sébastien Ourselin

a a_{TranslationalImagingGroup,CMIC,UniversityCollegeLondon,London,UK}

b_{InstituteforWomen’sHealth,UniversityCollegeLondon,London,UK} c_{DepartmentofRadiology,UniversityHospitalsKULeuven,Leuven,Belgium} d_{DepartmentofRadiology,UniversityCollegeLondonHospital,London,UK} e_{DepartmentofObstetrics,UniversityHospitalsKULeuven,Leuven,Belgium}

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received20November2015 Revised6April2016 Accepted23April2016 Availableonline3May2016

Keywords: FetalMRI Interactivemethod Co-segmentation GraphCuts Randomforests

a

b

s

t

r

a

c

t

SegmentationoftheplacentafromfetalMRIischallengingduetosparseacquisition,inter-slicemotion, andthewidelyvaryingpositionandshapeoftheplacentabetweenpregnantwomen.Weproposea min-imallyinteractiveframeworkthatcombinesmultiplevolumesacquiredindifferentviewstoobtain accu-ratesegmentationoftheplacenta.Inthefirstphase,aminimallyinteractiveslice-by-slicepropagation methodcalledSlic-Segisused toobtainaninitialsegmentationfromasinglemotion-corruptedsparse volumeimage.Itcombineshigh-levelfeatures, onlineRandomForestsand ConditionalRandom Fields, andonlyneedsuser interactionsinasingleslice.Inthesecondphase, totakeadvantageofthe com-plementaryresolutioninmultiplevolumesacquiredindifferentviews,wefurtherproposea probability-based4DGraphCutsmethodtorefinetheinitialsegmentationsusinginter-sliceandinter-image consis-tency.Weusedourminimallyinteractiveframeworktoexaminetheplacentasof16mid-gestation pa-tientsfromMRIacquiredinaxialandsagittalviewsrespectively.Theresultsshowtheproposedmethod has1)agoodperformanceevenincaseswheresparsescribblesprovidedbytheuserleadtopoor re-sults withthe competitivepropagation approaches;2) agood interactivity withlow intra- and inter-operatorvariability;3) higheraccuracythanstate-of-the-artinteractivesegmentationmethods;and 4) animprovedaccuracyduetotheco-segmentationbasedrefinement, whichoutperformssinglevolume orintensity-basedGraphCuts.

© 2016ElsevierB.V.Allrightsreserved.

1. Introduction

The placenta plays a critical role in the growth and develop-ment of the fetus during pregnancy. Placental abnormalities are a cause of poor maternal and fetal outcome. Placental attach-ment disorders (Mazouni et al., 2007 ) such as placenta accreta are due to an abnormally adherent placenta invading the my-ometrium, and are associated with life-threatening postpartum hemorrhage. Image-baseddiagnosisofplacentaaccreta allowsfor multidisciplinary planning in an attempt to minimize risks dur-ing the delivery. In monochorionic multiple pregnancy, twin-to-twin transfusion syndrome (TTTS) (Deprest et al., 2010 ) can re-sultinunequalblooddistributionandseverebirthdefectsforone or both twins.Furthermore, selectiveintrauterine growth

restric-∗ _{Correspondingauthor.}

E-mailaddress:[email protected](G.Wang).

tion (sIUGR) (Chalouhi et al., 2013 ) leads to poor growth in the twinwithinsufficient nourishmentfromthe placenta. Minimally-invasivefetoscopicsurgeryprovidesaneffectivetreatmentforTTTS and sIUGR, and surgical planning (Pratt et al., 2015 ) can poten-tiallyreducetreatment-relatedmorbidityandmortality.Most com-monly,placental insufficiencyduetopoorplacentationisamajor causeoffetalgrowthrestrictionwhichcanresultincerebralpalsy (Spencer et al., 2014 ). Betterplacental imagingmayallow predic-tionofplacentalinsufficiencyandtargetedinterventions.

An image-based diagnosis and surgical planning system re-quiresaccurate androbust extraction ofthe placentafrom imag-ingmodalitieswithahighspatialresolution,goodsofttissue con-trast,andlarge fieldofview suchasmagneticresonanceimaging (MRI). However, high-quality 3D fetal MRI is difficult to achieve, sincethefreemovementofthefetuscancauseseveremotion ar-tifacts(Kainz et al., 2014 ).The Single ShotFastSpin Echo(SSFSE) allows themotion artifactsto be nearly absent ineach slice, but

http://dx.doi.org/10.1016/j.media.2016.04.009

(a) axial view

(b) sagittal view

(c) coronal view

(d) axial view

Fig.1. ExamplesoffetalMRI.(a),(b)and(c)arefromonepatientwhile(d)isfromanother.Notethemotionsanddifferentappearancebetweenslicesin(b)and(c).The placentaisanteriorin(a),butposteriorin(d).

inter-slicemotionsstill corruptthevolumetricdata.The slicesare acquiredin an interleaved spatial order,which leads to inhomo-geneousappearance betweenslices.Inaddition,fetal MRI is usu-allysparsely acquired with a large inter-slice spacing fora good contrast-to-noiseratio.Although some novel reconstruction tech-niques (Gholipour et al., 2010; Kainz et al., 2015 ) can get super-resolution volume data of fetal brain, they have yet to demon-strate their utility for placental imaging andrequire a dedicated non-standardacquisition protocol. Fig. 1 showssomeexamples of placentalMRIdemonstratingthecurrentchallengesforimage seg-mentationinfetalMRI.

Toaddressthisproblem,somerecentmethods,alldedicatedto thedevelopingfetalbrain, havereliedonpriorknowledgelearned fromagroupofpatientsoralarge populationtoenhancethe ac-curacyandrobustnessofthesegmentation. Forexample,ashape prior model was used to extract head structures from fetal MRI (Anquez et al., 2009 ), and propagated atlases were used to ob-tain arobust segmentation ofthe fetal brain(Habas et al., 2010; Gholipour et al., 2012 ).These methodsassume thatthe variances of the target organ’s shape and position are moderate or small across different individuals. However, the position and shape of theplacenta within the uterus vary greatly during gestationand betweenpregnancies (Fig. 1 ). This makes it more challengingto modelsuchstatisticalpriorknowledge,andalsobringsdifficulties toautomaticsegmentationoftheplacenta.

Interactivesegmentationmethods havebeenwidelyused (Gao et al., 2012; Zhao and Xie, 2013 ). Theyprovideabalancebetween manualdelineation, which givesaccurate androbust resultswith longsegmentationtime,andautomaticsegmentation,whichsaves timeforuserinteractionsbutoftenlacksinrobustness.Inpractical applications, an interactive segmentation method should achieve a high accuracy, minimize user interactions witha low variabil-ityamong users and be computationally fast. The way in which the user inputs are used and the number of user interactions have a great impact on the segmentation accuracy. User-guided 3D activecontour segmentation (Yushkevich et al., 2006; Xu and Prince, 1998 ) employstheuserinputsasseeds orinitialcontours ofthetargetorgan.GraphCuts(Boykov and Jolly, 2001 )takes user-providedscribblesashard constraintsanduses themto estimate theprobabilisticmodelofforegroundandbackground,whichis of-tenbasedonintensitydistributions(Boykov and Jolly, 2001; Freed- man and Zhang, 2005; Shi et al., 2012 ). GeodesicFramework(Bai and Sapiro, 2009 )andGeoS(Criminisi et al., 2008 )classifyapixel basedonitsweightedgeodesic distancetothescribbles. Random Walks(Grady et al., 2005 )assignsapixelwiththelabelforwhich arandomwalkerismostlikelytoreachfirst.GrowCut(Vezhnevets and Konouchine, 2005 )usesthescribblestosettheinitialstateof acellularautomationforthepixellabelingtask.Despitetheir suc-cessinmanyapplications,mostofthesemethodsrelyonlow di-mensionalfeatures and needa large numberof userinteractions to deal with imageswith low contrastand weak boundaries. To tacklewith this problem, machine learning based methods have beenproposedtolearntheuserintentionandgetanaccurate

seg-mentationwithfeweruserinteractions(Santner et al., 2009; Veer- araghavan and Miller, 2011; Park et al., 2014 ).Forexample,the4D ActiveCut proposed by Wang et al. (2014) actively selects candi-dateregionsforqueryingtheuser,withouttheneedtorefinethe segmentation sliceby slice. However, its abilityto deal with im-ages witha low resolution andmotioncorruptions hasnot been investigated.

Inrecentyears,co-segmentationmethods,whichcombine mul-tiple imagesthat provide complementary information,have been demonstrated to be able to achieve better segmentation results than methods working on a single image (Guo et al., 2015; Shi et al., 2012; Wang et al., 2012; Batra et al., 2010 ). For fetal MRI, thehighintra-sliceresolutionandlow inter-sliceresolutionmake itdifficulttogetagoodsegmentationresultfromasingle3D vol-ume. Fortunately,fetalMRI can beacquiredfromdifferent views. Although volumes acquired from different views are not com-pletely aligned due to motion, they can be used simultaneously withtheircomplementaryresolutionindifferentdirections. There-fore,co-segmentationoffetalMRIfrommultipleviewshasthe po-tentialtoprovideabetteraccuracyandrobustness.

To the best of our knowledge, there have been no previous works reportedon automatic orsemi-automatic segmentation of the placenta from fetal MRI. Recently we proposed a machine learningbasedmethodcalledSlic-Seg(Wang et al., 2015 )whichis designedto interactivelysegment theplacenta fromasingle vol-ume.Thismethodminimizestheuserinteractionsbyonly requir-ing user-provided scribbles in a single start slice. It learns from pixelsthatarelabeledbythescribblesandinfersthelabelsforall the remaining pixels by employinga combinationof online Ran-dom Forests (RF) (Breiman, 2001 ) using high-level features and Conditional Random Fields (CRF)(Boykov and Jolly, 2001 ). Good segmentationresultswere achievedinourinitialevaluation stud-ies (Wang et al., 2015 ). However, itonly workedon asingle vol-umeimage,thustheperformance mightbenegatively affectedby thesparselyacquireddata.Inaddition,itsinteractivityinpractice andimpactofhigh-levelfeaturesandCRFwerenotinvestigatedin detail.

In this paper, we extend the work of Wang et al. (2015) by using co-segmentation of multiple motion-corrupted volumes to overcome the low inter-slice resolution in a single sparsely-acquiredandmotion-corrupted volume.Weproposea refinement stepafterthe Slic-Seg-basedsinglevolumesegmentation.The re-finementtakesadvantageofcomplementaryresolutionindifferent volumesforahigheraccuracy.Wealsovalidatetheinteractivityof Slic-Segbyanalyzinghowitsperformanceisaffectedbythe num-berofuserinteractionsandmeasuringtheoperatorvariability. 2. Method

The workflow of our proposed method is shownin Fig. 2 . It consistsoftwo mainphases. Inthefirstphase, asingle sparsely-acquired and motion-corrupted volume is initially segmented by single volume Slic-Seg with minimal user interactions. In the

Fig.2. Theworkflowofourproposedsegmentationmethod.Inthefirstphase,Slic-Segisusedtosegmentasinglevolumeimagewithminimaluserinteractions.Inthe secondphase,initialsegmentationsofsinglevolumeSlic-Segarerefinedbycombiningvolumesacquiredindifferentviewsofthesamepatientforanimprovedaccuracy.

second phase, a probability-based 4D Graph Cuts framework is usedtorefinetheinitialsegmentationbycombiningtwoormore volumesacquiredindifferentviews.

2.1. Segmentationofasinglevolumeimagewithminimal user-interactions

The singlevolume Slic-Seg requiresthat a user selectsa start sliceanddraws a few scribblesinthat sliceto indicatethe fore-groundandthebackground.OnlineRFefficientlylearnsfromthese inputsandpredictstheprobabilitythatanunlabeledpixelbelongs tothe foregroundorthebackground.Totakeintoaccount spatial consistency,thatprobabilityisincorporatedintoaCRF.New train-ingdataisautomaticallyobtainedfromtheoutputoftheCRFand addedtothetrainingsetofaRFpredictoronthefly.The segmen-tationispropagatedtoother slicessequentiallyandautomatically withouttheneedformoreuserinteractions.Afterthepropagation, avolumetricprobabilityimageandaninitialsegmentationare ob-tained bystacking the outputof thecombinedRF andCRF inall theslicesrespectively.

2.1.1. Preprocessandfeatureextraction

To correct the motion between slices, a block-matching al-gorithm was implemented using the NiftyReg package (Ourselin et al., 2001 ). Feature extraction is implemented after the regis-tration. Foreach pixel, features are extractedfroma 9 × 9pixel region of interest (ROI) centered on it. In each ROI, we extract graylevelfeaturesincludingmeanandstandarddeviationof inten-sity, texturefeatures acquired by graylevel co-occurrencematrix (GLCM)andwaveletcoefficientfeaturesbasedonHaarwavelet.

2.1.2. Onlinerandomforeststraining

ARandomForest (Breiman, 2001 )is acollectionofbinary de-cisiontreescomposedofsplitnodesandleafnodes.Eachtreehas amaximumdepthofD.The trainingsetofeachtreeisrandomly sampledfromthe entirelabeled trainingset(label 1forthe pla-centa and label 0 forthe background). At a split node, a binary test isexecuted to minimize theuncertainty ofthe classlabelin the subsets based on Information Gain. At a leaf node, labels of all the training samples that havebeen propagated to that node areaveraged,andtheaveragelabelisinterpretedastheposterior

probability ofa samplebelonging tothe placenta,given that the samplehasfallenintothatleafnode.

Thetrainingdatain ourapplicationisobtainedin oneoftwo waysaccordingtothesegmentationstage.Forthestartslice, train-ingdatacomesfromthescribblesprovidedbytheuser.Duringthe propagation,afteronesliceSiissegmented,skeletonizationofthe

placenta is implemented by morphological operators to get new positive trainingdata, andthe background iseroded by a kernel witha given radius (i.e., 10 pixels) to get new negativetraining data.The newtrainingdataobtainedinSiareaddedtothe

exist-ing trainingset of RF on the fly. The RF is updated and used to testthenextsliceSi+1.Thisresultsinaprobabilitymap,whichis combinedwithaCRFtogetthelabelofSi+1.

We use the online Bagging (Saffari et al., 2009 ) method to model the sequential arrival of trainingdata as a Poisson distri-butionPois(

λ

), where

λ

issettoaconstantnumber.Aseachnew trainingsamplearrives,eachtreeisupdatedbychoosingthat sam-ple k times where k is a random number generated by Pois(

λ

). Eachsampleisexpectedtobeused

λ

timesbyeachtreesincethe expectationofkisE

(

k

)

₌

λ

.

2.1.3. Onlinerandomforeststesting

During the testing, each pixel sample x i in a slice ˜I is

prop-agatedthrough all trees. Forthe nthtree, a posteriorprobability

pn

(

c i

|

x i,I˜

)

isobtainedfromtheleafthatthetestsamplefallsinto,

where c i is the labelof x i. The final posterior isachieved asthe

averageacrossalltheNtrees. p

(

ci

|

xi,I˜

)

= 1 N N n=1 pn

(

ci

|

xi,I˜

)

(1)

2.1.4. Inferenceusingconditionalrandomfields

In the testing stage of RF, the posterior probability for each pixelisobtainedindependently,thustheresultissensitivetonoise andlacksspatialconsistency.Toaddressthisproblemandinferthe labelsetforallthepixelsinaslice,aCRFisusedforglobalspatial regularization.ThelabelsetC˜ofasliceisdeterminedby minimiz-ingthefollowingenergyfunction:

E

(

C˜

)

= i∈I˜

(

ci

|

xi,I˜

)

+

λ

1 {i,j}∈N1

(

ci,cj

|

I˜

)

(2)

(

ci

|

xi,I˜

)

=−logp

(

ci

|

xi,I˜

)

(3)

(

ci,cj

|

˜I

)

=Bi,j·

δ

i,j (4)

where

λ1

isacoefficienttoadjusttheweightbetweentwo poten-tials.Theunarypotential

(

c i

|

x i,I˜

)

measuresthecost for

assign-ingaclass label c i totheithpixel ina sliceI˜,andpcomesfrom

theoutput ofRF. N1 is the set ofall unordered pairs of {i, j} of neighboringpixelsintheslice.Thepairwisepotential

(

c i,c j

|

I˜

)

is

definedasacontrastsensitivePottsmodel.

δ

i,jequalsto1if c i=

c jand0otherwise.Bi,jmeasurestheenergyduetothedifference

inintensitybetweentwoneighboringpixels: Bi,j= 1 dist

(

i,j

)

·exp

−

(

I˜

(

i

)

−I˜

(

j

))

2 2

σ

2 1

(5) whereI˜

(

·

)

denotesthe intensityofone pixel.dist(i, j) is the spa-tialdistancebetweentwo neighboringpixels,and

σ1

controlsthe sensitivityof difference between I˜

(

i

)

and I˜

(

j

)

. The energy mini-mizationin Eq. (2) issolvedbyamaxflowalgorithm(Boykov and Jolly, 2001 ).ACRFisusedinevery sliceofthevolumetricimage. After the propagation, we stackthe segmentation ofall slicesto constructthevolumetricsegmentationresult.

2.1.5. VariationsofsinglevolumeSlic-Seg

Inordertoanalyzehoweachcomponentoftheabovedescribed methodaffects the segmentation, we consider threeof its varia-tionsforcomparison:

Offline Slic-Seg: thiscounterpart only uses user inputs inthe startsliceastrainingdatafortheRF.TheRF isnotupdated when alabelimage isobtainedfora newsliceduringthepropagation. It uses the samehigh-level features andCRF asin the proposed Slic-Seg.

Slic-Seg usinglow-level features:thisvariation isthe sameas ourproposedSlic-Seg exceptthatitemploys onlyintensity-based featuresratherthanhighdimensionalfeaturesincludingGLCMand Haarwavelet.

Slic-SegwithoutCRF:thismethodusesthesamehigh-level fea-turesandonlineRFasintheproposedSlic-Seg,butomitsCRF.To getthebinarysegmentationlabel,theoutputofRF isthresholded (thresholdprobabilityis0.5)andthenthelargestconnected com-ponent isselected. After that morphological openingand closing operationsareusedtogetasmoothedresult.

2.2.Refinementbasedonco-segmentationofvolumesacquiredfrom differentviews

SincethesinglevolumeSlic-Segimplementsspatial regulariza-tionbyusingCRFineach2Dslice,theconsistencybetween neigh-boringslices is not explicitlymodeled. In addition, it deals with eachsinglevolumeimage independently,andthelargeinter-slice spacingmaycorruptsegmentationresultsduringthepropagation. Toaddressthisproblem,werefinethesegmentationresultsof Slic-Seg by using the complementary resolution of volumes acquired fromdifferentviewsinaprobability-based4D GraphCuts frame-work. A Fast Free-Form Deformation algorithm (Rueckert et al., 1999; Modat et al., 2010 )isusedtoregisterthesagittal view vol-umeofone patienttothe axialview volumeofthesamepatient (performedat3levelswithfinalgridspacing:6mm _× 6mm _× 12 mm), but mis-alignment of placenta between them may not beperfectlyaddresseddueto themotion anddeformation.Thus, we do not impose the use of a single underlying segmentation (i.e.hard constraint)for allvolumes, butrather penalize discrep-anciesbetweenthesegmentation ofdifferentvolumesafter regis-tration(i.e,softconstraint).

CorrespondingtoI˜andC˜usedin Section 2.1 torepresenta2D sliceandits labelrespectively,we useIandCto representa3D

volumeimageanda3Dlabelingresultgivenbysinglevolume Slic-Seg, respectively. ConsideringK motion-corrupted volumetric im-agesI1,I2,...IKofthesamepatientsparselyacquiredfromdifferent

views, theuser provides scribblesina start sliceofeach volume respectivelyforthesinglevolumeSlic-Seg.TheoutputsofSlic-Seg forthemareP1,C1,P2,C2,...,PK,CK respectively,wherePkdenotes

a probability image and each of the resulting labeled images C_k

isassignedwithtemporaryvalues.Torefinethesetemporary seg-mentationsandgetthefinallabelsC1,...,CK, Eq. (2) isextendedby

incorporatinginter-sliceandinter-imageconsistency: E

(

C1,...,CK

)

= K k i∈Ik

(

ci

|

xi,Ik

)

+

λ

1 {i,j}∈N1 Bi,j·

δ

i,j +

λ

2 {i,j}∈N2 B_i,j·

δ

i,j+

λ

3 {i,j}∈N3 B_i,j·

δ

i,j (6)

where

andBi,j aredefinedin Eqs. (3) and (5) respectively. Bi,j

andB_i,jaretheinter-sliceandinter-imagebinaryenergyterm, re-spectively.

λ2

and

λ3

arecoefficientstoadjusttheweightoftheir correspondingterms.N2 andN3 arethesetofallunorderedpairs {i,j}ofcorrespondingpixelsfromtwoneighboringslicesandtwo volumeimages,respectively.

The three different types of neighboring pixels are shown in Fig. 3 .{a, b},{a, c},{a, d}and{a,e} showintra-sliceneighboring pixels that belongto N1.{d, f}showsinter-slice neighboring pix-elsinasinglevolumethatbelongtoN2.{a,g}showsinter-volume neighboringpixelsthat belongto N3.Togettheinter-imagepixel pairsfromtwo volumesI1 andI2,forone pixeli inavolume Ik1 (k1=1,2), its nearest pixel j in I1 and I2 is found, and {i, j} is addedtoN3 ifj∈Ik2 (k2=1,2)andk1=k2.

Toovercometheinhomogeneousappearancebetweendifferent slicesandbetweendifferentimages,theinter-slicetermand inter-imagetermaredefinedbasedontheprobabilityimageobtainedby theRFpredictioninthefirstphase,i.e.,singlevolumeSlic-Seg: B_i,j=_dist1

₍

_i ,j

)

·exp

−

(

Pk

(

i

)

−Pk

(

j

))

2 2

σ

2 2

(7) wherePk

(

i

)

=p

(

c i=1

|

x i,Ik

)

,and{i,j}∈N2. B_i,j=exp

−

(

Pk1

(

i

)

−Pk2

(

j

))

2 2

σ

2 3

(8) wherei∈Ik1,j∈Ik2,and{i,j}∈N3.

σ

2and

σ3

controlthe sensi-tivityofprobability difference.Sincethelasttermin Eq. (6) deals withcorresponding pixelsfromdifferent volumes,we donot use thedistancebetweensuchcorrespondingpixelstoweightthe en-ergyin Eq. (8) .Instead,wesettheweighttoaconstantvalue and it hasbeen incorporatedinto

λ3

.The energyminimization prob-lemin Eq. (6) issolvedbyMaxflow(Boykov and Jolly, 2001 ),after whichthefinal segmentationofI1,I2,..., IK areobtained

simulta-neously.

3. Experiments and results

3.1. Experimentdataandevaluationmethod

We collectedMRI scans of 16 fetusesin the second trimester intwo differentviews:1),axialview withslicedimension512 ×

448,voxelspacing0.7422mm_×0.7422mm,slicethickness3mm. 2) sagittal view with slice dimension 256 × 256, voxel spacing 1.484 mm × 1.484mm, slice thickness4 mm. The slicenumber rangesfrom50to70amongdifferentvolumes. Forsinglevolume Slic-Seg,astartsliceinthemiddleregionoftheplacentawas se-lected,andscribbleswereprovidedinthestartslice.Thealgorithm wasimplementedinC++withaMATLABGUIinterface.Feature ex-tractionwasimplementedin CUDAfora fasterspeed.The exper-imentswere performedona Maclaptop(OS X10.9.5)with16 G

Fig.3. ThreedifferentkindsofneighboringpixelsusedinEq.(6).{a,b},{a,c},{a,d},{a,e}areintra-sliceneighboringpixels(N1).{d,f}areinter-sliceneighboringpixels(N2).

{a,g}areinter-volumeneighboringpixels(N3).

Fig.4. Theeffectofparameterchangeonthesegmentationperformance.Therangesofλ1,λ2,λ3,σ2andσ3aredenotedbylogarithms.Thedashedlinesindicatethe

parametersettingintheexperiments.

RAM andan IntelCorei7CPUrunningat2.5GHzandan NVIDIA GeForce GT 750 M GPU. Parameter setting was:

λ

= 1, D = 10,

N =20,K= 2,

λ1

=40,

λ2

=10,

λ3

=3,

σ

1 =2.5,

σ2

= 0.005,

σ

3 = 0.08. The effect of parameter change on the segmentation performance ispresentedin Fig. 4 ,whichshowsstable segmenta-tionperformancewasachievedwiththechangeofeachparameter overalargerange.

The segmentationresults werecompared withmanual ground truth which was annotated by an experienced radiologist. For quantitativeevaluation,wemeasuredtheDicesimilaritycoefficient andtheaveragesymmetricsurfacedistance(ASSD).

Dice= 2

|

R s∩R g

|

|

R s

|

+

|

R g

|

(9)

where _Rs and Rg represent the region segmented by the

algo-rithmsandmanualdelineationofthesameimage,respectively.

ASSD=

_|

1 S s

|

+

|

S g

|

i∈Ss d

(

i,S g

)

+ i∈Sg d

(

i,S s

)

(10)

where Ss and Sg represent the set of surface points of the

pla-centa segmented by algorithms and manual delineation respec-tively.d

(

i_,Sg

)

istheshortestEuclideandistancebetweenthepoint

iandthesurfaceSg.

Toevaluatetheintra-andinter-uservariability,weaskedeight usersto performthe segmentation taskindependently. Eachuser provided the scribbles for segmentation twice. The agreement between different segmentations was measured by Fleiss’ kappa

Slic-Seg Slic-Seg with Low Level Features Slic-Seg without CRF Geodesic Framework ID-GC User Interactions

User-provided Foreground User-provided Background Segmentation Result Ground Truth

(a) Effects of different scribble positions (b) Effects of different scribble lengths

Fig.5. Segmentationoftheplacentabydifferentmethodsinthestartslice.(a)showstheeffectsofdifferentscribblepositions.(b)showstheeffectsofdifferentscribble lengths.NotethebettersegmentationofSlic-Segcomparedtoothermethods.

coefficient(Fleiss, 1971 ):

κ

=P¯a−P¯e

1−_P¯_e (11)

whereP¯a is therelative observed agreement,andP¯e is the

hypo-thetical probability of chance agreement. P¯a and P¯e are averaged

resultsacrossallthepixels.

3.2.SinglevolumesegmentationusingSlic-Seg

WecomparedSlic-Segwithtwootherslice-by-slicepropagation implementationswhichusedanintensitydistributionbasedGraph Cuts(Boykov and Jolly, 2001 )(ID-GCPropagation)andaGeodesic Framework1_{(Bai and Sapiro, 2009 )(Geo-Propagation)respectively.}

ForID-GC,theparameter

λ

mentionedin(Boykov and Jolly, 2001 ) wasset as10. ForGeodesicFramework, there was noparameter tunedbytheuser.Duringthepropagation, theyimplementedthe samemorphologicaloperationsasin Section 2.1.2 ontheobtained labelofoneslicetogeneratehardconstraintforthenextslice au-tomatically.Comparisons are also madebetween Slic-Seg andits threevariations: offlineSlic-Seg, Slic-Seg usinglow-level features andSlic-Seg withoutCRF.Allthesemethods usedthesame user-providedscribblesinthestartslices.

3.2.1. Segmentationinthestartslice

Fig. 5 showsexamplesofinteractive segmentation inthestart slicefromtwopatients. SinceSlic-SegandofflineSlic-Segarethe sameinthestartslice, weomittheofflineSlic-Seghere. Fig. 5 (a) showsthe results withdifferent scribble positions. It can be ob-served that with the given scribbles, Slic-Seg has the best seg-mentation accuracy. In addition, it is less sensitive to the posi-tion ofscribbles than other methods. Fig. 5 (b) shows theeffects of differentscribble lengths. Scribblesin the second column are

1_{Implementationfrom:http://www.robots.ox.ac.uk/}∼_{vgg/software/iseg/}

extendedfromthatinthefirstcolumn.Slic-Segcontinuesto pro-vide the best accuracy. Other methods have an improved perfor-mancewiththeextended scribbles,buttheystill havesome mis-segmentations, which require more user interactions to be cor-rected. This illustrates that Slic-Seg requires lessscribbles to get good segmentation in thestart slicethan other compared meth-ods.

3.2.2. Segmentationduringpropagation

Fig. 6 showsan exampleofthepropagationofdifferent meth-ods withthe same user inputs(scribble length: 495 mm) in the startslice(S0). Sirepresentstheithslicefollowingthestart slice.

In Fig. 6 ,thoughagoodsegmentationisobtainedinthestartslice duetoan extensiveset ofscribbles, theerrorsof offlineSlic-Seg, Geo-PropagationandID-GCPropagationbecomeincreasinglylarge duringthe propagation. ForSlic-Seg withlow-level features, in a slicethatisclosetothestartslice(e.g.i≤6), itcanobtaingood results. When a newslice isfurther away (e.g. i ≥ 12) fromthe start slice, it fails to track the placenta with high accuracy. For Slic-SegwithoutCRF,theperformance fluctuatesduringthe prop-agation.Incontrast,Slic-Seg hasa morestableandhigher perfor-mance.

Fig. 7 showstheDicecoefficientandASSDforeachsliceinone volumetricimagewhichwassegmentedby alltheusers.Foreach slice,weuseerrorbarstoshowthefirstquartile,medianandthe third quartileof theDice coefficient andASSD. Fig. 7 showsthat Slic-Seg andits variationshavea better performance inthe start sliceandduringthepropagationthanGeo-PropagationandID-GC Propagation. Offline Slic-Seg andSlic-Seg withlow-level features haveadecreasedaccuracyinremoteslices.Thefluctuating perfor-manceofSlic-SegwithoutCRFisalsoobviousin Fig. 7 .The com-parison showsthat Slic-Segoutperforms other methods. In addi-tion,thelower dispersionofSlic-Seg indicatesreducedvariability betweenusers.

Slic-Seg Slic-Seg with Low-level Features

Slic-Seg without CRF Offline Slic-Seg ID-GC Propagation Geo-Propagation S0 S3 S6 S9 S12 S15

User-provided Foreground User-provided Background Segmentation Result Ground Truth

Fig.6. Propagationinasinglevolumesegmentationofdifferentmethodswiththesamestartsliceandscribbles.Sirepresentstheithslicefollowingthestartslice(S0).

ScribblesinS0areextensiveandallmethodshaveagoodsegmentationinS0.Duringthepropagation,onlySlic-Segkeepsahighperformance.

Fig.7. Evaluationonsegmentationofasinglevolumewithscribblesgivenby8usersintermsofDice(left)andASSD(right)ineachslice.Sliceindex0indicatesthestart slice.

3.2.3. Interactivityanduservariability

Wealsomeasuredtheeffectsofscribblelengthontheaccuracy for segmentation of the total volume. During the user’s drawing scribbles, the orderofpointsonthe scribblesforforeground and backgroundwasrecorded,andtheserecordedscribbleswereused sequentially andincrementally for segmentation, withthe length changingfrom50mmto550mm.Theresultisshownin Fig. 8 .It can beseen thatSlic-Seg achievedahigheraccuracythan others, with its Dice and ASSD plateauing when the length of scribbles wasextended to around200–300mm. Fig. 8 alsoshowsthe use ofonlinetrainingofRF,high-level featuresandCRFimprovedthe accuracy.

Since the number of slices containing the placenta varies among different volume images, we measured runtime of the propagation-basedsegmentation in terms ofthe averageruntime forpropagationperslice,whichisdefinedastheratioofthetotal propagationtimeforthevolumeto thenumberofslices contain-ing theplacentainthat volume.Thetime consumption by differ-ent algorithms is listed in Table 1 . Note that the feature extrac-tions for Slic-Seg and its variations are implemented on a GPU,

and the propagations of all the methods are implemented on a CPU. Table 1 shows ID-GC Propagation has the smallest runtime andSlic-Seg hasalargerruntimewhichis1.05 _±0.13s perslice butstillacceptable.

The mean value andstandard deviationof Dice andASSD, as well astheintra-andinter-user Fleiss’kappa coefficient are pre-sentedin Table 2 ,which showsa low intra- andinter-user vari-ability.Thequantitativemeasurementacrossalltheuserswas0.82

± 0.02interms ofDice, and2.67 ± 0.63 mmin termsof ASSD. Inaddition,the intra-user

κ

rangedfrom0.931 to0.949,andthe inter-user

κ

was0.932,whichindicates ourinteractive segmenta-tionmethodhasahighintra-andinter-useragreementwithalow variability.

3.3.Refinementbasedonco-segmentationofmultipleimages

After the two volume images acquired in axial and sagit-tal views of one patient were segmented by single volume Slic-Seg respectively, they were co-segmented by our proposed 4D probability-based refinement (4D PR) using Graph Cuts. We

Fig.8. ThechangeofDice(left)andASSD(right)withincreasinglengthofscribblesthatwereprovidedinthestartslice.Theperformancewasevaluatedforthe segmen-tationofasinglevolumewithscribblesgivenby8users.

Table1

Averageruntimeperslice(inseconds)forthepropagationusingdifferentmethods.ThefeatureextractionsforSlic-Seganditsvariations areGPU-based,andthepropagationsofallthemethodsareCPU-based.

Slic-Seg OfflineSlic-Seg Slic-Segwithlow-levelfeatures Slic-SegwithoutCRF ID-GCPropagation Geo-Propagation

1.05 ± 0.13 0.84 ± 0.06 0.55 ± 0.10 0.93 ± 0.08 0.12 ± 0.04 0.61 ± 0.07

Table2

Intra-and inter-operator variabilityofSlic-Segfor segmentation of volume images. κ is the Fleiss’s kappacoefficientinEq.(11).

User Dice ASSD(mm) κ 1 0.81 ± 0.02 2.73 ± 0.62 0.931 2 0.82 ± 0.03 2.57 ± 0.60 0.936 3 0.81 ± 0.03 2.75 ± 0.61 0.949 4 0.80 ± 0.03 2.81 ± 0.73 0.941 5 0.82 ± 0.02 2.58 ± 0.61 0.948 6 0.82 ± 0.02 2.63 ± 0.61 0.945 7 0.82 ± 0.02 2.61 ± 0.74 0.941 8 0.81 ± 0.03 2.76 ± 0.67 0.936 All 0.82 ± 0.02 2.67 ± 0.63 0.932

compareitwiththreevariations:3Dprobability-basedrefinement (3DPR)using GraphCuts, 3D intensity-based refinement(3D IR) and4D intensity-based refinement(4D IR)usingGraphCuts.The 3Dmethodsonlyconsiderasinglevolumeforrefinement,andthe intensity-basedmethodsdefine theinter-sliceandinter-image bi-narytermbasedonpixelintensityratherthanprobability.

3.3.1. Refinementresults

Fig. 9 showsanexampleoftheinitialsegmentationbySlic-Seg anditsrefinedresultsby3D/4DIR/PRrespectively.ImageI1andI2 areacquiredintwoviewsfromthesamepatient.I1hasahigh res-olutioninaxialviewwithalowresolutioninsagittalview.I2 has a low resolution inaxial view with a high resolution in sagittal view.ThefirstcolumnshowstheinitialsegmentationofI1 andI2, bothofwhichhavesomeerrorscomparedwiththegroundtruth. Thefollowingcolumnsshowtherefinedsegmentationresults.The darkorange arrows in each rowindicate the difference between theinitial segmentationandtherefinedresults.Forthe intensity-basedmethods, although some errors inthe initial segmentation werecorrected(thedarkorangearrowsinthelastrow),additional mis-segmentationswere introduced (highlighted by the cyan ar-rows).Thusthesetwomethodsfailedtoimprovethesegmentation accuracy.Incontrast,theprobability-basedmethodsimprovedthe

segmentation withoutcausing extraerrors.The last two columns show4DPRoutperforms3DPRintherefinement.

We compared the above mentioned refinement methods, as well as four additional popular interactive segmentation meth-ods forsinglevolume segmentation:ITK-SNAP(Yushkevich et al., 20 06 ), GeoS (Criminisi et al., 20 08 ), 3D ID-GC(Boykov and Jolly, 2001 )andGrowCut(Kikinis and Pieper, 2011 ).Forthesefour meth-odsthatarenotdesignedtoacceptscribblesonlyinasingleslice, scribblesareprovidedin3D, andafterthe segmentationtheuser canprovidemorescribblesandexecutethealgorithmagainto cor-recttheresult.We taketheresultsafterseveralroundsof correc-tionwhentheuserconfirmstheyareacceptable.

Quantitative evaluation are shownin Table 3 , which lists the evaluation results of images acquired in axial and sagittal views respectively.TheresultshowsSlic-Segwith4DPRhasabetter per-formancethanotherinteractivesegmentationalgorithms.Interms of the refinement, 3D IR and 4D IR achieved lower Dice values and higher ASSD values compared with the initial segmentation given by singlevolume Slic-Seg, which indicates that they failed toimprovethesegmentation accuracy.Incontrast,higher accura-ciesthansinglevolumeSlic-Segwereachievedbythe probability-based refinement methods,and 4D PR hada better performance than3DPR.Thepvaluebetweenthemis6.9e−11 _{intermsofDice} and1.1e−10intermsofASSD.

4. Discussion

Intermsoftheinteractivesegmentation withpropagation, the experiments show that Slic-Seg achieved higher accuracy than GeodesicFramework andGraphCuts basedon intensity distribu-tions when scribbleswere givenonly ina singleslice. The latter two methods rely on gradient or intensityinformation to model theplacentaandbackground,whichmaynot beaccurate enough in fetal MRI images with poor 3D quality. Slic-Seg uses high-level features ofmultiple aspects includingintensity, textureand waveletcoefficients. Thisprovides a betterdescription ofthe dif-ferences between the placentaand background, which is further validatedbythecomparisonwithSlic-Segwithlow-levelfeatures.

Axial View of I1

Sagittal View of I1

Axial View of I2

Sagittal View of I2

Slic-Seg Slic-Seg + 3D IR Slic-Seg + 4D IR Slic-Seg + 3D PR Slic-Seg + 4D PR

Segmentation Result Ground Truth Difference between Methods Mis-segmentation Introduced by Intensity-based Refinement

Fig.9. ComparisonofinitialsegmentationbysinglevolumeSlic-Segandrefinementby3D/4DGraphCutsusingintensity/probabilityrespectively.I1andI2areacquiredin

twoviewsfromthesamepatientwithcomplementaryresolution.IR(PR)referstointensity(probability)-basedrefinement.

Table3

Quantitativeevaluationofrefinementmethodsbasedonco-segmentationandcomparisonbetweenpopularinteractive segmen-tationalgorithms.Theaxialviewimageshaveahighaxial-viewresolutionandalowsagittal-viewresolution.Thesagittalview imageshavealowaxial-viewresolutionandahighsagittal-viewresolution.Thebestvalueineachcolumnishighlightedbybold.

Methods Axialview Sagittalview

Dice ASSD(mm) Time(s) Dice ASSD(mm) Time(s)

ITK-SNAP 0.79 ± 0.03 2.94 ± 0.72 118.83 ± 15.35 0.81 ± 0.02 2.73 ± 0.48 106.94 ± 16.23 GeoS 0.81 ± 0.03 2.68 ± 0.67 166.72 ± 49.37 0.79 ± 0.03 3.40 ± 0.76 101.83 ± 38.84 3DID-GC 0.79 ± 0.02 3.19 ± 0.61 188.05 ± 30.19 0.79 ± 0.03 3.57 ± 0.96 97.58 ± 10.78 GrowCut 0.80 ± 0.03 2.78 ± 0.66 170.56 ± 23.18 0.78 ± 0.03 2.99 ± 0.85 120.38 ± 11.67 Slic-Seg 0.82 ± 0.02 2.35 ± 0.47 81.61±17.22 0.81 ± 0.03 2.84 ± 0.54 47.78±13.59 Slic-Seg+3DIR 0.80 ± 0.03 3.28 ± 0.62 109.96 ± 21.48 0.80 ± 0.04 3.29 ± 0.72 64.84 ± 14.94 Slic-Seg+4DIR 0.81 ± 0.03 3.00 ± 0.46 121.94 ± 23.74 0.81 ± 0.03 2.95 ± 0.58 88.11 ± 16.61 Slic-Seg+3DPR 0.87 ± 0.03 2.16 ± 0.26 107.14 ± 23.07 0.86 ± 0.02 2.41 ± 0.45 61.82 ± 15.85 Slic-Seg+4DPR 0.89±0.02 1.89±0.39 117.82 ± 25.53 0.88±0.02 1.99±0.38 83.98 ± 17.70

Inaddition,the onlinetrainingofRF overcomesthepotential ap-pearancechangewhen theslice-by-slicesegmentation propagates to aremote slice, andthe employmentofCRF addressesthe dis-connectivityoflabelsresultingfromRFpredictionbyspatial regu-larization.ThesefactorsallowSlic-Segtohaveagoodperformance duringthepropagation.Althoughtheuseofhigh-levelfeatures in-creasesthecomputationaltime,theaverageruntimeofSlic-Segon one slice is 1.05 s, which is acceptable for interactive segmenta-tion.Inaddition,itispossibletopre-computethefeaturessothat runtimecanbereducedduringthepropagation. Inthispaper,the high-level features aredesignedmanually,andtheyare not guar-anteedtobethemosteffectivefeaturesfordistinguishingthe pla-centa and the background.To improve the segmentation further, usingdeeplearning(Sermanet et al., 2013; Roth et al., 2015 ) asa featureextractionmethodmightbe helpfulsinceitcanlearn fea-turesautomaticallywithlargeamountoftrainingdata.

Theexperimentsshowthatwiththeincreaseofscribblelength, bettersegmentationswereachievedbyallthecomparedmethods, butSlic-Seg requiresfewer user interactions toreach theplateau accuracy.Thisresultsintheminimizationofuserinteractions, con-sideringitonlyneedsuser-providedscribblesinthestartslice.

Be-sides, Table 2 shows high intra- and inter-operator agreements, whichindicatesalowvariabilitywithinandbetweenusers.

There are three reasons to refine the segmentation results of singlevolumeSlic-Seginourapplication.First,thelargeinter-slice spacingandinhomogeneousappearance betweenslicesmake the accurate segmentation hard to achieve froma singlevolume im-agedata.Second,Slic-Segdoesnottakeintoaccounttheinter-slice connectivitybyapplyingCRFonlyin2Dslices,whichmayleadto jaggedsurfacesin3Dspace.Inaddition,post-segmentation refine-mentcanbehelpfulconsidering errorsintheautomatic propaga-tion.Wejustusedtheskeletonoftheforegroundanderoded back-groundinonesegmentedslicetoguidethesegmentationofa fol-lowingslice,whichmakestheerrorinonesliceislesslikelytobe propagatedtoafollowingslice.Asisshownin Fig. 7 ,the propaga-tionofSlic-Segisrobustinmostslices,andtheaccumulatederror becomeslargeonlyinterminalslicesduetoalargechangeofthe shapeoftheplacentabetweentwoslices.Wehaveshownthatour automaticrefinement leveragingmultiplevolumesandrelying on 4DGraphCutscanreduceerrorsrelatedtotheinitialpropagation. Tofurthercorrectthesegmentations,userfeedbackguided refine-mentwillbeconsideredinfuturework.

The refinement methodcombined the complementary resolu-tionof images acquiredin differentviews, andreduced the seg-mentationerrorsbyincorporatinginter-sliceandinter-image con-sistency.The experimentshowsintensity-based 3Dand4D Graph Cuts did not improvethe segmentation accuracy, indicating sole intensity information is not sufficient for good segmentation. In contrast, by defining the inter-slice and inter-image binary en-ergy based on probability learned from RF using high-level fea-tures, large improvement of accuracy was achieved as shown in Table 3 . In addition, the 4D PR achieved a better improvement in the refinement step than 3D PR, which demonstrates the co-segmentationoftwoimageslead tohigheraccuracythanusinga singlevolumeimage.Inourcurrentco-segmentation implementa-tion,theN3 neighborhoodisdefinedbased onthenearest voxels fromdifferentvolumes.Consideringthepotential alignmenterror, themethodmightbeimprovedbydefiningtheinter-image neigh-borhoodbasedonthe voxelsina localareaweightedbythe dis-tanceorsimilarity,thusmutualinformationorpatch-based analy-sis(Bai et al., 2013 )mightbehelpfulforamorerobustresult.Note thatalthoughtwoimagesareco-segmentedintheexperiment,the proposedmethod isformulated (Eq. (6) ) sothat it candeal with moreimagevolumes.

5. Conclusion

We presented an interactive, learning-based method for the segmentationofthe placentafrommotioncorruptedfetal MRI in multipleviews. Todealwithpoorimage qualitycausedbysparse acquisition and inter-slice motion, the proposed Slic-Seg com-bineshigh-levelfeatures,RandomForestsandConditionalRandom Fields,whichrequires minimal userinteractions toget good seg-mentation results. The segmentation was further refined by co-segmentationof imagesfromdifferent viewsusinga probability-based4DGraphCutsmethod.Theresultsdemonstratedthewhole segmentationframework hasagoodinteractivitywithstable per-formance between and within users, and large improvement of accuracybenefiting fromthe co-segmentation.Therefore, our ap-proachmightbesuitableforsegmentationoftheplacentain plan-ningsystemsforfetalandmaternalsurgery,andforrapid charac-terizationoftheplacentabyMRI.Itsfirstclinicalapplicationmight befetoscopicplacementoptimizationinthetreatmentof twin-to-twintransfusionsyndrome.

Acknowledgments

This work was supported through an Innovative Engineering for Health award by the Wellcome Trust (WT101957 ); Engineer-ingandPhysicalSciencesResearchCouncil(EPSRC)(NS/A000027/1, EP/H046410/1, EP/J020990/1, EP/K005278), the National Institute forHealthResearch University CollegeLondon Hospitals Biomed-icalResearchCentre(NIHRBRCUCLH/UCLHigh ImpactInitiative), aUCLOverseasResearchScholarshipandaUCLGraduateResearch Scholarship.

References

Anquez,J.,Angelini,E.D.,Bloch,I.,2009.Automaticsegmentationofheadstructures onfetalMRI.In:Proceedings-2009IEEEInternationalSymposiumon Biomed-icalImaging:FromNanotoMacro,ISBI2009.Boston,MA,USA,pp.109–112. doi:10.1109/ISBI.2009.5192995.

Bai,W.,Shi,W.,O’Regan,D.P.,Tong,T.,Wang,H.,Jamil-Copley,S.,Peters,N.S., Rueck-ert,D.,2013.Aprobabilisticpatch-basedlabelfusionmodelformulti-atlas seg-mentationwithregistrationrefinement:ApplicationtocardiacMRimages.IEEE Trans.Med.Imag.32(7),1302–1315.doi:10.1109/TMI.2013.2256922.

Bai,X.,Sapiro,G.,2009.Geodesicmatting:Aframeworkforfastinteractiveimage andvideosegmentationandmatting.Int.J.Comput.Vis.82(2),113–132.doi:10. 1007/s11263-008-0191-z.

Batra, D., Kowdle, A.,Parikh, D., Luo, J., Chen,T., 2010. iCoseg:Interactive co-segmentation with intelligentscribbleguidance. In:Proceedings ofthe IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.3169–3176.doi:10.1109/CVPR.2010.5540080.

Boykov,Y.,Jolly,M.-P.,2001.Interactivegraphcutsforoptimalboundary&region segmentationofobjectsinN-Dimages.In:ICCV2001,1,pp.105–112.doi:10. 1109/ICCV.2001.937505.

Breiman,L.,2001.Randomforests.Eur.J.Math.45,5–32.

Chalouhi,G.E.,Marangoni,M.A.,Quibel,T.,Deloison,B.,Benzina,N.,Essaoui,M.,Al Ibrahim,A.,Stirnemann,J.J.,Salomon,L.J.,Ville,Y.,2013.Activemanagementof selectiveintrauterinegrowthrestrictionwithabnormalDopplerin monochori-onicdiamniotictwin pregnanciesdiagnosedinthesecondtrimesterof preg-nancy.Prenat.Diagn.33(2),109–115.doi:10.1002/pd.4031.

Criminisi,A.,Sharp,T.,Blake,A.,2008.GeoS:geodesicimagesegmentation.In:ECCV 2008.,5302,pp.99–112.

Deprest, J.A., Flake, A.W.,Gratacos, E., Ville, Y.,Hecher, K.,Nicolaides, K., John-son, M.P., Luks, F.I., Adzick, N.S.,Harrison, M.R., 2010. The making of fetal surgery.Prenat.Diagn.30(7),653–667.doi:10.1002/pd.2571.

Fleiss,J.L.,1971.Measuringnominalscaleagreementamongmanyraters..Psychcol. Bull.76(5),378–382.doi:10.1037/h0031619.

Freedman, D., Zhang, T., 2005. Interactive Graph Cut BasedSegmentation with ShapePriors.In:ComputerVisionandPatternRecognition,2005.CVPR2005. IEEE Computer SocietyConference on. IEEE, pp.755–762.doi:10.1109/CVPR. 2005.191.

Gao,Y.,Kikinis,R.,Bouix,S.,Shenton,M.,Tannenbaum,A.,2012.A3Dinteractive multi-objectsegmentationtoolusinglocalrobuststatisticsdrivenactive con-tours.Med.ImageAnalysis16(6),1216–1227.doi:10.1016/j.media.2012.06.002.

Gholipour,A.,Akhondi-Asl,A.,Estroff,J.A.,Warfield,S.K.,2012.Multi-atlas multi-shapesegmentationoffetalbrainMRIforvolumetricandmorphometric anal-ysisofventriculomegaly.NeuroImage60,1819–1831.doi:10.1016/j.neuroimage. 2012.01.128.

Gholipour,A.,Estroff,J.A.,Warfield,S.K.,Member,S.,2010.Robustsuper-resolution volumereconstructionfromsliceacquisitions:applicationtofetalbrainMRI. Med.Imag.IEEETrans.29(10),1739–1758.

Grady,L.,Schiwietz,T.,Aharon,S.,Westermann,R.,2005.Randomwalksfor inter-activeorgansegmentationintwoandthreedimensions:implementationand validation.In:MICCAI2005,3750,pp.773–780.doi:10.1007/11566489_95.

Guo,F.,Yuan,J.,Rajchl,M.,Svenningsen,S.,PICapaldi,D.,Sheikh,K.,Fenster,A., Parraga,G.,2015.Globallyoptimalco-segmentationofthree-dimensional pul-monary1Handhyperpolarized3HeMRIwithspatialconsistenceprior.Med. ImageAnal.23(1),43–55.doi:10.1016/j.media.2015.04.001.

Habas,P.A.,Kim,K.,Corbett-Detig,J.M.,Rousseau, F.,Glenn, O.A.,Barkovich,A.J., Studholme,C.,2010.AspatiotemporalatlasofMRintensity,tissueprobability andshapeofthefetalbrainwithapplicationtosegmentation.NeuroImage53, 460–470.doi:10.1016/j.neuroimage.2010.06.054.

Kainz, B., Malamateniou, C., Murgasova, M., Keraudren, K., Rutherford, M., Haj-nal,J.V.,Rueckert,D.,2014.Motioncorrected3Dreconstructionofthefetal tho-raxfromprenatalMRI.In:MedicalImageComputingand Computer-Assisted Intervention-MICCAI2014,pp.284–291.

Kainz,B.,Steinberger,M.,Wein,W.,Kuklisova-murgasova,M.,Malamateniou,C., Ker-audren,K.,Torsney-weir,T.,Rutherford,M.,Aljabar,P.,Hajnal,J.V.,Rueckert,D., 2015.Fast volumereconstructionfrommotioncorrupted stacksof2D slices. IEEETMI34(9),1901–1913.

Kikinis,R.,Pieper,S.,2011.3DSlicerasatoolforinteractivebraintumor segmen-tation.In:ProceedingsoftheAnnualInternationalConferenceoftheIEEE En-gineeringinMedicineandBiologySociety,EMBS,pp.6982–6984.doi:10.1109/ IEMBS.2011.6091765.

Mazouni,C.,Gorincour,G.,Juhan,V.,Bretelle,F.,2007.Placentaaccreta:areviewof currentadvancesinprenataldiagnosis.Placenta28(7),599–603.doi:10.1016/j. placenta.2006.06.011.

Modat,M.,Ridgway,G.R.,Taylor,Z.a.,Lehmann,M.,Barnes,J.,Hawkes,D.J.,Fox,N.C., Ourselin,S.,2010.Fastfree-formdeformationusinggraphicsprocessingunits. Comput.MethodsProgramsBiomed.98(3),278–284.doi:10.1016/j.cmpb.2009. 09.002.

Ourselin,S.,Roche,A.,Subsol,G.,Pennec,X.,Ayache,N.,2001.Reconstructinga3D structurefromserialhistologicalsections.ImageVis.Comput.19(1-2),25–31. doi:10.1016/S0262-8856(00)00052-4.

Park,S.H.,Gao,Y.,Shi,Y.,Shen,D.,2014.Interactiveprostatesegmentationbasedon adaptivefeatureselectionandmanifoldregularization.Mach.Learn.Med.Imag. 8679,263–270.

Pratt,R.,Deprest,J.,Vercauteren,T.,Ourselin,S.,David,A.L.,2015.Computer-assisted surgicalplanningandintraoperativeguidanceinfetalsurgery:asystematic re-view.Prenat.Diagn.35,1–8.doi:10.1002/pd.4660.

Roth,H.R.,Lu,L.,Farag,A.,Shin,H.-C.,Liu,J.,Turkbey,E.B.,Summers,R.M.,2015. DeepOrgan:multi-leveldeep convolutionalnetworks forautomatedpancreas segmentation.In:MedicalImageComputingandComputer-Assisted Interven-tionMICCAI2015,9349,pp.556–564.doi:10.1007/978-3-319-24553-9.

Rueckert,D.,Sonoda,L.I.,Hayes,C.,Hill,D.L.,Leach,M.O.,Hawkes,D.J.,1999. Non-rigidregistrationusingfree-formdeformations:applicationtobreastMR im-ages..IEEETrans.Med.Imag.18(8),712–721.doi:10.1109/42.796284.

Saffari, A., Leistner,C., Santner, J., Godec,M., Bischof, H.,2009. Online random forests. In: ICCV Workshops 2009, pp. 1393–1400. doi:10.1109/ICCVW.2009. 5457447.

Santner,J.,Unger,M.,Pock,T.,Leistner,C.,Saffari,A.,Bischof,H.,2009.Interactive texturesegmentationusingrandomforestsandtotalvariation.In:BMVC2009, pp.66.1–66.12.doi:10.5244/C.23.66.

Sermanet, P., Eigen,D.,Zhang, X., Mathieu,M.,Fergus,R.,LeCun,Y.,2013. Over-Feat:Integrated Recognition, Localization and Detectionusing Convolutional Networks.arXiv1–16.

Shi,W.,Zhuang,X.,Wolz,R.,Simon,D.,Tung,K.,Wang,H.,Ourselin,S.,Edwards,P., Razavi,R.,Rueckert,D.,2012.Amulti-imagegraphcutapproachforcardiac im-agesegmentationanduncertaintyestimation.In:StatisticalAtlasesand Compu-tationalModelsoftheHeart.ImagingandModellingChallenges,7085.Springer BerlinHeidelberg,pp.178–187.doi:10.1007/978-3-642-28326-0_18.

Spencer,R.N.,Carr,D.J.,David,A.L.,2014.Treatmentofpoorplacentationandthe preventionofassociatedadverseoutcomes–whatdoesthefuturehold?Prenat. Diagn.34(7),677–684.doi:10.1002/pd.4401.

Veeraraghavan,H.,Miller,J.V.,2011.Activelearningguidedinteractionsfor consis-tentimagesegmentationwithreduceduserinteractions.In:Proceedings- In-ternationalSymposiumonBiomedicalImaging,pp.1645–1648.doi:10.1109/ISBI. 2011.5872719.

Vezhnevets,V.,Konouchine,V.,2005.GrowCut-interactivemulti-labelN-Dimage segmentationbycellularautomata.In:Graphicon,pp.150–156.doi:10.1016/j. ajodo.2004.07.036.

Wang,B.,Liu,W.,Prastawa,M., Irimia,A.,Vespa,P.M., Horn,J.D.V.,Fletcher,P.T., Gerig,G.,2014.4Dactivecut:aninteractivetoolforpathologicalanatomy mod-eling.In:BiomedicalImaging(ISBI),2014IEEE11th InternationalSymposium on,005149,pp.529–532.

Wang,G.,Zuluaga,M.A.,Pratt,R.,Aertsen,M.,David,A.L.,Deprest,J.,Vercauteren,T., Ourselin,S.,2015.Slic-Seg:slice-by-slicesegmentationpropagationofthe pla-centainfetalMRIusingone-planescribblesand onlinelearning.In:MICCAI 2015,pp.29–37.

Wang,L.,Shi,F.,Yap,P.T.,Gilmore,J.H.,Lin,W.,Shen,D.,2012.4Dmulti-modality tissuesegmentation ofserialInfant Images.PLoSONE7(9),e44596. doi:10. 1371/journal.pone.0044596.

Xu,C.,Prince,J.L.,1998.Snakes,shapes,andgradientvectorflow.IEEETIP7(3), 359–369.doi:10.1109/83.661186.

Yushkevich, P.A.,Piven, J., Hazlett, H.C.,Smith, R.G., Ho, S., Gee,J.C., Gerig, G., 2006.User-guided3D activecontoursegmentation ofanatomicalstructures: significantlyimprovedefficiencyandreliability.NeuroImage31(3),1116–1128. doi:10.1016/j.neuroimage.2006.01.015.

Zhao,F.,Xie,X.,2013.Anoverviewofinteractivemedicalimagesegmentation.Ann. BMVA2013(7),1–22.