Reconstructing Surfaces by Volumetric Regularization Using Radial Basis Functions

City, University of London Institutional Repository

Citation

:

Dinh, H. Q., Turk, G. and Slabaugh, G. G. (2002). Reconstructing Surfaces by

Volumetric Regularization Using Radial Basis Functions. IEEE Transactions on Pattern

Analysis and Machine Intelligence, 24(10), pp. 1358-1371. doi:

10.1109/TPAMI.2002.1039207

This is the accepted version of the paper.

This version of the publication may differ from the final published

version.

Permanent repository link:

http://openaccess.city.ac.uk/4701/

Link to published version

:

http://dx.doi.org/10.1109/TPAMI.2002.1039207

Copyright and reuse:

City Research Online aims to make research

outputs of City, University of London available to a wider audience.

Copyright and Moral Rights remain with the author(s) and/or copyright

holders. URLs from City Research Online may be freely distributed and

linked to.

Reconstructing Surfaces By Volumetric Regularization Using Radial Basis Functions

Huong Quynh Dinh 1

, Greg Turk 1

, and Greg Slabaugh 2

1

Graphics,Visualization,andUsabilityCenter 2

CenterforSignalandImageProcessing

CollegeofComputing SchoolofElectricalandComputerEngineering GeorgiaInstituteofTechnology GeorgiaInstituteofTechnology

Abstract|We present a new method of surface recon-structionthatgeneratessmoothand seamless modelsfrom sparse, noisy, non-uniform, and lowresolutionrange data. Data acquisitiontechniquesfrom computer vision, such as stereo range imagesand space carving, produce 3D point setsthatareimpreciseandnon-uniformwhen comparedto laser oroptical range scanners. Traditionalreconstruction algorithmsdesignedfordense andprecisedatado not pro-duce smoothreconstructions when applied to vision-based data sets. Our method constructs a 3D implicit surface, formulatedasasumofweightedradialbasisfunctions. We achievethreeprimaryadvantagesoverexistingalgorithms: (1) the implicit functions we construct estimate the sur-facewellinregionswherethereislittledata;(2)the recon-structed surface is insensitive to noise in data acquisition because we can allow the surface to approximate, rather thanexactlyinterpolate,thedata;and(3)thereconstructed surfaceis locallydetailed,yetglobally smooth, because we use radial basis functions that achieve multiple orders of smoothness.

Index terms: regularization, surfacetting, implicit func-tions,noisyrangedata

I. Introduction

The computervision community hasdeveloped numer-ousmethodsofacquiringthreedimensional datafrom im-ages. Someofthesetechniquesincludeshapefromshading, depthapproximationfromapairofstereoimages,and vol-umetricreconstructionfromimagesatmultipleviewpoints. The advantageof these techniques is that they use cam-eras, which are inexpensive resources when compared to laser andoptical scanners. Because ofthe aordabilityof cameras, these vision-basedtechniqueshavethepotential toenablethecreationofdigitalmodelsbyhomecomputer users who may not have professionalCAD training. On the other hand, models in popular use in the entertain-mentindustry(animationandgamingapplications),video and image editing, and computer graphics research come from denselaser scans or medicalscans, not from vision-basedtechniques. Therearesignicantdierencesinterms of quality and accuracy between data sets obtainedfrom activescanning technology (e.g. optical, laser, and time-of-ight range scanners) and passivescanning technology (e.g. shapefromshading,voxelcoloring)thatuseonly im-agesandcameracalibrationtoobtain3Dpointsets. Many ofthewell-knownandoftenusedreconstructionalgorithms weredesignedtogeneratesurfaces fromdenseandprecise data such as those obtainedfrom active scanners. These methodsarenotrobusttothechallengesposedbydata

ob-methodistobeabletoreconstructsmoothandcontinuous surfacesfrom themorechallengingvision-baseddatasets. Thenew approach presentedin this paperconstructsa 3Dimplicitfunctionfromvision-basedrangedata. Weuse ananalyticalimplicitrepresentationthatcansmoothly in-terpolatethesurfacewherethere islittleornodata, that iscompactwhencomparedtodiscretevolumetricdistance functions,and that can either approximate orinterpolate the data. The resultingsurfaces are inherentlymanifold, smooth,andseamless. Implicitsurfaces arewell-suitedfor operations such as collision detection, morphing, blend-ing,andmodelingwithconstructivesolidgeometrybecause theyareformulatedasasingleanalyticalfunction,as op-posed to a piecewise representation such as a polygonal model or a dense volumetric data set. Implicit surfaces canalsoaccuratelymodelsoftandorganicobjectsandcan easilybeconvertedtoapolygonalmodelbyiso-surface ex-traction.

Weconstructanimplicitsurfaceusingvolumetric regu-larization. This approach is based on thevariational im-plicitsurfacesofTurkandO'Brien[48]. Ourimplicit func-tion consists of a sum of weighted radial basis functions that are placed at surface and exteriorconstraint points denedbythedataset. Theweightsofthebasisfunctions aredeterminedbysolvingalinearsystemofequations. We canapproximatethedatasetbyrelaxingthelinearsystem through volumetricregularization. The ability to choose whether to approximate or interpolate the data is espe-ciallyadvantageousinthepresenceofnoise. Surfacedetail andsmoothnessareobtainedbyusingbasisfunctionsthat achievemultipleordersofsmoothness.

Our main contributions are: (1) introducing the useof variationalimplicitsurfacesforsurfacereconstructionfrom vision-basedrangedata,(2)theapplicationofanewradial basisfunctionthatachievesmultipleordersofsmoothness, (3)enhancementof nedetailand sharpfeaturesthatare often smoothed-over by the variational implicit surfaces, and (4) constructionof approximating, ratherthan inter-polatingsurfacestoovercomenoisydata.

our framework. Results from synthetic range imagesand fromrealspacecarveddatasetsareshownin SectionVII.

II. Related Work

Ourapproachtosurfacereconstructioncanbecompared to previousworksin theareasofshaperepresentation, re-construction, smoothing, and surfaceregularization. The large number of published methods in these areas makes it nearly impossible to perform a comprehensive survey. Instead, we describe some of the more well-known ap-proaches, with a bias towards those more closely related to ourownapproach. TableI summarizesthecomparison betweenrelatedreconstructionalgorithmsandourown.

A. SurfaceRepresentation

Three general classesof surface representationsinclude discrete, parametric, and implicit approaches. Discrete forms, such asacollectionofpolygonsandpointsamples, arethemostwidelyusedrepresentations. Theprimary dis-advantagesassociatedwiththemarethattheyareverbose, thattheycanonlyapproximatesmoothsurfaces,andthat they have xed resolution. In contrast, parametric sur-faces, such asB-splines and Bezier patches, may be sam-pled at arbitraryresolution and can be used to represent smooth surfaces. The main drawback of paramteric sur-faces is that several parametric patches need to be com-binedtoform aclosedsurface,resultingin seamsbetween the patches. Implicit representations,on the other hand, donotrequireseamstorepresentaclosedsurface.Implicit representationscome in bothanalyticaland discrete sam-pled forms. Analytical representations, such as our own, are more compact than sampled representations. Exam-plesofsampledimplicitfunctionsinclude griddedvolumes and octree representations such asthose used by Szeliski et al. [39],Friskenet al. [18],andCurlessandLevoy[12].

B. SurfaceReconstruction

Inthissection,wediscuss themorepopular reconstruc-tionalgorithms. Theshapereconstructionmethodswe de-scribeincluderangedatamergingandmeshreconstruction, regiongrowing,algorithmsbasedoncomputational geom-etry,andalgebraictting.

Although our work does not focus on reconstructing surfaces from dense and precise range data, methods that mergemultiplerangeimagesand reconstructsmooth meshesaddressissuessimilartoourown. Issuesthatarise in suchwork includemergingmultiple rangeimages, clos-ingofgapsinthereconstruction,andhandlingofoutliers. Curless and Levoy [12] and Hilton et al. [20] construct signed distance functions from the range images and ob-tain a manifold surface by iso-surface extraction. Soucy and Laurendeau [37] and Turk and Levoy [47] merge tri-angulations of the range points. Note that all of these methods require rangedata using structuredlightthat is muchmoreaccuratethancanbemeasuredpassivelyusing photographsalone.

Another approach is region growing, and examples

in-TangandMedioni'sworkontensorvoting[26,40]. Hoppe usesaplaneistted toaneighborhoodaroundeachdata point, providinganestimate ofthesurfacenormalfor the point. Thesurfacenormals are propagated using a mini-mal spanning tree,and then asigneddistance function is contructedinsmallneighborhoodsaroundthedatapoints. LeeandMedioni'stensorvotingmethod issimilarin that neighboringpointsareusedtoestimatetheorientationsof data points. The tensor is the covariance matrix of the normal vectors of a neighborhood of points. Each data pointvotesfortheorientationofotherpointsin its neigh-borhood using its tensor eld. In [40], the surface is re-constructed by growing planar, edge, and point features until they encounter neighboring features. Both methods described aboveare sensitiveto noiseinthe databecause theyrelyongood estimatesforthenormalvectorat each datapoint.

Several algorithms based on computational geometry constructacollectionofsimplexesthat form theshapeor surfacefrom a set of unorganized points. These methods exactlyinterpolatethedata|theverticesofthesimplexes consist of thegiven datapoints. A consequenceof thisis thatnoiseandaliasinginthedatabecomeembeddedinthe reconstructedsurface. Ofsuchmethods,threeofthemost successful areAlpha Shapes[15],theCrust algorithm[1], andtheBall-Pivotingalgorithm[4]. InAlpha shapes,the shapeiscarvedoutbyremovingsimplexesoftheDelaunay triangulationof thepointset. Asimplex is removedif its circumscribingsphereislargerthanthealphaball. Inthe Crust algorithm, Delaunay triangulation is performed on theoriginal set ofpointsalong withVoronoiverticesthat approximate the medial axisof theshape. The resulting triangulation distinguishes triangles that are part of the objectsurfacefrom those thatare ontheinteriorbecause interiortriangleshaveaVoronoivertexasoneoftheir ver-tices. Both the Alpha Shapesand Crust algorithmsneed nootherinformationthanthelocationsofthedatapoints andperformwellondenseandprecisedatasets. Theobject modelthattheseapproachesgenerate,however,consistsof simplexesthatoccurclosetothesurface. Thecollectionof simplexesisnotamanifoldsurface,andextractionofsuch a surfaceis a non-trivial post-processing task. The Ball-Pivoting algorithm is a related method that avoids non-manifoldconstructions bygrowingamesh from aninitial seedtriangle that is correctlyoriented. Starting with the seed triangle, a ball of specied radius is pivoted across edges ofeach trianglebounding thegrowingmesh. If the pivotedballhitsverticesthatarenotyetpartofthemesh, a new triangle is instantiated and added to the growing mesh. In Figure 1 (right panel), the Crust algorithm is applied to real range data obtained from the generalized voxelcoloringmethodof[11]. Althoughthegeneralshape ofthetoydinosaurisrecognizable,thesurfaceisroughdue tothenoisynatureoftherealrangedata.

TABLEI

ComparisonofRelatedWorks

Methods Shape Arbitrary Complex Robust FillsGaps Representation Topology Models toNoise

DistanceFields discrete yes yes no no

RegionGrowing piecewisecontinuous yes yes no no Computational piecewisecontinuous yes yes no no Geometry

AlgebraicMethods analytical yes no yes yes

Deformable analytical no no yes yes

Superquadrics

Volumetric analytical yes yes yes yes

Regularization

function ormany functions that are pieced together. Ex-amplesofreconstructionbyglobalalgebraicttingarethe worksofTaubin[41,42],GotsmanandKeren[22,23],and Blane et al. [5]. Taubin ts a polynomial implicit func-tiontoapointsetbyminimizingthedistancebetweenthe pointsetandtheimplicitsurface. In[41],Taubindevelops arstorder approximationof theEuclidean distanceand improvesthe approximationin [42]. Gotsman andKeren create parameterized families of polynomials that satisfy desirable properties, such as tness to the data or conti-nuitypreservation. Suchafamilymustbelargesothat it can includeasmanyfunctionsaspossible. Thistechnique leadsto anover-representationof thesubset, in that the resulting polynomial will often have more coeÆcients for which to solve than the simpler polynomials included in the subset, thus requiringadditional computation. Blane et al. performspolynomialttingofpointsonazerolevel setand(forstability)tspointsontwoadditionallevelsets closetothezerolevelset|oneinternalandoneexternal levelset. Theprimarylimitationofglobalalgebraic meth-ods is theirinability to reconstructcomplex models. The highest degree polynomialsthat have been demonstrated arearounddegree12,andthisisfartoosmalltorepresent complexshapes.

In[3],Bajajovercomesthecomplexitylimitationby con-structing piecewisepolynomialpatches(called A-patches) thatcombinetoformonesurface. BajajusesDelaunay tri-angulationtodividethepointsetintogroupsdelineatedby tetrahedrons. An A-patchisformedbyttingaBernstein polynomialtothedatapointswithineachtetrahedron. By constructingapiecewisesurface,Bajaj'sapproachlosesthe compactcharacteristicofaglobalrepresentation,and oper-ationssuchascollisiondetection,morphing,blending,and modeling with constructive solid geometry become more diÆcultto perform sincetherepresentationisnolongera singleanalyticalfunction.

Examples ofalgebraicmethods developed earlierin the visioncommunitythat providebothsmoothglobal tting and accurate local renement include the works of T er-zopoulos and Metaxas on deformable superquadrics [46] and Pentland and Sclaro on generalized implicit func-tions [32,34]. Both methods use superquadric ellipsoids as the global shapeand add local deformations to t the

constructed model into global parameters dened by the superquadric coeÆcients, and local displacements dened asalinearcombinationofbasisfunctions. Theglobaland local deformation parameters are solved using dynamics. Pentland and Sclarodene a generalizedimplicit model that consists of asuperquadric ellipsoid and amodal de-formationmatrix. Themodaldeformationparametersare foundbyiterativelyndingtheminimumRMSerrortothe data points. Theresidual errorafter thedeformation pa-rametershavebeenfoundareincorporatedintoa displace-ment map to bettert the data. As with mostalgebraic methods,thedrawbackofthesetechniquesistheirinability tohandlearbitrarytopology.

Ourapproachissimilartoglobalalgebraicttinginthat weconstructoneglobalimplicitfunction,althoughour ba-sis functions are not polynomials. Previouswork that is mostcloselyrelatedtoourownaremethodsbasedon reg-ularization whichwedescribenext.

C. SurfaceRegularization

Surfacereconstructionisanill-posedinverseproblem be-cause there are innitely many surfaces which may pass through a given set of points. Surface regularization re-stricts the class of permissible surfaces to those which minimize a given energy functional. Terzopoulos pio-neered nite-dierencing techniques to compute approx-imate derivatives used in minimizing the thin-plate en-ergy functional of aheight-eld. He developed computa-tionalmoleculesfrom thediscreteformulationsofthe par-tialderivativesandusesamulti-resolutionmethodtosolve forthesurface. Boultand Kendercompareclassesof per-missiblefunctionsanddiscusstheuseofbasisfunctionsto minimizetheenergyfunctionalassociatedwitheachclass. Using synthetic data, they show examples of overshoot-ingsurfaces thatare oftenencounteredin surface regular-ization. As exemplied by these two methods, many ap-proachesbased on surfaceregularization are restrictedto heightelds.

tiveofthe curvein this method isevaluated withrespect to theparametricvariable. Eachcurveisformulatedasa sum ofweightedbasis functions. Fangand Gossard show examplesusingHermitebasis. Theapproachwepresentin thispaperhassimilarelements. Wealsousebasisfunctions to reconstruct aclosed surface which minimizesa combi-nationofrst,second,andthird orderenergies. Wedier from the previous work in that we reconstruct complex 3D objects using a single implicit function; we perform volumetricrather thansurfaceregularization; and we use energy-minimizingbasisfunctionsasprimitives.

Because our method of reconstruction applies regular-ization, comparisons can also bemade to other classesof stabilizers (or priors) and other energy-minimizing basis functions. We postpone the discussion of other prior as-sumptionsandresultingbasisfunctionstoSectionVwhere weintroducethemulti-orderbasisfunctionthat weuseto reconstructimplicitsurfaces. Theuseofradialbasis func-tions for graphical modelling was introduced by Blinn[6]. Since then, methods have been published that use this surface representation for surface reconstruction, includ-ing Muraki[29]and Savchenko[33]. Our work diersfrom these methods in that we use a basis function that min-imizes multiple energies in 3D, including thin-plate and membrane. Comparison withreconstructions using Gaus-sianandthin-platebasisfunctionswillbeaddressedin Sec-tionV-A.

D. SurfaceSmoothing

Acloselyrelatedtopicisthatofmeshsmoothing,where alow-passlterisappliedtoameshtoreducenoise. Exam-plesofthismethodinclude theworksofTaubinetal. [43] and Desbrun et al. [13]. Theprimary drawbackof mesh smoothing methods is that they require an initial mesh. Ourapproachcreatesandsmoothesasurfaceinonestep. Regularization and smoothingare closely tied. The re-lationshipbetweenregularizationandsmoothinghasbeen studied by many, including Girosi et al. [19], Terzopou-los [44],and Nielson et al[30]. In Section V-A,weuse a volumetricdataset todemonstratethesimilaritybetween regularization and spatial smoothing. Ourreconstruction ofthedataset(whichusesnoinformationaboutthe grid-ded structure ofthevolume)comes verycloseto amodel obtained byspatially smoothingthe 3D dataset priorto iso-surfaceextraction. Theadvantageofourreconstruction algorithm is that it may beapplied to data sets that are unstructured andnon-uniform. Spatialsmoothingcannot easilybeappliedtosuchdata.

E. Activeversus Passive Scanning Technology

Many of the methods described abovereconstruct sur-faces from dense and precise data obtained from active scanning. In this paper, we address the problem of re-constructingsmoothandseamlesssurfaces usingdata ob-tained from passive scanning. In passive scanning, only imagesandcameracalibrationinformationareusedto ob-tain3Dpointsets. Activescanningtechnology(e.g. light

Fig. 1. Left: Stanford Bunny data set from cyberware scanner. Right: Thetoydinosaurdata setfromvoxelcoloring. Both re-constructions were generated using the Crust algorithm. The dinosaurdata setobtainedfrompassivescanningisnoisierand lowerinresolution.

scanning technology (e.g. shape from shading, voxel col-oring)in termsofquality,accuracy,andcost. Thetypical scanningresolutionofcyberwarescannersis0.5mm,while that ofthevoxelcoloringdatasets weuseasexamplesin thispaperareapproximately1.25mm. Datafrompassive scanningis comparativelymore noisy, morenon-uniform, and moresparse than data from active scanners. In par-ticular,surfacereconstructionmethodssuchas[12,20,47, 37]are notsuitedfor creatingmodels from datacaptured usingpassivescanningtechniques.

Figure1isacomparisonbetweendatasetsobtainedfrom laserscannersandthatobtainedfromvoxelcoloring. Both datasets were reconstructedusing theCrustalgorithmof Amenta et al. which exactlyinterpolates alldata points. The toy dinosaur data set obtained from voxel coloring is signicantly lower in resolution and accuracy than the StanfordBunnyobtainedusingacyberwarescanner. The primaryadvantageofpassivescanningmethods isthelow cost ofdigitalcameras (less than$1000)that areused to capturetheimages. Cameracalibrationisobtainedusinga calibrationgridthatiscapturedintheimages. Incontrast, thecurrentcostofactiverangescannersisfrom$10,000to over$100,000.

III. Overview of the Approach

Ourapproach tosurfacereconstructionisbasedon cre-atingasingleimplicitfunctionf(x)bysummingtogether a collectionof weightedradialbasis functions. We adopt theconventionthattheimplicitfunctionispositiveinside thesurface, zeroon thesurface, andnegativeoutside the surface. The nature ofthe radialbasisfunctions that are usedisimportanttothequalityofthereconstructions,and wediscuss thebasisfunction selectionindetailin Section V. As inputto implicitfunction creation,ourmethod re-quiresacollectionofconstraintpointsc

i

con-pointsbeexplicitlyidentiedas beingoutsidethesurface, and wecall these exterior constraints. Scattereddata ap-proximationofthesurfaceandexteriorconstraintsisthen used to construct the implicit function. In Section IV-B wedescribethedetails oftheimplicitformulation,andin SectionVIwediscussthesamplingofsurfaceandexterior constraintsfrom themeasureddataofanobject.

IV. Volumetric Regularization

Thesurfacereconstructiontechniquethatwepresentin this paper is an extension of the variational implicit sur-faces of [48]. This approach is based on the calculus of variationandissimilar tosurfaceregularizationin that it minimizes anenergy functional toobtain thedesired sur-face. Unlike surface regularization, however, the energy functionalisdenedinR

3

ratherthanR 2

. Hence,the func-tionaldoesnotactonthespaceofsurfaces,butrather,on the space of 3D functions. We call this volumetric regu-larization. We use volumetric regularization to obtain a smooth3Dimplicitfunction whosezerolevelsetisour re-constructed surface. BySard's theorem [8,17], the setof nonregular values of such a smooth implicit function is a null set. Hence, the surface described by the zero level set of ourimplicitfunction doesnotcontain pathological, or non-dierentiable, points. In this section, we describe howweconstructanapproximatingsurfaceandobtainthe implicitfunctionrepresentingthesurfaceusingvolumetric regularization.

A. Approximation vs. Interpolation

Scattered data interpolation is the process of estimat-ingpreviouslyunknowndatavaluesusingneighboringdata values that are known. In thecase of surface reconstruc-tion, the surface passes exactly through the known data points and isinterpolated betweenthe data points. Data interpolationis appropriatewhen thedatavaluesare pre-cise. In vision-baseddata, however, there is some uncer-tainty in the validity of the data points. Using data in-terpolation to constructthe surfaceis nolonger ideal be-causethesurfacemaynotactuallypassexactlythroughthe givendatapoints. Thisispreciselytheproblemwith algo-rithms fromcomputationalgeometry thatgenerate polyg-onalmeshesusingdatapointsastheverticesofthemesh. If the uncertainty of the data points is known, a surface that better represents the data would pass close to the data pointsratherthanthroughthem. Constructingsuch a surface is known as data approximation. Many vision-based techniques for capturing 3D surfacepoints havean associated error distribution for the data points. In this section, wediscusshowdata approximationisachievedin ourframeworkusing volumetricregularization.

Inregularization,theunknownfunctionisfoundby min-imizing acostfunctional,H,ofthefollowingform:

H[f]=[f]+ 1

n X

i=1 (y

i f(x

i ))

2

(1)

Intheaboveequation,f istheunknownimplicitsurface

λ

=2.0

λ

=0.0

λ

=0.001

λ

=0.03

Fig.2. Reconstructionofasyntheticrangeimageofacubecorner usingvariousvaluesof.

function; [f] is the smoothness functional, such as thin-plate; n is the number of constraints, or observed data points; y

i

are the observed values of the data points at locations x

i

; and isa parameter(often called the regu-larization parameter)toweighbetweentnesstothedata points and smoothness of the surface. We can allow the surfaceto pass close to, but not necessarily through, the known data points by setting > 0. When = 0, the functioninterpolatesthedatapoints. Thevaluesmaybe assignedaccordingtothenoisedistributionofthedata ac-quisitiontechnique. Figure2showstheresultsofapplying dierent valueson thesamedata set. As approaches zero,thesurfacebecomesrougherbecauseitisconstrained topassclosertothedatapoints. At=0,thesurface in-terpolatesthedata,andovershootsaremuchmoreevident. Atlargervaluesof,thereconstructedmodelissmoother andapproachesanamorphousbubble.

B. ASolution tothe Regularizing CostFunctional

Derivations presented in [19, 49] show that the cost functional given in Equation 1 is minimized by asum of weightedradialbasisfunctions asshownbelow:

f(x)=P(x)+ n X

i=1 w

i

(jx c i

j) (2)

Intheaboveequation, f(x)isanimplicitfunction that evaluates to zero on the surface, negatively outside, and positivelyinside;istheradiallysymmetricbasisfunction; nisthenumberofbasis;c

i

arethelocationsofthecenters ofthe basis;andw

i

aretheweightsforthebasis. In[48], TurkandO'Briencenterabasisfunctionateachconstraint point. Wedothesameinthis work. Theconstraintsmay bepointsonthesurfaceoftheobjecttobereconstructedor pointsexternaltotheobject. Thepolynomialterm,P(x), spans thenull space of thebasis function. Forthin-plate energy,thepolynomialtermconsistsoflinearandconstant terms because thin-plate energy consists of second order derivatives. In3Dwherex=(x;y;z),thepolynomialterm forthin-plateisP(x)=p

0 +p

1 x+p

2 y+p

3

z. Theunique implicitfunctionisfoundbysolvingfortheweights,w

i ,of theradialbasisfunctionsandforthecoeÆcients,p

0 ,p

1 ,p

2 , andp

3

2 6 6 6 4 (r 11 )+ 1 (r 1n

) 1 c

1 . . . . . . 1 . . . (r n1

) (r

nn )+

n 1 c

n

1 1 1 0 0

c1 cn 0 0

3 7 7 7 5 2 6 6 6 4 w 1 . . . w n p 0 p 3 7 7 7 5 = 2 6 6 6 4 f(c 1 ) . . . f(c n ) 0 0 3 7 7 7 5

rij=jci cjj

(3) In the above equation, p

0

and p = (p 1

;p 2

;p 3

) are co-eÆcients of P(x). The function value, f(c

i

), at each constraint point is known since we havedened the con-straintpointstobeonthesurfaceorexternaltotheobject. f(c

i

)=0forallc i

onthesurface. Allexteriorconstraints areplacedatthesamedistanceawayfromthesurface con-straints and are assigned a function value of -1.0 (more details will begivenin Section VI-A onselection of exte-riorconstraints). Noticethat in theabovesystemmatrix, appears on the diagonal. By increasingthe value of , the systemmatrix becomesbetter conditionedbecause it becomesmorediagonallydominant. Theadditionofdoes notinvalidate Equation2because

P n i=1

w i

=0(asseen inrown+1ofthematrix). Theuseoffortradingo in-terpolationandapproximationisfoundinnumerousother publications,includingthoseofGirosiet al. [19],Yuilleet al. [51],andWahba[49]whereadetailedderivationcanbe found.

Itispossibletoassigndistinctvaluestoindividual con-straints. Inthiscase,

P n i=1 i w i

6=0,butinstead,becomes part of the constant in the null space term, P(x). This exibilityisespeciallyimportantwhenweuseexterior con-straintsbecausetheyareaddedonlytoprovideorientation to thesurfacebut donotrepresentrealdata. Inpractice, wehavefound thatworkswellas asemi-global regular-izing parameter,where one valueis used for allsurface constraints,andanotherforallexteriorconstraints.Using one valueforallsurfaceconstraintsisappropriatewhen thespatial distributionof noiseisisotropic. This isa rea-sonable model for many vision-based data sets including the voxel-coloringdataset that welater useasexamples. With other noise models, it may be moreappropriate to useasalocalttingparameterbyassigningavaluefor each surfaceconstraint basedon the condence measure-mentofthepoint. Alargevaluesuchas2.0isoftenused for exterior constraints, while small values such as 0.001 is often used for surfaceconstraints. This choice of for surfaceconstraintswas foundthroughmeasures of tness and curvature applied to the voxel coloringdata set of a toy dinosaur. We foundthat apractical upperbound for for surfaceconstraints from these types of data sets is 0.003. A detailed description of the tnessmeasures and resultsforvariousvaluesofcanbefoundinourtechnical report [14].

The implicit formulation described by Equation 2 has been used in anumber of previouswork, including those [6, 9, 28, 29, 31, 33, 48, 50, 51]. In [6, 29, 51], the basis function,, wasaGaussian,whilein [9,31,33, 48,50], inherentlyminimizedthin-plate energy. In[6,29], the

ba-regularizationwasnotappliedtoobtaintheweightsforthe implicitfunction. Instead,Murakiiterativelyadded Gaus-sianbasisfunctionsuntilasuÆcientlyclosetisobtained. In[28,48,50],reconstructionswereperformedonaccurate, densecyberwarescanneddata. Hence,regularization was notnecessaryandsimplyusingbasisfunctionswhich min-imize adesiredenergywassuÆcient. Inthenextsection, wecomparethevarious choicesofanddiscussour selec-tion ofa basisfunction that minimizesmultiple ordersof energy.

Figure 3isacomparison ofreconstructions of atoy di-nosaur. TheCrustalgorithm wasused to reconstruct the surfaceshown in (a) which exactlyinterpolatesall 20,120 data points; thin-plate basis functions were used to con-structtheinterpolatingimplicitsurfaceshownin (b);and in(c),thin-platebasisfunctionswereusedtoconstructthe approximating implicitsurface with set to 0.001. Only 3000surfaceand264exteriorconstraintswere usedto re-construct the implicit models. The approximating thin-plate surface is much smoother than either of the other twosurfaces. Theovershootsarelessapparent, andthere are fewerprotruding bumps and fewersmall pockets em-bedded in the surface. Unfortunately, the toy dinosaur's features areblobbyand amorphous,especiallyat the feet and hands. Distinct limbs, such as the feet and tail, are fused together. It is apparent from this result that the thin-platebasisfunction usedbyTurkandO'Brien gener-atesmodelswhicharetooblobby.

V. A RadialBasis FunctionforMultipleOrders of Smoothness

TheresultsinFigures3(a),(b),and(c)showthata bal-anceisneededbetweenatightlytting,orshrink-wrapped, surface, and a smooth surface. A tightly tting surface separatesthefeaturesof themodelbut ispronetojagged artifacts. Forexample,theCrustreconstruction,shownin Figure 3(a), is an exact t to the data with no smooth-nessconstraint. Ontheotherhand,asmoothsurfacemay becometooblobbyasseenin Figures 3(b)and (c), which show that minimizing the thin-plate energy alone is not suÆcienttoproduceasurfacethat separatesfeatureswell andislocallydetailed.

In[10],ChenandSuterderiveradialbasisfunctions for the family of Laplacian splines. The basis functions are comprisedofjrj

k , jrj

k

logjrj,exponential,andBessel func-tionterms, where ris thedistance from thecenter of the radiallysymmetric basis. The value of k depends on the dimension and order of smoothness. Turk and O'Brien use (r) = jrj

2

logjrj for 2D thin-plate interpolation, and (r) = jrj

3

for 3D thin-plate interpolation. Figure 4(a) showsthatthesefunctionsexhibitglobalinuencebecause thevalue ofthefunction tendstowardinnity asthe dis-tancefromitscenterincreases. Thesystemmatrix,which consistsoftheevaluationofthebasisfunctionatdistances between pairs of constraints, is dense because constraint pointsareuniformlyspreadacrosstheregionofinterest.

(b)

(c)

(d)

(e)

(a)

Fig.3. Reconstructionsofthetoydinosaur. (a)Crustreconstruction. (b)Exactinterpolationusingthin-platebasisfunction. (c)Surface approximationusingthin-platebasisfunction.(d)SurfaceapproximationusingGaussianbasisfunction.(e)Surfaceapproximationusing multi-orderbasisfunction.

Multi-Order Basis

0

1000

2000

3000

4000

0

100

200

300

400

500

600

700

800

900

Timing Comparison

number of constraints

time (sec.)

thin−plate

multi−order

(c)

(d)

(b)

0

0.5

1

1.5

2

−1

0

1

2

3

4

5

6

7

8

Thin−Plate Basis

radius, r

φ

(r)

r

2

_log|r|

|r|

3

(a)

0

1

2

3

4

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

Varying

δ

,

τ

=0.01

radius, r

φ

(r)

δ

=5

δ

=10

δ

=25

0

0.2

0.4

0.6

0.8

1

0

1

2

3

4

5

6

7

8

x 10

−3

Varying

τ

,

δ

=10.0

radius, r

φ

(r)

τ

=0.01

τ

=0.05

τ

=0.1

Fig.4. (a)Crosssections ofradiallysymmetricbasisfunctions forjrj 2

l ogjrjand jrj 3

. (b)and(c)Crosssectionsofbasisfunctions fora combinationof rst,second, and thirdordersmoothnessforvariousvaluesofÆ and. (d)Comparisonofrunningtimestosolvefor weightsforthethin-plateandandforthemulti-orderbasisfunctions.

plateenergyisequivalenttosecondorderenergy,and mem-brane to rst order energy. Surprisingly, a radial basis functionthatminimizesacombinationofrst,second,and third order energies quickly falls toward zero, yielding a betterconditionedsystemmatrixthanonethatminimizes thin-plate energyalone. In[38],Suterand Chenused ba-sis functions that minimizemultiple orders ofsmoothness (beyondtherst andsecond order)to reconstructhuman cardiacmotion. Theyfoundthatamodelminimizingthird and fourth order energyresulted in thesmallestRMS er-ror. They concluded that basis functions that minimize more than just the rst and/or second order energy gen-erate more accurate reconstructions. In addition, as the space dimension increases, the order of continuity of the thin-plate splineat datapointsdecrease. Suterand Chen showthat in 3D,thethin-platesplinebasishas discontin-uous rst order derivatives at the data points. We chose to use abasis that achievesrst, second,and third order smoothness because, unlike motion, object surfaces may containsharpfeatures thatare C

1

discontinuous. The re-sultingimplicitfunction hascontinuousderivativesdueto

the additionalthird order smoothness (although, the iso-surfacemaynothavecontinuousderivatives).The geomet-ric analogyto minimizing third order energy iscurvature continuity. IthasbeenshowninpreviousworkbyFangand Gossard[16]thatincludingcurvaturecontinuityresultsin improvedcurveandsurfacetting. Terzopoulosalso spec-ulatesontheuseofcurvaturecontinuousstabilizersin[44]. In [10], Chen and Suter derive such a basis, using a smoothness functional comprised ofthe rst, second,and thirdorderLaplacianoperator. Theassociatedpartial dif-ferentialequationis similarto Laplace'sequation f = 0,butalsohashigherorderterms:

Æf+ 2

f

3

f =0 (4)

IntheaboveequationtheLaplacianoperator in3Dis:

f =

@ 2

f

@x 2

+ @

2 f

@y 2

+ @

2 f

@z 2

(5)

amountofsecondordersmoothness. Theradialbasis func-tionthatinherentlyminimizestheaboveenergyfunctional in 3Dasderivedin [10]is:

(r)= 1 4Æ 2

r (1+

we p

v r v w

ve p

w r v w

)

v= 1+

p 1 4

2 Æ

2 2

2

w= 1

p 1 4

2 Æ

2 2

2

(6)

Intheaboveequations,risthedistancefromthecenter oftheradialbasisfunction. Thepolynomialtermspanning thenullspaceofthemulti-orderbasisfunction issimplya constant, P(x)= p

0

. Figures 4(b) and (c) show plotsof the above function for various valuesof Æ and . Unlike theplotfor(r)=jrj

3

,these plotsshowthat thevalueof thebasisfunctionquicklyfallstowardzeroasthedistance fromitscenterincreases.

A. Comparison with Gaussian, Thin-Plate Radial Basis Functions,andSpatialSmoothing

Themulti-orderbasisfunction describedbyEquation6 has several advantages over the thin-plate and Gaussian basis functions used by Blinn,Muraki, Yuille, and others [6, 51, 29]. The system matrix formed by the thin-plate basis function is dense, and non-zero values grow larger awayfrom the diagonal. Computation time increases sig-nicantly as more constraints are specied. In contrast, the system matrix formed by the multi-order basis func-tion isdiagonally dominantand isespeciallyamenableto the biconjugate gradient method of solving linear equa-tions. Eventhoughthe matrixformed bythemulti-order basisisdense, non-zerovaluesdiminishawayfromthe di-agonal. Timing resultsshowthat theunknownweightsof Equation2weresolvedin1.5minutesusingthemulti-order basisfunction withÆ=10 and =0:01,whilethesystem matrixgeneratedforthesamesetof3264constraintsusing thethin-platebasisfunction required7.9minutestosolve onanSGI Originwith 195MHzMIPSR10000processor. Figure 4(d)is acomparisonofrunningtimes versus num-berofconstraintsfor thethin-plateand multi-orderbasis functions. The increase in running time as the number of constraints increase is fairly linear for the multi-order basisfunctionasopposedtothethin-platebasis. The sys-tem matrix formed using Gaussian basis functions (with = 0:01) is sparse, requiring only 2.6 minutes to solve. Thesystemmatrixissolvedevenmorequicklywithsmaller valuesof,butat thecostofworsereconstructions.

Intermsofreconstructionquality,themulti-orderbasis function is ableto reconstructmorelocally detailed mod-elswhile stillretaining globalsmoothness. Boththe thin-plate and the Gaussian basis functions result in models with overshooting surfaces. The Gaussian basis actually formsholesembedded inthesurface. Thethin-platebasis creates poorer reconstructions than the multi-order basis because the thin-plate basis forces the surface to be too smooth,resultinginblobbymodels. TheGaussianbasisis an innite mixture of Tikhonovstabilizers, also resulting in surfaces thataretoosmooth. Figure 3is acomparison ofreconstructions ofthetoydinosaurusing thethin-plate

tions. Notethat theround protrusionbeneath thearmis the wind-up key for the toy and that the bumps on the back are the scales and spines of the actual toy dinosaur (seeFigure9fortwooftheoriginalimages).

Another dierence between reconstruction using the multi-order and the thin-plate basis is in use of non-zero interiorandexteriorconstraints. Reconstructionusingthe thin-plate basis is much more dependent on the dense placement of exterior constraints to prevent the surface fromovershootingintoregionswherethemodelshouldnot existandontheplacementofinteriorconstraintstodene the orientation of the surface. In [48],Turk and O'Brien paireach surfaceconstraintwith anormal constraintthat is interior to the surface and has a function value of 1.0. The multi-orderbasis does not overshoot asmuch as the thin-platebasis. Hence,asparse, uniform spreadof exte-riorconstraints areenough to orientthe implicit surface. Wehavefoundinpractice,thatapproximatelyoneexterior constraintforeverytensurfaceconstraintsissuÆcientand thatinteriorconstraintsareunnecessary. Moredetailsare provided in Section VI-A on how exteriorconstraintsare obtained.

The real voxel coloring data sets we use, described in Section VII, are embedded in aglobal gridstructure. In such cases, it is possible to spatially smooth the data in 3Dandobtainasmoothreconstructionthroughiso-surface extraction. Note that this is not truein thegeneral case wheretheinputdatasetmaybeunstructured. Asitturns out,themulti-orderprior weusecangivereconstructions that are verysimilar to spatial smoothing when Æ and areappropriatelysettobesmooth. Figure5comparesthe reconstruction of the toy dinosaur using spatial smooth-ing and using the multi-order basis. The similarity of these reconstructions show that the multi-order basis is indeed closely related to spatial smoothing. As noted in [43], spatial smoothing tends to shrink features (such as thepawsofthedinosaur),whilevolumentricregularization doesnot. Anaddedadvantageofusingenergy-minimizing basis functions is that it can create smooth reconstruc-tionsofunstructuredandnon-uniformdata,towhich spa-tiallysmoothingcannoteasilybeapplied. Uniformspatial smoothingofunstructureddatawouldrequirearesampling stepto integratealldata pointsinto astructured grid,as was done in [12]. In addition, the parameters, Æ and , associated with the multi-order basis allows ner control overhowmuchsmoothingisapplied. Forexample,in Fig-ure3(e),Æand weresettopreservethescalesandspines onthebackofthetoydinosaurwhich islostbytoomuch smoothinginFigure5.

VI. Constraint Specification

(a)

(b)

Fig. 5. (a) Iso-surface extraction of volumetric data after spatialsmoothing using a Gaussian lter with a radius of four voxels. (b) Reconstructionusingthemulti-orderbasisfunctionwith3000surfaceconstraintsandÆ=10:0and=0:025.

section,wedescribethemethodbywhichweobtainsurface andexterior(negative)constraintsusedinthe reconstruc-tion. We alsoaddressthesampling requiredto guarantee that thetopology of theobject is correctlyreconstructed and howthis sampling density is mapped to the selected valuesfortheparameters,Æand,controllingtheamount ofrstandthirdorder smoothnessrespectively.

A. ExteriorConstraints

The computer vision community has developed many methods to acquire 3D positional information from pho-tographicimages takenby cameras. Thegoalof allthese methodsistodetermineacollectionof3Dpointsthatlieon agivenobject'ssurface.Whensuchacollectionofpointsis acquiredusingcameras,thecamerapositionanddirection provideadditionalinformationthatcanbeusedforsurface reconstruction. Ifasurfacepointisseenfromaparticular camera, there are no other surfaces between the camera andthepoint. Wecalltheregionbetweenthecameraand thesurfacefree space. Otherapproachesto surface recon-structionmakeuseofthisinformationaswell[12]. Wecan use thisapriori knowledgeaboutthe object surface loca-tionsandthefreespacetodeneconstraintsthatlieonor outsideoftheobject,asseeninFigure6.

Recall that the exterior constraintsare those locations wherewewantourimplicitfunctiontobenegative,andthe surfaceconstraintsare wherethe implicitfunction should evaluatetozero. Inpractice,weplaceexteriorconstraints atthesamedistanceawayfromthesurfaceconstraints to-wardsthe camera viewpoints and assign them afunction valueof-1.0. As mentionedin SectionIV-B, exterior con-straintsdonotrepresentactualdata,butrather,arehints to the surface orientation. Hence, a sparse sampling of exteriorconstraintsissuÆcienttoproperlyorientthe sur-face, and a large value of , such as 2.0, indicates that the negative data point should be highly approximated. Wehavefoundthatoneexteriorconstraintpertensurface constraints works well in practice. An additional sparse

*

-free space

*

*

*

*

*

--

- -

--

-

--

_-

--Fig.6. Freespaceiscarvedoutbyraysprojectingfromthecamera totheobjectsurface.Surface(*)andexterior(-)constraintsare denedbythefreespace.

sphere around the object helps to constrain the surface, and alone,is oftensuÆcientto dene thesurface orienta-tion. Next,wediscusshowwesubsampleboththeexterior andsurfaceconstraints.

B. Subsampling SurfaceConstraints

Becauseourmethod ofreconstructionrequiresthe solu-tionofalinearsystem,itiscomputationallylimitedinthe number of constraints that can be used to construct the surface. Examplesshown in this paperhave used around 3000 surfacepoints, sampled from aset of around 20,000 surfacepoints. Usingtheentiredatasetwouldnotonlybe intractable, but would also result in an implicit function that is equal in size to the original discrete data set. In thiscase,therepresentationwouldnolongerbecompact.

Fig.7. Reconstructionsoftheheadofthetoydinosaur.Left: recon-structionusingfulldataset(3173surfacepoints). Right: recon-structionusingasubsetofavailabledata(477surfacepoints).

subsamplethevoxelcoloringdataandexteriorconstraints previouslydescribed.

ExperimentsshowthatthereduceddatasetissuÆcient tocapturethedetailspresentinthenoisydata. Figure7is acomparisonbetweenreconstructionsfromtheentiredata set and fromasampled subset. The fulldata set consists of3173surfacepoints,whilethereducedsetconsistsof477 points. Thetotal distance, or error,between the original 3173surfacepointsandthesurfacereconstructedfromthe full data set was0.008,while the total errorbetween the 3173surfacepointsandthesurfacereconstructedfromthe reducedsetwas0.009. Themodelitselfwasconstrainedto bewithina222box.

Adaptivelyincreasingthesamplinginhighlydetailed re-gions is not appropriate in many vision-based data sets. Detailed regionsare oftensynonymouswith areasofhigh curvature andsmall area. Inavision-basedsystem,these smallareasmaptofewpixelsintheacquiredimages, result-inginlowcondenceforsuchregions. Increasingthe sam-pling density in these small, detailed regions would taint thereduceddatasetwithmanylowcondencepoints.

Itispossibletopartition thedataset,constructa sepa-rate implicitsurfaceforeach partition,andthen combine thesurfaces. However,the resultingrepresentationwould notbecompact. Weoptednottotakethisapproachsince thedierenceinthetnesserrorsbetweenthefullandthe reduceddata setswasminimal. Yngveand Turk [50]and Carr et al. [9] havealso shown that it is unnecessary to haveabasisfunctionforeachsurfacepoint. Theirapproach wastoiterativelyaddbasisfunctionsuntilthetnesserror wassuÆcientlylow. Weavoidaniterativesolutionby uni-formlysamplingthedataset. Onedrawbackoftheuniform samplingapproachisthatnoiseatthescaleoffeatures can-notberemoved. Someexamplesofthiseectareshownin thetoydinosaur'schestarea.

C. Mapping SurfaceSampling DensitytoÆand Values

RecallfromSectionVthatÆcontrolstheamountofrst ordersmoothness,while controlstheamountofthird or-der smoothness. The values of Æ and that correspond tothebest reconstructionofasurfaceisdependentonthe samplingdensityofthesurfaceandthedesiredsmoothing. Inourwork,wemaintainconsistentaveragesampling

den-andbyusingnearlythesamenumberofsurfaceconstraints tocoverthedataset. Wescaleallthemodelstoliewithina 222box. Byapplyingthisnormalization,thefeature size, average sampling density, and choice of Æ and are consistentacrossall models. This normalizationis appro-priate becauseall ourinputdatasets haveapproximately thesameresolution. Onemeasureofthisnormalizationis theaverage minimaldistance between samplepoints. We computethisdistance byaveragingthedistances between eachsamplepointanditsclosestneighboringsamplepoint. WeshowlaterinSectionVIIwherewediscussthedatasets inmoredetailthatthisaverageminimaldistanceissimilar acrossalldatasetsafter normalizationandsampling.

We chose appropriate valuesfor Æ and by comparing modelsthathavebeenreconstructedatvariousvaluesofÆ and. Wehavetwomethodsofvalidationandcomparison between the reconstructed models. These methods are a measureoftnesserrorandameasureofaveragecurvature. Wedenetnesserrortobetheaggregatedistancebetween theoriginaldatapointsandthereconstructedsurface. To measuretheaveragecurvatureofasurface,werstextract apolygonalmodelfromtheimplicitfunction. Wemeasure curvature at each vertex of thepolygonalmodel using an approximationthatwasdevelopedforthesmoothing oper-atorin[13]. Theaveragecurvatureisobtainedbydividing the aggregatecurvature by the number of vertices in the polygonalmodel. Highcurvatureis associatedwith sharp features in the surface, while low curvature is associated withovershootsandblobbysurfaces.

Weappliedthemeasuresoftnessandcurvatureto the toydinosaurdatasettoguideselectionofappropriate val-ues for , Æ, and . Fordetails on the selection of these values, see our technical report [14]. We have found in practicethatvaluesofbetween0.001to0.003,Æbetween 5.0to40.0and between0.005to0.025canbeusedto pro-ducelocallydetailed,yetgloballysmooth,reconstructions withminimalerroronavarietyofdatasets.

D. HandlingOutliers

Outliersare handled bya preprocessingstepthat nds thelargestconnectedcomponentin the dataset. Forthe voxelcoloringdata set, wetraverse thevolumeof surface points and grouptogether voxels that are within the 26-neighborhoodofeachother. Thesingle,largestconnected componentis kept, and allother surfacepointsare elimi-nated. Ifn componentsexist(where n>1), thenwecan sortthecomponentsinthedatasetaccordingtotheirsize, andkeeponlytherstn largestcomponents.

E. Topology Adaptation

ex-exteriorconstraintisneedednearthetorusholetoforcethe existenceoftheholeinthemiddleofthetorus. Aslongas thesurfaceand exteriorspaceareuniformly sampled, the topologyiscorrectlyreconstructed. However,sinceweare usingdatafromvision-basedmethods, viewocclusionora lackofreferenceviewsmaypreventcorrectsamplingofthe space. For example,if no views of the torus showing the holein thecenterare available,thentheholemaynotbe correctlyreconstructed. Weargue,thatinsuchacase,the topologyofthereconstructedmodel isconsistentwiththe ambiguityofthetopologyinthedataset.

VII. Results

We now show that volumetricregularization generates globallysmooth,yetdetailed,surfacesanddiscussthe ad-dition of color to the models. We reconstructed surfaces usingthemulti-orderbasisontwotypesofdata{synthetic rangedataandrealvoxelcoloringdata. Ourmethodof re-construction cangeneratesmooth surfaces from datasets thataregloballyunstructuredandnoisy. Althoughneither typesofdatasetsweusehaveboththese features,eachis anexampleofonefeature. Thesyntheticrangedataisnot embeddedinaglobalgrid,whilethevoxelcoloringdatais quitenoisyincomparisontoactiverangescanningdata.

A. SyntheticRangeData

We useamodiedray-tracer [24] to generatesynthetic range images as one test of our approach. We used the Stanford Bunnyasourtestmodel,andcreatedthree syn-thetic range images from positions separated by 120 de-grees on a circle surrounding the model. For each range image, surfaceconstraintsarecreatedbyuniformly down-sampling the range image to reduce the size of the data set. For everyten surfaceconstraints, oneexterior nega-tiveconstraintiscreatedwithinthefreespacedescribedin Section VI. Additional exteriorconstraintsaredened on aspheresurrounding thebounding boxof theobjectat a distance fartherawayfrom the object. Figure8(a) shows theoriginalStanfordBunnymodelconsistingof69,451 tri-angles,while(b),(c),and(d)showtheimplicitsurface re-constructedfrom2168surfaceand193exteriorconstraints using themulti-orderbasisfunction. Figures8(c)and (d) also show thedistribution of the constraintsoverlayedon topofthereconstruction. Theaverageminimumdistance betweensurfacesamplesusedinthereconstructionis0.051. Valuesof=0:001,Æ=10, and =0:01wereusedto re-constructthesurface. Theimplicitsurfaceisquitesimilar to the ground truth. Our method of reconstruction pro-ducesplausiblesurfaceseveninlocationswherethedatais sparse. Themodelisclosed onthetopand bottom ofthe Bunnyeventhoughfewconstraintpointswereplacedthere. The model is closed at these places dueto the inherently manifold nature of implicit surfaces, and it is smooth at theselocationsbyvirtueofminimizingthecostfunctional.

B. Real Volume-Carved Data

Synthetic datadoesnothavethenoisycharacteristicof

that we use and how we dene the surface and exterior constraints. We use three data sets of real objects ob-tained through voxel coloring [35, 11] { a toy dinosaur (fromSteveSeitz[35]),abroccolistalk, andastackoftoy tori(fromBruceCulbertsonandTomMalzbender[36]and referredtoasthetowersdataset). Bothdatasetswere ob-tainedbytakingabout20imagesapproximatelyonacircle aroundeach object. Thin-shelled,voxelized surfaces were thenconstructedusingthegeneralizedvoxelcoloring algo-rithm [11]. The space is carved by splatting each visible voxeltowardseachcalibratedcameraanddeterminingthe consistencyofthecoloracrosstheimages. If thevariance in colorintensityis below aspeciedthreshold, thevoxel iskeptaspartofthe objectsurface. Otherwise, itis cast outandassignedazeroopacityvalue. Thedataconsistsof red,green,andbluechannels. Non-emptyvoxelsrepresent thepresenceofasurface,asdeducedbythevoxelcoloring algorithm. Figure 10 shows the real voxel coloring data sets.

We apply the technique described in Section VIto ob-tainsurfaceandexteriorconstraintsforthevoxelcoloring dataset. Non-emptyvoxelsaresurfacelocations. Exterior constraints are found by projecting each surfacevoxel in thevolumeto theimage planeof each camera. Iftheray fromthesurfacevoxeltoacameraintersectsothersurface voxels, then the viewof the voxelis blocked. Otherwise, thecamerahas anunobscured view,and anexterior con-straint can be placed at a small distance away from the surface voxel along the ray towards the camera, as de-picted in Figure 6. Note that for each surface voxel, an exteriorconstraintiscreatedforeach camerathathas an unobscuredviewofthesurfacevoxel. Again,onlyasubset ofthe surfaceandexteriorconstraintsareselectedby the Poisson discsampling techniquein Section VI-B. Once a speciednumber of constraints havebeen collected, they are given to the reconstruction algorithm. In this paper, we have used from 2000 to 3000 surface constraints. We havefound that 100to 300 exteriorconstraintssuÆce to dene theorientation of thesurface. Figure 10shows ex-amplesofourreconstructionsfromspacecarveddata. The averageminimumdistancebetweensurfacesamplesusedin thereconstructionforthetoydinosaur,broccoli,and tow-ersdatasetsare0.035,0.041,and0.042,respectively. Note that thebumpsonthebackofthedinosaurarethescales andspinesoftheactualtoy. Thesmallprotrusionnearthe baseofthebroccolistalkisanactualleafthathasbeen ac-curatelydetectedbythevoxelcoloringalgorithmand has beencorrectly sampled and reconstructed by the method wedescribein thispaper. TherunningtimeforMarching Cubes[27]toextractaniso-surfaceisdependentonthe de-siredresolutionofthemodel andthenumberof terms(or constraints)in theimplicitfunction. Surfaceextractionof thetoydinosaurattheresolutionshowninFigure10took 14.5minutes.

C. Model Coloring

be-(a)

(b)

(c)

(d)

Fig.8. Part(a)istheoriginalStanfordBunnyconsistingof69,451triangles. Parts(b), (c),and(d)showthereconstructedsurfaceusing themulti-orderbasisreconstructionmethodofthispaper. Parts(c)and(d)showthesurfaceconstraints(bluesquares)andtheexterior constraints(greensquares)usedinthereconstructionoverlayedontopofthereconstructedsurface. Notethatthereconstructedsurface isclosedonthetopandbottomeventhoughfewconstraintsarepresent.

Fig.9. Eachpairofimagesisacomparisonoftheoriginalinputimageusedtogeneratethevoxelcoloringdataset(left)andthereconstructed implicitmodelrenderedfromthesamecameraviewpoint(right). AnovelviewpointoftheimplicitmodelisshowninFigure10

surfaceextractionusingMarchingCubes[27]. Weassigna colortoeachtriangleofthepolygonalmodelby reproject-ing the trianglesback to the originalinput images. Each triangleinthepolygonalmodelissubdivideduntilits pro-jected footprint in the images is subpixel in size, so that it can simply take on the color of the pixel to which it projects. In most cases, atriangle is visible in several of theoriginalimages. Wecombinethecolorsfromthe dier-entimages using aweighted average. The weightof each colorcontribution iscalculatedbytaking thedotproduct betweenthetrianglenormalandtheviewdirection ofthe camerathatcapturedtheparticularimage. Cameraswith viewingdirectionsthatarenearlyperpendiculartothe tri-anglenormalcontributelessthanthosewithviewing direc-tions that are nearly parallel to the triangle normal. We usez-bueringto ensurethat onlycameraswithan unob-scured viewof the triangle cancontribute to the triangle color. Figure10showsthenalmodelsofthetoydinosaur, broccoli, andtowersfrom novelviewpointsafter colorhas beenapplied. Figure9isacomparisonoftwoofthe origi-nalinputimagesofthetoydinosaurwithrenderedimages

ofthereconstructedimplicitsurfacefromthesamecamera viewpoints.

D. Limitationsof VolumetricRegularization

Surface reconstruction using volumetric regularization does notgenerate surfaces with boundaries. Instead, our methodclosesovergapsinthedatasettoconstructa man-ifold surface. Open surfaces canbe generated by placing limitsontheiso-surfaceextraction.

AsnotedinSectionVI,thefeaturesandtopologyofthe reconstructedmodelisdependentonthedensityofthe in-putdataset. Featuresthatarenotinherentinthedatawill notbereconstructed. Conversely,noisethat isthesize of featureswillbecomeembeddedinthereconstruction. This limitation is common to most methods of reconstruction andsmoothing.

88679 voxels

δ

= 5,

τ

= 0.025

29882 voxels

δ

= 5,

τ

= 0.025

20120 voxels

δ

= 25,

τ

= 0.01

sets using thethin-plate spline oersan eÆcient solution to the variationalimplicit method. Webelieve that their workusingtheFastMultipoleMethodcanalsobeapplied herewiththemulti-orderbasis.

VIII. Conclusionand FutureWork

Thereconstruction algorithm wehavepresentedin this paper generates models that are smooth, seamless, and manifold. Ourmethodisableto addresschallenges found in real datasets, includingnoise, non-uniformity, low res-olution, andgaps in thedata set. Wehavecompared our techniqueto anexact interpolation algorithm (Crust), to thin-plate and Gaussian variational implicits, and to the original volumetric reconstruction using the toy dinosaur as a running example. Obvious advantages to the mod-els generated by volumetric regularization are that there arenodiscretizationartifactsasfoundinvolumetric mod-els, and the surface is not jagged as in the Crust recon-struction. Volumetricregularization cangenerate approx-imating, rather than interpolating, surfaces, and is most closely related to the thin-plate variational implicit sur-faces. It compares favorably to thethin-plate variational implicitsurfacesincomputationtimeaswellasinthe sur-faces that aregenerated. Usingthemulti-orderradial ba-sisfunction,volumetricregularizationgenerateslocally de-tailed,yetgloballysmoothsurfacesthatproperlyseparate thefeaturesofthemodel.

We have adapted the variational implicit surfaces ap-proachtorealrangedatabydevelopingmethodstodene surface and exteriorconstraints. Althoughsurface points aredirectlysuppliedbytherangedata,wehaveintroduced newmethods forcreatingexteriorconstraintsusing infor-mation aboutthe camerapositions used in capturing the data. Wehaveapplied thistechniqueto spacecarved vol-umetricdataandsyntheticrangeimages.

We plan to look at several potential improvements to our approach, including use of condence measurements and modifying the basis functions locally. For each 3D surfacepointobtainedfromthegeneralizedvoxelcoloring algorithm,theregularizationparameter,,canbeassigned based on the variance of the colors to which the surface voxelprojectsin theinputimages. Anotheralternativeis to assigndierentÆand valuesforthemulti-orderbasis according to the curvature measure at constraint points. Thesefuturedirectionsholdpromiseoffurtherreningthe sharp features of reconstructed surfaces of real world ob-jects.

IX. Acknowledgements

We thank James O'Brien for stimulating discussions about alternate basis functions for surface creation. We also thank Marc Levoy for discussions on using implicit functions to performsurfacereconstruction. Wesincerely acknowledgeSteveSeitz forprovidingus withtheoriginal input images of the toy dinosaur and camera calibration information, and Bruce Culbertson and Tom Malzbender of Hewlett-Packard forthe inputimages and camera

cali-funded byONRgrant#N00014-97-1-0223andNSF grant #ACI-0083836.

References

[1] Amenta, N., M. Bern, and M.Kamvysselis. \A New V oronoi-Based Surface Reconstruction Algorithm", SIGGRAPH Pro-ceedings,Aug.1998,pp.415{420.

[2] Bajaj,C.L.,J. Chen,and G.Xu.\FreeFormSurfaceDesign withA-Patches",GraphicsInterfaceProceedings,1994,pp.174{ 181.

[3] Bajaj,C.L.,F.Bernardini,andG.Xu.\Automatic Reconstruc-tionofSurfacesandScalarFieldsfrom3DScans",SIGGRAPH Proceedings,Aug.1995,pp.109{118.

[4] Bernardini,F.,J.Mittleman,H.Rushmeier,andC.Silva.\The Ball-Pivoting Algorithm for Surface Reconstruction", IEEE Transactions on Visualization and Computer Graphics, Oct.-Dec.1999,5(4),pp.349{359.

[5] Blane, M.M., Z. Lei, H.Civi,and D.B.Cooper. \The 3L Al-gorithm for Fitting Implicit Polynomial Curves and Surfaces to Data", IEEETransactions onVisualization and Computer Graphics,Mar.2000.

[6] Blinn, J. \A Generalization of Algebraic Surface Drawing", ACMTransactionsonGraphics,July1982,1(3).

[7] Boult,T.E.andJ.R.Kender.\VisualSurfaceReconstruction Us-ingSparseDepth Data",Computer Visionand Pattern Recog-nitionProceedings,1986,pp.68{76.

[8] Bruce,J.W.andP.J.Giblin.CurvesandSingularities,2nded., CambridgeUniversityPress,1992,pp.59{98.

[9] Carr, J., R.K. Beatson, J.B. Cherrie, T.J. Mitchell, W.R. Fright, and B.C. McCallum. SIGGRAPH Proceedings, Aug. 2001,pp.67{76.

[10] Chen, F.and D.Suter.\MultipleOrderLaplacianSpline- In-cludingSplineswithTension", MECSE1996-5, Dept. of Elec-trical and Computer Systems Engineering Technical Report, MonashUniversity,July,1996.

[11] Culbertson,W.B.,T.Malzbender,andG.G.Slabaugh. \Gen-eralizedVoxelColoring",VisionAlgorithms: Theoryand Prac-tice.Part of Series: Lecture Notesin Computer Science, Vol. 1883Sep.1999,pp.67{74.

[12] Curless,B.andM.Levoy.\AVolumetricMethodforBuilding ComplexModelsfromRangeImages",SIGGRAPHProceedings, Aug.1996,pp.303{312.

[13] Desbrun, M., M. Meyer, P. Schroder, and A.H. Barr. \Im-plicitFairingofIrregularMeshesUsingDiusionandCurvature Flow",SIGGRAPHProceedings,Aug.1999,pp.317{324. [14] Dinh, H. and G.Turk. \Reconstructing Surfaces by V

olumet-ricRegularization",GVU-00-26,CollegeofComputing,Georgia Tech,December2000.

[15] Edelsbrunner H. and E.P. Mucke. \Three-Dimensional Alpha Shapes",ACMTransactionsonGraphics,Jan.1994,13(1),pp. 43{72.

[16] Fang, L. and D. Gossard. \Multidimensional Curve Fitting to Unorganized Data Points by Nonlinear Minimization", Computer-AidedDesign,Jan.1995,27(1),pp.48{58.

[17] Fomenko,A.andT.Kunii.TopologicalModelingfor Visualiza-tion,Springer,1997.

[18] Frisken, S.F., R.N. Perry, A.P. Rockwood, and T.R. Jones. \Adaptively Sampled DistanceFields: AGeneral Representa-tion of Shape for ComputerGraphids", SIGGRAPH Proceed-ings,Aug.2000,pp.249{254.

[19] Girosi, F., M. Jones, and T. Poggio. \Priors,Stabilizers and Basis Functions: From Regularization to Radial, Tensor and AdditiveSplines",A.I.MemoNo.1430,C.B.C.L.PaperNo.75, MassachusettsInstituteofTechnologyArticialIntelligenceLab ,June,1993.

[20] Hilton, A., A.J. Stoddart, J. Illingworth, and T. Windeatt. \ReliableSurfaceReconstructionfromMultipleRangeImages", Proceedings of FourthEuropean Conference onComputer Vi-sion,1996.

[21] Hoppe,H.,T.DeRose,andT.Duchamp.\Surface Reconstruc-tionfromUnorganizedPoints",SIGGRAPH Proceedings, July 1992,pp.71{78.

ConstrainedImplicitPolynomials",IEEETransactionson Pat-ternAnalysisandMachineIntelligence,21(1),pp.21{31,1999. [24] Kolb, C., The Rayshade Homepage.

http://www-graphics.stanford.edu/cek/rayshade/info.html. [25] Kutulakos, K.N.and S.M.Seitz.\ATheoryofShapebySpace

Carving", International Journal of Computer Vision, Aug. 1999.

[26] Lee,M.andG.Medioni.\InferringSegmentedSurface Descrip-tionfromStereoData\,ComputerVisionandPattern Recogni-tionProceedings,1998,pp.346{352.

[27] Lorensen,W.E.andH.E.Cline.\MarchingCubes",SIGGRAPH Proceedings,July1987,pp.163{169.

[28] Morse,B.S.,T.S.Yoo,P.Rheingans,D.T.Chen,andK.R. Sub-ramanian.\InterpolatingImplicitSurfacesfromScatteredData UsingCompactly Supported RadialBasis Functions" Proceed-ingsof ShapeModelingInternational,May,2001,pp.89{98. [29] MurakiS.\VolumetricShapeDescriptionofRangeDatausing

Blobby Model", SIGGRAPH Proceedings, July1991, pp.227{ 235.

[30] Nielsen,M.,L.Florack,andR.Deriche.\Regularization, Scale-Space, and EdgeDetection Filters", Journal onMathematical ImagesandVision,1997.

[31] Nielson,G.M.\ScatteredDataModeling",ComputerGraphics andApplications,1993,pp.60{70.

[32] Pentland, A.\AutomaticExtractionofDeformablePart Mod-els", International Journal of Computer Vision, 1990, Vol.4, pp.107{126.

[33] Savchenko, V.V., A.A. Pasko, O.G.Okunev, and T.L.Kunii. FunctionRepresentationofSolidsReconstructedfromScattered SurfacePointsand Contours.ComputerGraphicsForum,Vol. 14,No.4,1995,pp.181{188.

[34] Sclaro, S.and A. Pentland. \Generalized Implicit Functions for ComputerGraphics", SIGGRAPH Proceedings, July 1991, pp.247{250.

[35] Seitz,S.M.andC.R.Dyer.\PhotorealisticSceneReconstruction byVoxelColoring",InternationalJournalonComputerVision, 1999,35(2),pp.151{173.

[36] Slabaugh,G.,B.Culbertson,T.Malzbender,andR.Schafer.\A Survey of Methods for Volumetric Scene Reconstruction from Photographs", International Workshop on Volume Graphics, June2001,pp.51{62.

[37] Soucy,M.andD.Laurendeau\AGeneralSurfaceApproachto theIntegrationofaSetofRangeViews",IEEETransactionson Pattern Analysisand MachineIntelligence,April1995,17(4), pp.344{358.

[38] Suter,D.andF,Chen.\LeftVentricularMotionReconstruction BasedonElasticVectorSplines",ToappearinIEEE Transac-tionsonMedicalImaging,2000.

[39] SzeliskiR.andS.Lavallee.\Matching3DAnatomicalSurfaces with Non-Rigid Deformations using Octree-Splines", Interna-tionalJournalofComputerVision,1996,18(2),pp.171{186. [40] Tang, C. and G. Medioni. \Inference of Integrated Surface,

Curve,andJunctionDescriptionsFromSparse3DData",IEEE Transactions on Pattern Analysis and Machine Intelligence, Nov.1998.,20(11).

[41] Taubin,G.\EstimationofPlanarCurves,Surfaces,and Nonpla-narSpacesCurvesDenedbyImplicitEquationswith Applica-tionstoEdgeandRangeImageSegmentation",IEEE Transac-tionsonPatternAnalysisandMachineIntelligence,Nov.1991, 13(11).

[42] Taubin, G.\AnImprovedAlgorithmfor AlgebraicCurveand SurfaceFitting", Proceedings FourthInternational Conference onComputerVision,May1993,pp.658{665.

[43] Taubin,G.\ASignalProcessingApproachtoFairSurface De-sign",SIGGRAPHProceedings,1995,pp.351{358.

[44] Terzopoulos,D.\RegularizationofInverseVisualProblems In-volvingDiscontinuities",IEEETransactionsonPattern Analy-sisandMachineIntelligence,July1986,8(4).

[45] Terzopoulos,D.\TheComputationofVisibleSurface Represen-tations",IEEETransactions onPatternAnalysisandMachine Intelligence,July1988,10(4).

[46] Terzopoulos, D. and D. Metaxas. \Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics", IEEE Transactions on Pattern Analysis and Machine Intelli-gence,July1991,13(7).

[47] Turk,G.andM.Levoy.\ZipperedPolygonMeshesfromRange

[48] Turk,G.andJ.F.O'Brien.\ShapeTransformationUsingV aria-tionalImplicitFunctions",SIGGRAPHProceedings,Aug.1999, pp.335{342.

[49] Wahba,G.\SplineModelsforObservationalData",CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM, 1990.pp.11{14,pp.45{65.

[50] Yngve, G.andG.Turk.\RobustCreationof ImplicitSurfaces from PolygonalMeshes", To appearinACMTransactionson VisualizationandComputerGraphics.