Modelling3DAvatar forVirtualTryon
N A D I A M AG N E N AT Ǧ T H A L M A N N
D I R E C T O R M I R A L A B – U N I V E R S I T Y O F G E N E VA
D I R E C T O R I N S T I T U T E F O R M E D I A I N N OVAT I O N , N T U, S I N G A P O R E W W W. M I R A L A B . C H /
CreatingDigitalHumans
Gypsum Statue of
Humphrey Bogart Meshgrid plotted on the statue with a pencil
Vertex by vertex, each coordinate is transferred into
the computer intuitively
Model is rendered
Vertex by vertex design
Someothermodelsthatareusedformodeling(1987)
© MIRALab
Sculpting a geometric primitive
Box model
Intensiveuserinteraction
Ǧ FreeǦforminteractivemodelingwithageometricprimitive.
Rendered Model
FinalMesh
Peter Ratner, 3-D Human Modeling and Animation, Wiley Pub., 2003
CreatingDigitalHumans
3D Authoring tools for human body design
Digital human modeling software
©POSER
ǦUses Template ǦModels ǦPostures ǦMotions ǦGestures ǦTextures
©Facegen
CreatingDigitalHumans
3D Body Scanners
ǦAcquiresprecisesurfacestructure.
CreatingDigitalHumans
• 3Dbodyscanner
• Imagebased3Dscanner
• 80compactcamerassynchronizedandcontrolfromasinglecomputerplacedonhexagonalstructure
• Output:
• 80picturestakensimultaneouslyfromvariousangles
• 3Dreconstructedavatar
6
3Dscannersetup
CreatingDigitalHumans
• Postprocessing usingagisoft photoscan
• Generateautomaticallymasktofacilitatereconstruction
• Removegreenartefactontheskintoimprovetexturequality
7
Rawdata Mask Color correction
CreatingDigitalHumans
Rawdata (Pointcloud)
Ǧ OutlierFilt.
Ǧ NoiseFilt.
Ǧ Smoothing Ǧ HoleFilling
Ǧ Skeletonize Ǧ Skinning
Triangulation Skeleton
Adaptation
Animated
Model
I/O I/O I/O
Efficient
Rendering
I/O
Customized
BodySize Segmentation
ModelingPipeline
BodyScan Post-processing Animation Body sizing
Almost automatic 3DS Max and MIRALab plugins some user interaction
Industry Standard COLLADA format Scanner
System
Rapid Visualization Environment
AnimatingDigitalHumans
• B.Lafleur,N.MagnenatǦThalmann,D.Thalmann,ClothAnimationwithSelfǦCollision
Detection.Proc.IFIPConf.onGraphicsModeling,Tokyo,Japan,Springer,pp.179Ǧ197,
1991
• M.Carignan,Y.Yang,N.MagnenatǦThalmann,D.Thalmann,DressingAnimated
SyntheticactorswithComplexClothes.ComputerGraphics(Proc.SIGGRAPH'92),
ACMPress,Vol.26,No.2,pp.99Ǧ104,1992
EarlyClothSimulationModelsatMIRALab
MechanicalPropertiesofCloth
• Howtodescribethemechanicalbehavior ofcloth.
MechanicalModeling
• Howtosimulatethesepropertiesonvirtualcloth.
NumericalIntegration
• Howtosolvethedifferentialequationsresultingfromthemechanicalmodel.
CollisionDetection
• Howtodetectefficientlycontactsbetweenclothandotherobjects.
CollisionResponse
• Howtotakeintoaccountthesecontactsinthesimulation.
ProblemstobesolvedinClothingResearch
1.MechanicalProperties
•InternalForces(Fromsurfacedeformations)
• Elasticity(metric,curvature)
• Viscosity
• Plasticity
•ExternalForces(Fromenvironmentinteractions)
• Gravity,AirViscosity
• Contactreaction,Friction
• MiscellaneousInteractions
2.MechanicalModeling
• RepresentationofMechanicalParametersonaVirtual
ClothSurface
• Geometricalmodeling:
Surfacediscretization
• Triangles,Quadrangles,...
• Polygons,Bezierpatches,…
• Regulargrid,Arbitrarytopology,...
• Animation:
Timediscretization
• Successiveframesalongtime.
3.NumericalIntegration
•Themechanicalmodeldefinesalawrelatingforcetopositionand
speed.
• Newton’slawrelatesaccelerationtoforceandmass:
•Adifferentialsystemshouldberesolvedalongtime
• Hugenumberofdegreesoffreedom:
• Efficientmethodsareneeded.
F(P(t),P (t))c
F M ccP (t)
M P (t) cc F(P(t), c P (t))
4Ǧ5CollisionDetectionandResponse
• IntegratingCollisionEffectsintheMechanicalModel
• ReactionEffects
• Preventinterpenetrationofsurfaces.
• Necessaryforgeometricalconsistency.
• FrictionEffects
• Dissipativeforcesthatopposesliding.
• Contributetorealisticcontacteffects.
ExhibitionRobertPiguet
•ExhibitionRobertPiguet :“HighFashioninEquations”
• ProjectincollaborationwithTheSwissFashionMuseumofYverdonǦlesǦ Bains,Switzerland
•Contextofthecollaboration
• TheSwissFashionMuseumofYverdonǦlesǦBains hadtheprivilegeofreceiving3,000original
sketchesmadebyseveraltraineesofrenownedDesignerRobertPiguet.Theexhibit,titled
ExpositionRobertPiguet:From1933to1951,tracesMr.Piguet’s career.
ExhibitionRobertPiguet
•Newideasareillustratedwithaestheticdrawings.
Sketches by Marc Bohan, Hubert de Givenchy and Serge Guérin
ExhibitionRobertPiguet
•Havingthedesired3Dshapeinmind,the2Dpatternsare
designedbyexpertsaccordingtopatternconstructionrules:
ExhibitionRobertPiguet
Model designed by Givenchy
ExhibitionRobertPiguet
Model designed by Guèrin
ExhibitionRobertPiguet
Model designed by Givenchy
ExhibitionRobertPiguet alsofilmselectedatSIGGRAPH
ELECTRONICTHEATER(2007,SanDiego,USA)
WhatisaVirtualTryOn?
• An(online)application,allowingyoutotryonvirtualobjects(beforepurchase).
• Therearemany(online)examples
• Jewelry
• Watches
• Glasses
• Garments
AVirtualTryOnforgarments
• AVirtualTryOnforgarmentshasbeenapproachedfrommanydifferentangles
• Evaluategarmentsforstyle
• Thecommercialmajorityfallsintothiscategory
• Evaluategarmentsforfit
• Academicresearchandsomecommercialsolutions
• Aspectrumofapproaches
2D 2.5D 3D
•Photos of actual garments or Drawings
•Overlaid on a model’s body
•like paper dolls
•2D elements viewable from many angles
•Gives the illusion of 3D
•Input can come from either 2D or 3D
•Full 3D real-time rendering
•Both models and garments are meshes
•Possibly includes animation and simulation
VTOApproaches:2.5D
My Virtual Model Mimic Me
Optitex 3D Virtual Clothing
OurVirtualTryOn:Overview
• 3DApplication
• RealǦtimesimulationofgarments
• TryǦOnusingVirtualHuman
• RealǦtime3Davatarcreationbasedonusermorphology
• Garmentscustomisation
• VirtualmirrorusingMicrosoftKinect
• MultiǦdeviceandmultiǦusersystem
OurVirtualTryOn:Bodysizing
• Anaccuratebodyisessential
• To“tryon”clothing,youneedavirtualbodythatrepresentsyourown
• Startfromatemplatebody
• Generateabodywithyoursizesbasedonanthropometricdata
• Useapictureoftheusertoincreaseimmersionandrealism
InterfaceandInteraction
• Kinectclient
• Morphologyextractionusingdepthimage
InterfaceandInteraction
• Kinectclient
• RealǦtimereconstructionoftheanimation
