This may be the author’s version of a work that was submitted/accepted for publication in the following source:
Ross, Maureen,Cruz, Rena,Brooks-Richards, Trent,Hafner, Louise, Pow-ell, Sean, &Woodruff, Mia
(2018)
Comparison of three-dimensional surface scanning techniques for captur-ing the external ear.
Virtual and Physical Prototyping, 13(4), pp. 255-265.
This file was downloaded from: https://eprints.qut.edu.au/119416/
c
Consult author(s) regarding copyright matters
This work is covered by copyright. Unless the document is being made available under a Creative Commons Licence, you must assume that re-use is limited to personal use and that permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then refer to the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe that this work infringes copyright please provide details by email to [email protected]
Notice: Please note that this document may not be the Version of Record
(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) can be identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.
Comparison of three-dimensional scanning techniques for capturing the external ear
1
Maureen T Ross1, Rena Cruz1, Trent L Brooks-Richards1, Louise M Hafner1, Sean K Powell1,
2
Maria A Woodruff1 3
1. Institute of Health and Biomedical Innovation, Queensland University of Technology (QUT),
4
2 George Street, Brisbane, Australia 4000
5 6
2
Abstract
7
Congenital facial anomalies, such as microtia (malformation of the external ear), lead to
8
significant psychosocial effects starting from early childhood. Three-dimensional (3D)
9
scanning and advanced manufacturing are being investigated as a cheaper and more
10
personalised method of fabricating reconstructive treatments for patients compared to
11
traditional approaches. To date, most case studies have used expensive 3D scanners, yet, there
12
is potential for low-cost devices to provide comparable results. This study aimed to investigate
13
these different approaches.
14
Both ears of 16 adult participants were scanned with three devices: Artec Spider (Artec
15
Group), Intel® RealSense™ (Intel), and the Apple iPhone® 7 (Apple Inc.) combined with 16
photogrammetry using 90, 60 and 30 photographs. The scanning time, processing time,
17
accuracy, completeness, resolution and repeatability of each technique were assessed using the
18
Artec Spider as a reference scanner.
19
Our results show that the iPhone had the longest processing time however, this decreased
20
nine-fold when reducing the number of photos from 90 to 30. There was no significant
21
difference in the accuracy, completeness or repeatability of the iPhone scans with 90
22
photographs (1.4 ± 0.6 mm, 79.9%, 1.0 ± 0.1 mm), 60 photographs (1.2 ± 0.2, 79.3%,
23
0.9 ± 0.2 mm) or 30 photographs (1.2 ± 0.3 mm, 74.3%, 1.0 ± 0.2 mm). The Intel RealSesne
24
performed significantly worse in each parameter (1.8 ± 03 mm; 46.6%, 1.4 ± 0.5). Additionally
25
the RealSense had significantly lower resolution with not enough detail captured for the
26
application.
27
These results demonstrate that the ear can be accurately 3D scanned using iPhone
28
photographs. We would recommend capturing between 30 and 60 photographs of the ear to
29
create a scan that is accurate but without the downfall of long processing time. Using these
30
methods we aim to provide a more comfortable setting for the patient and a lower-cost and
31
more personalised ear prosthesis.
32 33
Keywords: 3D scanning; photogrammetry; advanced manufacturing; microtia; ear
34
reconstruction; ear prosthesis
3
Introduction
36
Microtia is a congenital malformation of the outer ear (auricle) which affects approximately
37
one in every 5,000 births worldwide (Luquetti et al. 2011). It is most commonly unilateral
38
(~90% of cases) (Brent 1999). This condition has the potential to cause significant psychosocial
39
and emotional effects for children, which can become permanent without intervention. There
40
are currently three restorative treatments available; surgical reconstruction using autografted
41
rib cartilage, surgical implantation of an alloplastic device, or the use of a personalised silicone
42
prosthesis (Tollefson 2006). Research suggests that early reconstructive intervention provides
43
favourable psychological outcomes for children by minimising the opportunity for harassment
44
from their peers (Johns et al. 2015, 2017).
45
Several case studies to date have shown that three-dimensional (3D) scanning, modelling
46
and printing are effective techniques for fabricating facial prostheses (Ciocca et al. 2007,
47
Subburaj et al. 2007, Liacouras et al. 2011, Mohammed et al. 2018, Unkovskiy et al. 2018) as
48
well as templates for autograft ear reconstruction (Flores et al. 2017, Lin et al. 2018, Ross et
49
al. 2018) and customised tissue-engineered ear scaffolds (Reiffel et al. 2013, Zhou et al. 2018). 50
However, the majority of these studies have used expensive scanners, software and 3D printers.
51
He et al.(2014) presented a low-cost approach using a desktop 3D printer to fabricate silicone
52
ear prostheses, demonstrating significantly lower costs than handmade approaches. There are
53
various techniques that can be used by 3D scanners capture the morphology of the patient’s
54
anatomy, as illustrated in Figure 1, however few studies have to date have looked at lower-cost
55
methods. Recently, Salazar-Gamarra et al. (2016) presented a case study demonstrating the
56
potential for a smartphone to capture 3D scans of a patient’s face using a free software
57
application (Autodesk 123D Catch®) that works by multi-photogrammetry. Unfortunately, this 58
software is no longer available although similar software exists on both smartphone and
59
computer platforms.
4
To date, limited studies have compared different surface scanning techniques for facial or
61
ear scanning (Koutny et al. 2012, Reichinger et al. 2013, Knoops et al. 2017, Volonghi et al.
62
2018). Despite having smaller sample sizes, these studies established well defined quantifiable
63
methods for comparing 3D scans. Aung et al. (1995) revealed that deviations larger than 2 mm
64
from a reference measurement (or scan) are unreliable, and do not result in accurate 3D models.
65
Based on this, Knoops et al. (2017) was able to conclude that certain 3D scanning devices were
66
accurate for 80-94% of 3D points within this range. However, the devices used in the
67
aforementioned case studies were all relatively expensive (USD 1,000 – 20,000). This study
68
aimed to quantitatively assess the potential for low-cost 3D scanning of the external ear using
69
smartphones. An Artec 3D Scanner was implemented as a reference against two low-cost
70
devices. It was expected that the smartphone would capture comparable scans to that of the
71
reference scanner for the purpose of creating 3D modelled designs for ear prostheses.
72 73 Methods 74 Participants 75
This study included 16 healthy adult participants without microtia (mean age: 26 ± 8 years,
76
8 Male, 8 Female). Institutional approval was obtained and all participants provided consent
77
for image acquisition and scientific publication (ethics approval number: 1600000770). Each
78
participant had both left and right ears scanned.
79
3D Scanning and Processing 80
Three different scanning devices with software unique to that device (Table 1) were used:
81
the Artec Spider Scanner (‘Artec’, Artec Group, Luxembourg), a metrology-rated structured
82
light scanner; the Intel® RealSense™ Camera SR300 (‘RealSense’, Intel, Santa Clara, CA, 83
USA) and an Apple iPhone® 7 (‘iPhone’, Apple Inc., Cupertino, CA, USA), with the
84
photographs processed using Agisoft PhotoScan (Agisoft LLC, Russia).
5
For each participant, both ears were scanned using all three scanning devices. Participant’s
86
hair was held back from around the ear using hair-ties and hair-pins, to prevent obsturction
87
during scanning. The participants were seated and asked to remain still during the scan
88
acquisition. To ensure reliable comparisons and to negate user error, all scans were captured by
89
the same operator at the same location.
90
Each scanning approach used different methods as described below:
91
1. Artec – the handheld scanner was moved around the participant’s ear at a distance of
92
0.17 – 0.3 m until the ear captured from all angles. The data was processed in Artec
93
Studio 11 (Artec Group, Luxembourg) using the autopilot tool. The points were cropped
94
to eliminate excess information around the ear. The autopilot tool was used to process
95
a final rendered 3D scan. This was exported into Meshlab (Cignoni et al. 2008) where
96
further cropping around the area of interest was performed before being exported as a
97
stereolithography (STL) file.
98
2. RealSense – the 3D point cloud was captured with the DF_3DScan software (Intel,
99
Santa Clara, CA, USA). The handheld scanner was held away from the participant’s ear
100
at approximately 0.5 m so that the ear was in the field of view. Once the scan started the
101
scanner was moved to capture around the ear. The scan was automatically rendered and
102
exported as an OBJ file. Meshlab was used to crop the scan and scale to the dimensions
103
of the Artec scan of the same ear. The scan was then exported as a STL file.
104
3.
iPhone – All photographs were taken using the Camera + iOS application (tap tap tap105
LLC, USA). This provided improved control of the white balance, ISO, shutter speed
106
and focus of the photographs compared to the default iPhone camera application. At the
107
start of each scan, these settings were selected and locked to remain constant for the
108
scan duration. 90 photographs were captured of each ear at a distance of approximately
109
10-15 cm. Figure 2a shows a guide for taking the photographs to ensure consistency.
6
The photographs were uploaded and imported into the program Agisoft Photoscan
111
Standard Edition (‘Agisoft’, Agisoft LLC, Russia) as JPEG files. Three steps were
112
completed: alignment of photographs, generation of a dense point cloud and finally
113
meshing. These steps were all completed using the highsetting on the Agisoft software.
114
Each scan was exported as an STL file and subsequently imported into Meshlab for
115
cropping and scaling. Before being exported, the scan was smoothed using a Laplacian
116
smoothing filter with a step of 15 (assessment of smoothing filter can be seen in
117
Supplementary Materials Figure 1).
118
3.1 To investigate whether taking fewer photos would lead to lower 3D scan quality, we
119
compared different amounts of photograph sets. A MATLAB® (The MathWorks Inc., 120
Natick, MA, USA) script was written to randomly select sets of 60 and 30 photographs
121
from the full 90 photograph set for each participant’s ear. 3D scans were then produced
122
for each of these small photograph sets using the method previously described. These
123
were compared according to the analysis parameters described hereafter. A minimum
124
of 30 photographs was used for this method as below this was found to have a large
125
effect on results (Supplementary Material – Table S1).
126
Analysis 127
Six parameters were used to assess and compare the methods; acquisition time, processing
128
time, accuracy, completeness, resolution, and repeatability. 129
Acquisition and processing time were measured and recorded for each ear scan performed 130
to assess the suitability of each method.
131
Accuracy was defined as the deviation between scans (RealSense/iPhone) and the reference 132
scan (Artec), which is metrology rated and acts as the gold standard. Using the software
133
CloudCompare (Girardeau-Montaut 2017), all scans were aligned to the reference scan,
134
cropped to identical boundaries, and then exported as an STL file (Figure 2b). These STL files
7
were used for the analysis parameters described hereafter. In CloudCompare, the
‘cloud-to-136
mesh’ tool was used to calculate the directional deviation (mm) between each 3D point in the 137
scans (RealSense/iPhone) and the closest 3D point in the reference scan (Artec). The results
138
were exported in software specific data format (VTK). For each participant scan, the root mean
139
square (RMS) value was calculated. The VTK file was also imported into Paraview (Kitware
140
Inc., Clifton Park, NY, USA) to visualise scaled colour maps of the absolute deviation between
141
scans.
142
Completeness was defined as the percentage of points in the RealSense/iPhone scans that 143
were within 2 mm of the corresponding points in the Artec scan (Aung et al. 1995) The data
144
was analysed using the ‘cloud-to-mesh’ tool in CloudCompare with the exception that the scan
145
being analysed was set as the reference scan and the Artec scan was the compared experimental
146
scan. By defining the scans in this way, the number of points in the Artec scan would be
147
considered ‘100%’. The number of points in the Artec scan that were within 2 mm of the scan
148
being analysed would then be considered ‘captured’. The VTK file containing the data was then
149
exported and the number of points within 2 mm were described as a percentage of the total
150
number of points within the scans.
151
Resolution was calculated using the ‘Area and Volume’ plugin (Vonc 2015) for MAXON 152
Cinema 4D (MAXON Computer GmbH, Friedrichsdorf, Germany). The polygon count and
153
area (mm2) of each scan was recorded. A higher polygon count per area indicated higher
154
resolution compared to a lower count per area.
155
Repeatability was analysed by scanning one participant’s ear five times with each device. 156
The scans were all taken using the same method as detailed previously. A 20 minute break was
157
taken between each set of scans to mitigate some of the learning effect from repeated scanning.
158
To compute the repeatability measure, the RMS value between all corresponding 3D points in
159
each pair of 3D scans was calculated (Scan 1 vs 2, 1 vs 3, 1 vs 4, 1 vs 5, 1 vs 6, 2 vs 3, etc.,
8
with ten pairs in total). The RMS values were calculated for each, and then an overall RMS
161
(mean and standard deviation) for each device was given.
162
Statistical Analysis 163
Graphpad Prism 7 (GraphPad Software Inc., San Diego, CA, USA) was used for statistical
164
analysis. A one-way ANOVA with Tukey’s multiple comparison test (or the non-parametric
165
equivalent as required) was used to compare each parameter. P-values of < 0.05 were
166 considered significant. 167 168 Results 169
The results from each of the analysis parameters: scanning time, processing time,
170
accuracy, completeness, resolution, repeatability and smoothing are detailed below. Figure 3
171
displays the 3D scans captured from one of the participants.
172 173
Acquisition Time 174
All three scanning devices had scan time per ear of 2 minutes or less (Artec:
175
2.0 ± 0.7 minutes; RealSense: 0.5 ± 0.2 minutes; iPhone: 1.9 ± 0.2 minutes). This is incredibly
176
important for the translation to clinical use with children as they find it difficult to remain still
177
for long periods, hence keeping scan times as short as possible is preferential. The RealSense
178
had a significantly shorter scan time (p < 0.0001) compared to the Artec and iPhone with no
179
significant difference found between Artec and iPhone scanning times.
180
Processing Time 181
A larger variation in processing time was observed between techniques, as shown in
182
Figure 4a. Although the RealSense instantly produced a 3D scan, it required manual cropping
183
and scaling (1.2 ± 0.5 minutes). The Artec on average took 8.9 minutes (± 1.8 minutes) to
184
render the final 3D scan. The processing time varied greatly for the iPhone, with
9
photogrammetry depending on the number of photographs used to create the scan (90
186
photographs: 211.4 ± 115.4 minutes; 60 photographs: 80.8 ± 62.5 minutes; 30 photographs:
187
24.6 ± 12.0 minutes). Furthermore, it was noted that the more photographs in the set, the larger
188
the standard deviation. It should be noted that the processing times for the iPhone scans only
189
required an operator to be present for approximately 10 minutes of the total processing time.
190
Accuracy 191
The directional deviation values were used to calculate RMS distance for each scan
192
(Figure 4b). A lower RMS value indicates the scan is of higher accuracy. The RealSense had a
193
significantly higher RMS distance (1.8 ± 0.3 mm) compared to each of the iPhone experimental
194
groups (p < 0.0001). Interestingly, RMS distance appeared to be independent of the number of
195
iPhone photographs used (90 photographs: 1.4 ± 0.6 mm; 60 photographs: 1.2 ± 0.2 mm;
196
30 photographs: 1.2 ± 0.3 mm, with no significant difference between groups p > 0.2). Colour
197
maps were also created to visualise areas of higher deviation (Supplementary Materials Figure
198
2).
199
Completeness 200
Figure 4c illustrates the completeness results averaged over all scans in the study. The
201
RealSense scanner had significantly lower completeness compared to each iPhone scan
202
(46.6 ± 7.8%, p < 0.001). There was no significant difference in the completeness of the iPhone
203
scans using 90, 60 or 30 photographs (79.7 ± 9.6%; 79.6 ± 8.6%; 74.3 ± 9.1% respectively, p >
204
0.07 between groups). This is consistent with the accuracy results. Figure 5 shows the points
205
that were captured by the RealSense and iPhone within 2mm of the Artec scan for one
206
participant as a representative of the cohort (additional participants can be seen in
207
Supplementary Materials 3 and 4). It was observed that the cavities of the ear and the lower and
208
upper sections behind the ear were the most common features not able to be captured within a
209
reliable range.
10 Resolution
211
Resolution was extremely low for the RealSense (2.2 ± 0.4 polygons/mm2, p < 0.0001) 212
with very little detail attained, and much of the scan had been over smoothed.. As expected, the
213
Artec scanner had the highest resolution of 68.3 ± 17.8 polygons/mm2. There was no significant
214
difference in the resolution of the different experimental groups for iPhone
215
(90 photographs: 31.2 ± 9.5; 60 photographs: 27.0 ± 7.2; 30 photographs: 25.5 ± 10.8, p > 0.4
216
when comparing 90 vs 30, 90 vs 60 and 60 vs 30) although the iPhone with 30 photographs was
217
rougher than the other scans (Supplementary Material – Table S2).
218
Repeatability 219
Figure 4d shows the repeatability measure for each scanning technique. The Artec
220
demonstrated a significantly higher repeatability than the RealSense and the iPhone in all cases
221
(0.5 ± 0.1 mm, p < 0.002). The RealSense was significantly less reliable than all other devices
222
(1.4 ± 0.5 mm, p < 0.01). There was no significant difference between the repeatability of scans
223
produced by the iPhone (90 photographs: 1.0 ± 0.1 mm; 60 photographs: 0.9 ± 0.2 mm; 30
224 photographs 1.0 ± 0.2 mm, p > 0.9). 225 226 Discussion 227
3D scanning and advanced manufacturing of prosthetics offer significant advantages over
228
traditional hand-crafting approaches. The technology can provide a more comfortable
229
procedure for the patient, increase customisation and satisfaction whilst reducing cost and
230
labour. Much of the current literature demonstrating the advantages of this technology however,
231
are based on the use of relatively expensive 3D scanners and 3D printers. These costs can limit
232
their translation to clinical use. With the advent of sophisticated computer software and readily
233
available smartphones containing high-quality cameras, the possibility exists for high-quality
234
scanning to be achieved at a comparatively low-cost compared to high-end scanners. As such,
11
the aim of this study was to use a metrology rated Artec 3D scanner as a reference to compare
236
two low-cost scanning devices, the RealSense and an iPhone for 3D surface scanning of the
237
ear.
238
Each of the scanning techniques was fundamentally different in their approach to scan the
239
subject; the Artec and RealSense scanners projecting patterns onto the object and measuring
240
changes in these patterns due to the shape of the object with cameras (Artec – three, Realsense
241
– one), , and the iPhone photogrammetry approach using several photograph of the object taken
242
from different locations and triangulating common features in the photographs to establish 3D
243
locations . All of the devices chosen were hand held, rather than static. While static scanners
244
are cheap and cost effective, multiple scans from different angles are still required requiring the
245
patient to move and rotate into potentially uncomfortable positions. As such hand held scanners
246
would increase patient comfort and reduce post-processing required by aligning multiple scans
247
(Volonghi et al. 2018). By establishing the validity of a new, more cost-effective scanning
248
technique, the clinical take-up of additive manufacturing for prosthetic fabrication could be
249
accelerated. In order to evaluate their suitability as alternatives to the relatively expensive Artec
250
scanner, we compared the scanning devices based on scanning and processing time, accuracy,
251
completeness, resolution and repeatability (Table 2).
252
The accuracy and completeness of each scanning device was quantified by calculating how
253
far each point deviated from the reference scan. The scans generated using the iPhone were
254
found to be significantly more accurate and complete than those generated by the RealSense
255
(Figure 4). Whilst the RealSense had an accuracy within the 2 mm threshold(Aung et al. 1995),
256
it only captured 47% of the ear, thus making it unacceptable for clinical use. It was a significant
257
finding of this study that the accuracy and completeness of the scans generated from the iPhone
258
was statistically independent of whether 90, 60 or 30 photographs were used. This would lead
259
to shorter scanning time (beneficial for the patient and the clinician) without compromising
12
scan quality. Furthermore, the processing time would be significantly reduced (up to nine fold)
261
resulting in a more streamlined prostheses fabrication process. The iPhone average of
262
75 - 80% completeness (no significance between the number of photographs used) is only
263
slightly lower than what has been shown for facial scanning with other devices
264
(80 – 94% completeness)(Knoops et al. 2017), with the differences mostly due to the difficulty
265
capturing features behind the ear and around the ear canal.
266
Of the three techniques used, the Artec was the only device able to give real-time feedback
267
whilst scanning. During scanning with the Artec, the operator was able to report that tracking
268
was frequently lost behind the ear (tracking was recovered before continuing the scan as to not
269
affect the results). This is likely due to poor accessibility by the scanner (it was reportedly easier
270
to scan participants with ears that projected at a larger angle from the head). This loss of detail
271
behind the ear was also noted in the scans produced from the RealSense and iPhone after
272
processing. This is in agreement with the finding of Reichinger et al. (2013) detailing the
273
difficulties in scanning the external ear. However, it should be considered that as the back of
274
the prosthetic ear would not likely be visible once the prosthesis is attached to the patient, the
275
missing information could be interpolated or a parametric model can be used when 3D
276
modelling the prosthesis design prior to 3D printing (Bos et al. 2015). The disadvantage,
277
however, is the increased modelling time required to add this detail to the prosthetic design.
278
Additionally, other areas around the ear canal that were also hard to capture would interface
279
with the patient’s skin thus not requiring modelling of a silicone substitute (Mohammed et al.
280
2018).
281
While this study provided a valuable quantitative assessment of low-cost scanning
282
approaches, the participant demographic was healthy, unaffected adults. This was chosen on
283
the guidance of our University ethics committee and a clinical ethics committee as children are
284
known to be a vulnerable participant group. As it has been shown that the ear reaches 87% of
13
its size by the time a child is five years of age, as well as patients not being limited to children
286
we believe our data is still novel (Farkas et al. 1992). Nonetheless it would be beneficial in
287
further studies to include child participants (16 years and under) who are and are not affected
288
by microtia to develop intuitive, practical and clinically applicable methodologies. It is
289
expected that children will add a level of complexity due to less self-control of movement which
290
could interrupt the scan acquisition. However, it is hoped that providing a comfortable
291
environment with the use of technology and video, it will be possible to have the child focus on
292
a point of interest to keep them still for the duration of the scanning. Additionally, this study
293
focused on using the iPhone 7 for capturing the photographs for photogrammetry. It is our goal
294
in future research to translate this low-cost and accessible technique to other devices as well as
295
older and newer generations of smartphones. This is important given the variation in the
296
resolution of smartphone cameras and automatic processing of the photographs by the camera’s
297
software. One additional consideration that will have to be addressed is regulation surrounding
298
smartphone use in a clinical setting. It is known that doctors and clinicians already use
299
smartphones to capture information about patients (Burns and Belton 2013, Abbott et al. 2017)
300
but proper regulations and procedures around the storage of this content will need to be
301
established.
302
For photogrammetry to accurately capture 3D scans of ears for 3D printed prostheses,
303
photographs of the ear must be taken from well-distributed distances and angles to ensure
304
effective coverage (Salazar-Gamarra et al. 2016). The Autodesk 123D Catch® application 305
guided the operator on where to take photographs around the object they were scanning. To
306
replicate this, we used preliminary experiments to develop an operator guide on the best way
307
to take 90 photographs of the ear for 3D scanning with photogrammetry. To further increase
308
the repeatability of iPhone scans, a guide for taking 30 photographs, as well as a short training
309
session or video would be recommended for clinical use.
14
In conclusion, 3D scanning and advanced manufacturing show great potential to generate
311
new innovative treatments for microtia. This study aimed to compare low-cost devices for 3D
312
surface scanning the ear with a relatively high-cost, high-end industrial 3D scanner. The
313
iPhone 7 combined with photogrammetry produced 3D scans accurate to 1.2 mm with potential
314
for 3D modelling, and ultimately printing of a personalised prosthetic ear. We found the
315
accuracy, completeness and repeatability of the 3D scans produced by the iPhone 7 were 316
significantly greater than those produced by the RealSense scanner (p < 0.0001). Of particular
317
interest was the observation that the quality of the iPhone photogrammetry approach appeared
318
independent of whether 90, 60 or 30 photographs were used. This is an important finding as the
319
processing time was nine-fold faster with 30 photographs used (25 minutes), compared to 90
320
(211 minutes). The use of smartphones for low-cost scanning of the ear could provide an
321
accessible and affordable way for clinics to introduce new best practice methods to the clinic.
322
With the assumption that a smartphone with camera capabilities would already be available,
323
the cost of this technique would equate to ~USD 179 (Agisoft Photoscan software) which is
324
less than 1% of the cost of an Artec Spider 3D scanner (~ USD 22,500), and is a highly
325
compelling reason to adopt this approach.
326 327
15
Conflict of interest
328
The authors have no conflicts of interest to disclose.
329 330 331 332 333 334 335 336 337 338 339 340
16 References
341
Abbott, L.M., Magnusson, R.S., Gibbs, E., and Smith, S.D., 2017. Smartphone use in
342
dermatology for clinical photography and consultation: Current practice and the law.
343
Australasian Journal of Dermatology, (August 2016), 101–107. 344
Aung, S.C., Ngim, R.C.K., and Lee, S.T., 1995. Evaluation of the laser scanner as a surface
345
measuring tool and its accuracy compared with direct facial anthropometric
346
measurements. British Journal of Plastic Surgery, 48 (8), 551–558.
347
Bos, E.J., Scholten, T., Song, Y., Verlinden, J.C., Wolff, J., Forouzanfar, T., Helder, M.N.,
348
and Van Zuijlen, P., 2015. Developing a parametric ear model for auricular
349
reconstruction: A new step towards patient-specific implants. Journal of
Cranio-350
Maxillofacial Surgery, 43 (3), 390–395. 351
Brent, B., 1999. Technical advances in ear reconstruction with autogenous rib cartilage grafts:
352
Personal experience with 1200 cases. Plastic and Reconstructive Surgery, 104 (2), 319–
353
334.
354
Burns, K. and Belton, S., 2013. Clinicians and their cameras: Policy, ethics and practice in an
355
Australian tertiary hospital. Australian Health Review, 37 (4), 437–441.
356
Cignoni, P., Cignoni, P., Callieri, M., Callieri, M., Corsini, M., Corsini, M., Dellepiane, M.,
357
Dellepiane, M., Ganovelli, F., Ganovelli, F., Ranzuglia, G., and Ranzuglia, G., 2008.
358
MeshLab: an Open-Source Mesh Processing Tool. Sixth Eurographics Italian Chapter
359
Conference, 129–136. 360
Ciocca, L., Mingucci, R., Gassino, G., and Scotti, R., 2007. CAD/CAM ear model and virtual
361
construction of the mold. Journal of Prosthetic Dentistry, 98 (5), 339–343.
362
Farkas, L.G., Posnick, J.C., and Hreczko, T.M., 1992. Anthropometric Growth Study of the
17 Head. The Cleft Palate-Craniofacial Journal, 29 (4), 303–308.
364
Flores, R.L., Liss, H., Raffaelli, S., Humayun, A., Khouri, K.S., Coelho, P.G., and Witek, L.,
365
2017. The technique for 3D printing patient-specific models for auricular reconstruction.
366
Journal of Cranio-Maxillofacial Surgery, 45 (6), 937–943. 367
Girardeau-Montaut, D., 2017. CloudCompare - An open source project.
368
He, Y., Xue, G.H., and Fu, J.Z., 2014. Fabrication of low cost soft tissue prostheses with the
369
desktop 3D printer. Scientific Reports, 4, 1–7.
370
Johns, A.L., Lewin, S.L., and Im, D.D., 2017. Teasing in younger and older children with
371
microtia before and after ear reconstruction. Journal of Plastic Surgery and Hand
372
Surgery, 51 (3), 205–209. 373
Johns, A.L., Lucash, R.E., Im, D.D., and Lewin, S.L., 2015. Pre and post-operative
374
psychological functioning in younger and older children with microtia. Journal of
375
Plastic, Reconstructive and Aesthetic Surgery, 68 (4), 492–497. 376
Knoops, P.G.M., Beaumont, C.A.A., Borghi, A., Rodriguez-Florez, N., Breakey, R.W.F.,
377
Rodgers, W., Angullia, F., Jeelani, N.U.O., Schievano, S., and Dunaway, D.J., 2017.
378
Comparison of three-dimensional scanner systems for craniomaxillofacial imaging.
379
Journal of Plastic, Reconstructive and Aesthetic Surgery, 70 (4), 441–449. 380
Koutny, D., Palousek, D., Koutecky, T., Zatocilova, a, Rosicky, J., and Janda, M., 2012. 3D
381
Digitalization of the Human Body for Use in Orthotics and Prosthetics. World Academy
382
of Science, Engineering and Technology, 72 (12), 1487–1494. 383
Liacouras, P., Garnes, J., Roman, N., Petrich, A., and Grant, G.T., 2011. Designing and
384
manufacturing an auricular prosthesis using computed tomography, 3-dimensional
18 photographic imaging, and additive manufacturing: A clinical report. Journal of
386
Prosthetic Dentistry, 105 (2), 78–82. 387
Lin, A.J., Bernstein, J.L., and Spector, J.A., 2018. Ear Reconstruction and 3D Printing: Is It
388
Reality? Current Surgery Reports, 6 (2), 4.
389
Luquetti, D.V., Leoncini, E., and Mastroiacovo, P., 2011. Microtia-anotia: A global review of
390
prevalence rates. Birth Defects Research Part A - Clinical and Molecular Teratology, 91
391
(9), 813–822.
392
Mohammed, M.I., Cadd, B., Peart, G., and Gibson, I., 2018. Augmented patient-specific
393
facial prosthesis production using medical imaging modelling and 3D printing
394
technologies for improved patient outcomes. Virtual and Physical Prototyping, 0 (0), 1–
395
13.
396
Reichinger, A., Majdak, P., Sablatnig, R., and Maierhofer, S., 2013. Evaluation of methods
397
for optical 3d scanning of human pinna. International Conference IEEE, 390–397.
398
Reiffel, A.J., Kafka, C., Hernandez, K.A., Popa, S., Perez, J.L., Zhou, S., Pramanik, S.,
399
Brown, B.N., Ryu, W.S., Bonassar, L.J., and Spector, J.A., 2013. High-Fidelity Tissue
400
Engineering of Patient-Specific Auricles for Reconstruction of Pediatric Microtia and
401
Other Auricular Deformities. PLoS ONE, 8 (2).
402
Ross, M.T., Cruz, R., Hutchinson, C., Arnott, W.L., Woodruff, M.A., and Powell, S.K., 2018.
403
Aesthetic reconstruction of microtia: a review of current techniques and new 3D printing
404
approaches. Virtual and Physical Prototyping, 13 (2), 1–14.
405
Salazar-Gamarra, R., Seelaus, R., Da Silva, J.V.L., Da Silva, A.M., and Dib, L.L., 2016.
406
Monoscopic photogrammetry to obtain 3D models by a mobile device: A method for
407
making facial prostheses. Journal of Otolaryngology - Head and Neck Surgery, 45 (1),
19 1–13.
409
Subburaj, K., Nair, C., Rajesh, S., Meshram, S.M., and Ravi, B., 2007. Rapid development of
410
auricular prosthesis using CAD and rapid prototyping technologies. International
411
Journal of Oral and Maxillofacial Surgery, 36 (10), 938–943. 412
Tollefson, T.T., 2006. Advances in the treatment of microtia. Current Opinion in
413
Otolaryngology and Head and Neck Surgery, 14 (6), 412–422. 414
Unkovskiy, A., Spintzyk, S., Brom, J., Huettig, F., and Keutel, C., 2018. Direct 3D printing of
415
silicone facial prostheses: A preliminary experience in digital workflow. Journal of
416
Prosthetic Dentistry, 1–6. 417
Volonghi, P., Baronio, G., and Signoroni, A., 2018. 3D scanning and geometry processing
418
techniques for customised hand orthotics: an experimental assessment. Virtual and
419
Physical Prototyping, 13 (2), 105–116. 420
Vonc, C., 2015. Aire et Volume [online]. Available from: http://code.vonc.fr/?a=40.
421
Zhou, G., Jiang, H., Yin, Z., Liu, Y., Zhang, Q., Zhang, C., Pan, B., Zhou, J., Zhou, X., Sun,
422
H., Li, D., He, A., Zhang, Z., Zhang, W., Liu, W., and Cao, Y., 2018. In Vitro
423
Regeneration of Patient-specific Ear-shaped Cartilage and Its First Clinical Application
424
for Auricular Reconstruction. EBioMedicine, 28, 287–302.
425 426 427
20 Table 1. The scan devices used to scan each participant.
428
Device Artec Spider (Artec Group Luxembourg)
Intel® RealSense™
Camera SR300 (Intel, Santa Clara, CA,USA)
Apple iPhone® 7 (Apple
Inc., Cupertino, CA, USA)
Technique Structured Light Structured Light Photogrammetry Manufacturers Specifications Depth Range: 0.17–0.3 m Frame/Sec: 7.5 3D Resolution: 0.1 mm 3D Point Accuracy: 0.05 mm Texture Resolution: 1.3 MP Depth Range: 0.3-1.5 m Frame/Sec: 30 RGB Resolution: 1080p Resolution: 12 MP Aperture: ƒ/1.8 Cost (USD) $22,500 $150 $549
Software Artec Studio 11 (Artec Group, Luxembourg)
DF_3DScan (Intel, Santa Clara, CA, USA)
Camera+ iOS application (tap tap tap LLC, USA) and Agisoft PhotoScan Standard Edition (Agisoft LLC, Russia)
Cost (USD) 20 seats included with scanner
Included with scanner $4.49 (Camera+) $179 (Agisoft) 429
21 Table 2. Summary of results. Mean and standard deviation shown for all participants (n = 32).
431 Artec Spider Intel RealSense iPhone 7 90 Photos iPhone 7 60 Photos iPhone 7 30 Photos Time to Scan (minutes) 2.0 ± 0.7 0.5 ± 0.2 1.9 ± 0.2 1.3 # 0.6# Time to Process (minutes) 8.9 ± 1.8 1.2 ± 0.5 211.4 ± 115.4 80.8 ± 65.5 24.6 ± 12.0 Accuracy (RMS (mm)) Reference 1.8 ± 0.3 1.4 ± 0.6 1.2 ± 0.2 1.2 ± 0.3 Completeness (%) Reference 46.6 ± 7.9 79.7 ± 9.6 79.6 ± 8.6 74.3 ± 9.1 Resolution (polygons/mm2) 68.3 ± 17.8 2.2 ± 0.38 31.2 ± 9.5 27.0 ± 7.2 25.5 ± 10.8 Reliability (RMS (mm)) 0.5 ± 0.1 1.4 ± 0.5 1.0± 0.1 0.9 ± 0.2 1.0 ± 0.2
# As these experimental groups were randomly selected from the 90 photographs taken these time 432
are an estimate based on the scan time for 90 photographs. 433
22
434
Figure 1. Schematic of different techniques by which 3D scanners capture information. A)
435
Structured light scanning, B) laser triangulation, C) time of flight, D) stereo-photogrammetry,
436
E) multi-photogrammetry.
23
438
Figure 2. A) Guide for taking 90 photographs of the ear with the iPhone 7 i) 10 photographs
439
moving horizontally from 5 different viewing angles, ii) 10 photographs moving vertically from
440
2 viewing angles – in front and behind the ear, iii) 20 photographs at closer range to capture
441
narrow cavities are more detailed features. B) Method for aligning 3D scans for comparison i)
442
the Artec scan was used as the reference, ii) the RealSense and iPhone scans were aligned to
443
the reference, iii) and then all scans were cropped to the same boundaries.
444 445
24
446
Figure 3. The 3D scans captured of one participants left ear viewed from the front and behind
447
the ear. Photographs of the participants ear on the far left. Screenshots of the 3D scans are on
448
the right. Highlighted are areas of hard to scan regions where the Artec and RealSense software
449
has poorly interpolated missing data, creating incorrect anatomy. For the Artec (reference scan)
450
this interpolated data made up 1% of the total scan area.
451 452
25
453
Figure 4. Results from analysis, n = 32 except where stated otherwise. A) Processing time
454
– including cropping, scaling, rendering and any other computer processing time. B) Accuracy
455
– defined by the root mean square (RMS) distance (mm), RMS distances closer to zero are more
456
accurate. C) Completeness – percentage of data points in the Artec scan (reference) that were
457
captured within 2 mm by each device. D) Repeatability – defined by the RMS (mm). One
458
participant was scanned five times with each device and scans were compared for comparison
459
(n = 10).* p < 0.01 when the RealSense was compared to the iPhone for all groups, ** p < 0.002
460
when the Artec was compared to all other devices, *** p < 0.0001 when the RealSense was
461
compared to the iPhone for all groups.
462 463 464
26
465
Figure 5. Completeness of each scanning technique displayed by the points in the reference
466
scan (Artec, 100%) that were captured within 2 mm. Scans for one participant’s ear are shown
467
at two viewing angles: in front of the ear and behind the ear. Individual completeness scores
468
for participant are included.
469 470
27
Comparison of three-dimensional surface scanning techniques for
471
capturing the external ear
472
Maureen T Ross, Rena Cruz, Trent L Brooks-Richards, Louise M Hafner, Sean K Powell, Maria A 473 Woodruff 474 Supplementary Material 475 476
Figure S1. Assessment of step level for Laplacian smoothing filter. A) Effect of smoothing
477
filter on iPhone scan from 30 photographs for one participant. The effect of smoothing
478
appears to plateau past step 15. B) The accuracy results of the iPhone with 30 photographs
479
with different filter steps (n = 32). C) The completeness results of the iPhone with 30
480
photographs with different filter steps (n = 32). Based on these results a step of 15 was chosen
481
as past this no major benefit was seen.
482 483
28
484
Figure S2. Colour maps of the absolute deviation (mm) which was calculated as the distance
485
between a point in the scan (RealSense, iPhone) and its closest point in the reference scan
486
(Artec). A) Participant with average RMS value to represent the cohort. B) Participant with
487
the best RMS value. C) Participant with a poor RMS value. Artec shown in grey as reference
488
scan. Deviation scale set from 0 mm (blue) to 5 mm (red). Individual RMS values are
489
displayed for each participant scan.
490 491
29
492
Figure S3. Additional examples of percentage completeness for participants. A) Participant
493
with best percentage completeness. B) Participant with poor percentage completeness.
494
Completeness of each scanning technique displayed by the points in the reference scan (Artec,
495
100%) that were captured within 2 mm. Individual completeness cores are displayed for each
496
participant scan.
497 498 499
30 Table S1. Assessment of lower threshold of photos to use for photogrammetry with the
500
iPhone 7. One participant was sampled for batches of 20, 15 and 10 photographs in triplicate.
501
These was assessed using the same accuracy and completeness parameters used in the study.
502
It was noted that one replicate for the 10 photographs group did not process as too few points
503
were found in the set. There was a large reduction in completeness after 20 photos.
504 505 Photographs 90 60 30 20 15 10 Accuracy (mm) 0.93 0.94 1.23 1.09 0.99 1.01# Completeness (%) 94.1 93.4 82.3 71.1 70.9 64.4#
# one triplicate from this group did not process as not enough common points were found throughout 506
the set. 507
31 Table S2. Median surface roughness of each scanning technique. Roughness was averaged
509
over three sample areas of the ear for each participant. Each sample area was exported in ply
510
format and then assessed using a Matlab script based off “Gridfit” (D’Errico 2016). The
511
iPhone 7 with 30 photographs was the only group that was statically rougher than the Artec
512
(reference). The Intel RealSense was smoother however not significant (p = 0.0875).
513 514 Artec Spider Intel RealSense iPhone 7 90 Photos iPhone 7 60 Photos iPhone 7 30 Photos Median Surface Roughness (mm) 0.012 0.005 0.019 0.025 0.040* P value Reference 0.0875 >0.9999 0.2528 0.0031
* Statistically significant compared to the Artec (p < 0.05) 515
