Article
Published Version
Creative Commons: Attribution 4.0 (CCBY)
Open Access
Ojha, V. K., Griego, D., Kuliga, S., Bielik, M., Bus, P.,
Schaeben, C., Treyer, L., Standfest, M., Schneider, S., König,
R., Donath, D. and Schmitt, G. (2019) Machine learning
approaches to understand the influence of urban environments
on human’s physiological response. Information Sciences,
474. pp. 154169. ISSN 00200255 doi:
https://doi.org/10.1016/j.ins.2018.09.061 Available at
http://centaur.reading.ac.uk/82144/
It is advisable to refer to the publisher’s version if you intend to cite from the
work. See
Guidance on citing
.
To link to this article DOI: http://dx.doi.org/10.1016/j.ins.2018.09.061
Publisher: Elsevier
All outputs in CentAUR are protected by Intellectual Property Rights law,
including copyright law. Copyright and IPR is retained by the creators or other
copyright holders. Terms and conditions for use of this material are defined in
the
End User Agreement
.
www.reading.ac.uk/centaur
CentAUR
Central Archive at the University of Reading
Reading’s research outputs online
urban
environments
on
human’s
physiological
response
Varun
Kumar
Ojha
a,∗,
Danielle
Griego
a,
Saskia
Kuliga
b,
Martin
Bielik
b,
Peter
Buš
a,
Charlotte
Schaeben
a,
Lukas
Treyer
a,
Matthias
Standfest
a,
Sven
Schneider
b,
Reinhard
König
c,
Dirk
Donath
b,
Gerhard
Schmitt
aa Chair of Information Architecture, ETH Zürich, Switzerland
b Chair of Computer Science in Architecture, Bauhaus University, Weimar, Germany c Computational Architecture, Bauhaus University, Weimar, Germany
a
r
t
i
c
l
e
i
n
f
o
Article history:
Received 4 February 2018 Revised 7 August 2018 Accepted 26 September 2018 Available online 26 September 2018
Keywords: Signal processing Data fusion Features selection Wearable devices Physiological data
a
b
s
t
r
a
c
t
Thisresearchproposesaframeworkforsignalprocessingandinformationfusionof spatial-temporalmulti-sensordatapertainingtounderstandingpatternsofhumansphysiological changesinan urban environment.The framework includes signalfrequency unification, signal pairing,signal filtering,signal quantification,and datalabeling.Furthermore, this papercontributestohuman-environmentinteractionresearch,whereafieldstudyto un-derstandtheinfluenceofenvironmentalfeaturessuchasvaryingsoundlevel,illuminance, field-of-view, orenvironmentalconditions onhumans’ perception was proposed.In the study,participantsofvariousdemographicbackgroundswalkedthroughanurban environ-mentinZürich,Switzerlandwhilewearingphysiologicalandenvironmentalsensors.Apart fromsignalprocessing,fourmachinelearningtechniques, classification,fuzzyrule-based inference,featureselection,andclustering,wereappliedtodiscoverrelevantpatternsand relationshipbetweenthe participants’physiologicalresponsesand environmental condi-tions. Thepredictivemodels withhighaccuracies indicatethatthe changeinthe field-of-viewcorrespondstoincreasedparticipantarousal.Amongallfeatures,theparticipants’ physiologicalresponseswereprimarilyaffectedbythechangeinenvironmentalconditions andfield-of-view.
© 2018 The Authors. Published by Elsevier Inc. ThisisanopenaccessarticleundertheCCBYlicense. (http://creativecommons.org/licenses/by/4.0/)
1. Introduction
Understanding influenceoftheenvironmentalconditionsonhumanperception iscomplex.Variousenvironmental fea-tures,e.g.,soundlevel,temperature,andilluminanceaffectoursenses.Therefore,weadoptedenhancedmeasurement and analysis techniques to define andmeasure what influences citizens in dynamic urban environments. The environmental featuresmeasuredinthisresearchincludesoundlevel,dust,temperature,humidity,illuminanceandthefield-of-viewsince theyinfluenceaperson’ssensethat,inthisresearch,wasrepresentedbythephysiologicalstateofaperson,whichwas mea-suredthroughelectro-dermalactivity(EDA).Withtheadventoftechnology,researchersexploretheutilityofsensor-based
∗ Corresponding author.
E-mail addresses: [email protected] , [email protected] (V.K. Ojha).
https://doi.org/10.1016/j.ins.2018.09.061
0020-0255/© 2018 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license. ( http://creativecommons.org/licenses/by/4.0/ )
physiologicaldatainreal-worldscenarios.Thus,researchersnowhavethemeanstoexplorehowenvironmentalfeaturescan affectindividuals’physiologicalresponse-basedperceptualquality andoverall experience[23].Howto captureanddefine suchaperceptualqualityisanongoingresearchtopicinCognitiveScienceandBehavioralScience[21,36].
Thisresearch presents a controlled study, conductedin Zürich, Switzerland, to acquire data on humans physiological responsesandenvironmentalconditions.Inthestudy,30participantswereaskedtowalkthrough anurban environment, whileequippedwithwearablesensordevices[15].Thestudywasdesignedtoaddressthefollowingresearchquestions: (a) Canwepredictthephysiologicalresponsesofparticipantsbasedonparticularenvironmentalconditions?
(b)Canweinfertherelationshipbetweenthephysiologicalresponsesandtheenvironmentalconditions? (c)Whatarethemostsignificantenvironmentalfeaturesaffectingtheparticipants’physiologicalresponses?
(d) What arethepatternsinthe environmentalconditions,forwhichtheparticipants exhibitarousedandnormal physio-logicalresponses?
Thefeatures of thedata wererecorded through devicesandsensors atvarying frequencies,which hadboth temporal andspatial properties. The features had a temporal property due to continuousrecording, and the features had spatial characteristicsbecauseoftherecording’sassociationwiththechangeinlocations–globalpositioningsystem(GPS).Hence, inthisresearch,we proposed aframework that performssignal preprocessing,signal filtering,signalquantifications, data fusion,anddatalabelingtoanswerthedefinedresearchquestions.
Machine learning based techniques have been successfully applied for knowledge mining and pattern recognition in variousreal-world situations[32,39] since theyare usefulinidentifying theunderlying patternswithin data[1,25]. Thus, we formulated the processed data such that fourstate-of-the-art machine learning techniques,classification, fuzzy rule-based inference,feature selection, and clustering, were applied for discovering patternsin the participants’ physiological responsesrelatedtotheurbanenvironmentalconditions.
Thefirststepinthisresearchwastoassessthepredictabilityofparticipants’perception(physiologicalresponses)ofthe urbanenvironment.Thus,aten-foldcross-validationwasperformedonareducederror-pruningtree(REP-Tree)classification model[29].Following theclassificationapproach, afuzzyrule-basedlearninginferentialmodelwasbuiltusingfuzzy un-orderedruleinductionalgorithm(FURIA)[17]toinvestigatetherelationshipbetweentheurbanenvironmentalfeaturesand thephysiologicalresponse measures.Subsequently, theimportance ofvariousurban environmentalfeatures wasanalyzed byapplyingbackwardlinearfeatureeliminationfilter(BFE)[22].Furthermore,self-organizingmap(SOM)[18]wasapplied tovisualizetheimpactofurbanenvironment featuresonparticipants’physiologicalresponses.Inthefinal step,amethod forreferencingGPS location(geo-location) tocompute meanphysiologicalresponseacross allparticipantswasdeveloped. Sincevarious methodswere involvedindataprocessing, additionalgraphics andmultimedia canbe foundon theproject website[12].
Insummary,thefollowingarethreeessentialcontributionsofthisresearch: (a) afieldstudydesignforunderstandinghumanperceptionoftheurbanenvironment;
(b)aframeworkdesigncomprisingsignalprocessing,signalquantification,anddatafusionmethodsthatinvokesanovelof approachinphysiologicaldataquantification;
(c)acomprehensiveanalysisusingfourmachinelearningmethodstodiscoverthepatternswhicharecrucialtoour under-standingofhumanperceptioninurbansettings.
WeorganizedthispaperintosevenSections.Section2placesthisresearchinthecontextofliteratureanddescribesthe experimental procedure. Section 3 describessignal preprocessing, multi-sensorinformation fusion, andmachine learning techniques in detail. Section 4 is devoted to explaining the obtained results followed by a comprehensive discussion in
Section5.The challengesandopportunity oftheresearcharepresentedinSection6,andSection7concludesthefindings ofthisresearch.
2. Humanperceptionoftheurbanenvironment
2.1. Literaturereview
Theprocessofmeasuringphysiological dataasan indicatorofhumanperceptioniscomplex,particularlyinreal-world applicationsinceperceptioncanbeinfluencedbyvariousfactors[2].However,physiologicalpatternrecognitioncanderive significantevidenceabouthumanperception [27].Similar toourresearch,Picard etal.[27] focusedon physiological sen-sordata,specificallyskinconductance,andtheyrelatedhighandlowarousalsaspositiveandnegativebiologicalreactions. Also,Picardetal.[27]focusedonthecollectionandfilteringofthephysiologicaldatatoconstructgoodqualitydatavoidof failureandcorruptsignals.Theyformulated physiologicaldatasothat ak-nearest-neighborclassifier canpredict human’s physiologicalarousal-basedperception.Krauseetal.[19,20],ontheotherhand,usedwearabledevicedata,including physi-ologybasedsensordata(galvanicskinresponse),toidentifyuser’sstateintermsofphysiologicalandactivitycontextusing SOMbasedclustering. Specifically,they performedunsupervisedlearningto classifysensordatato determinethecontext fromwhichthesignalsweregenerated.
InWang et al.[38],pattern recognitionand classification ofphysiological sensor signals were performedby first de-composing signalsinto its constituentfeatures and by applyingsupport vector machine to classify negativeandpositive
Fig. 1. (a) Study neighborhood marked with the participants’ walking path (Wiedikon, Zürich, Switzerland); (b) a participant with sensor backpack [14] .
emotionlabels.Here,thelabelassociatedwiththesignalswerepredefinedduringtheexperimentbyexposingthe partici-panttonegativeandpositiveenvironmentsduringtherecordingofsignals.Ranietal.[31]performedanempiricalstudyof fourmachinelearningtechniques:k-nearestneighbor,regressiontree,Bayesiannetworkandsupportvectormachineforthe recognitionoftheemotionalstate fromphysiologicalresponsedata.Theyperformedsignalprocessingtoevaluatefeatures fromthephysiologicaldataandlabeledthemwiththeemotionalstatereportedbytheparticipants.
Sinceweinvestigate“causeandeffect” betweentheenvironmentalconditionsandthehuman’sperception,unlikeWang et al. [38] and Rani et al. [31], we performed signal processing on the physiological data to evaluate skin conductance response(SCR)arousals[40].Subsequently,weassignedlabelstosignalfragmentsbasedonthedegreeofarousalwithina specifiedtime.Whiledoingthis,weconsideredphysiologicaldataastheoutputintheclassificationmodelandthesignals fromthe environmentasthe inputs. Whereas, Wangetal.[38] andRani etal. [31]considered features oftheprocessed dataastheinputsandthereportedenvironmentastheoutput.Ourapproach,tofirstdeterminearousallevelwasadopted becauseofthecomplexitiesoftheurbanenvironmentandbecausewecannotaccuratelyconsideranurbanenvironmentto be positive ornegativetowards theperceptualquality ofa participant.Thus,we labeled environmentalconditionsasthe positiveandnegativebyconsideringphysiologicaldataasthetargetintheclassifier’straining.
Ragotetal.[30]foundthatthephysiologicalresponsesignalsfromtheEmpaticaE4wearabledevicewereclosely com-parabletolaboratory-basedmeasurementdevices.Theyalsofoundthatthedatafromsuchwearabledevicescouldbeused totrainasupport-vector-machineclassifiertorecognizetheparticipants’emotionalstate.Similarly,Pohetal.[28]confirmed that EDAdatafromwearable devicesiscomparableto laboratorydevicesandthedata area validphysiologicalmeasure. Hence,wasourapproachinthisstudytoemployEmpaticaE4toperformphysiologicalmeasure.
2.2. Studydesignandmeasurements
Wedesignedastudytounderstandthegeneralpattern(s)ofhumanperception relatedtoeventswhichoccur ina dy-namicurbanenvironment.Aneventindicatesthechangeintheenvironmentalcondition,andalso,asampleofthemeasured environmentaldata.Asacasestudy,weselectedaneighborhoodinZürich,Switzerland(Fig.1(a)),andinvitedparticipants totake a leisurewalkon apredeterminedpath(Fig.1(b)).Theparticipants wereequippedwitha“sensor backpack[14]” andanEmpatica E4wearable device[11].The 1.3kmwalkingpathwascarefullyselected,which coveredadiverseurban scenario[15],e.g.,spaciousandnarrowstreets,greenandurbanareas,andloudandquieterlocations.
Oursensorkit[14]measured thechanges insoundlevel(decibel,dB),theamount ofdust(mg/m3), temperature(°C),
relativehumidity(%),andilluminance(lx).Wealsocalculatedfield-of-viewbasedontheGPSinformationandspatial con-figuration oftheneighborhood.The field-of-view isformerlydescribedastheIsovist descriptor,whichrefers tothe open spaceapersoncan viewfromasinglevantagepoint [4].Since participantswerewalkingina forwarddirection,we con-sidered180° field-of-view witha distance of100 m. Subsequently, the Isovist descriptor foreach participants’ walk was measured by drawinga polygonaround the participants’180° field-of-viewattheir specific GPSlocations. From this, the following measures of the Isovist polygons were calculated: Area–polygon’s surface area; Perimeter–polygon’s perimeter
Table 1
Measured features in the study.
Data Type Features (sensors) Frequency (Hz) Urban environment (indicate the changes
in the urban environmental condition during a participants’ walk)
Spatial (in the context of this study) GPS position (Latitude and Longitude) 1.0
Sound level (dB) 0.4 Dust in air (mg/m 3 ) 0.4 Environmental temperature ( °C) 1.0 Relative humidity (%) 1.0 Illuminance (lx) 1.0 Participants field-of-view (computed
based on GPS position): Area, Perimeter, Compactness, Occlusivity
- -
Human perception (participants’ physiological response)
Spatial-temporal Electro-dermal activity (EDA) 4.0
length;Compactness–theratioofareatotheperimeter(relativetoanidealcircle);andOcclusivity–thelengthofoccluding edges.
TheEDAmeasurestheindividuals’physiologicalstate[6],whichwasrecordedusingEmpaticaE4wearabledevice, sim-ilartostudiesby[11–13].Weplacedthewearable deviceonparticipants’non-dominanthandandletitadjustfor10min accordingtoEmpaticaguidelines[11]. Thedatawere recordedonthe Empaticawebsiteandcorrectedformotionartifact
[11].TheEDAmeasure(physiologicalresponse)wasa time-seriessignalandhastemporal dependencies.Thesensor back-pack,ontheother hand,wasdesignedtocapturethecontextual-basedeventsthatoccurinanurban environment.Inthe contextofthisstudy,an eventisnon-temporalsince aneventisdependent ontheinstance ofitsobservation.Therefore, thecontinuoussignalsrecordedforenvironmentalfeaturesandthecontinuoussignalsrecordedforparticipants’ physiolog-icalresponseswerequantifiedintwodifferentmanners(Section3.2).Moreover,sincetherecordedsignalswereassociated withthe geographicallocation,they alsohadspatialproperties.Theprimary infrastructureoftheurban environmentand season(April2016) wereuniform.However, inherent diversityoccurredfromdifferentexperiment days,time-of-day,and participantsdemographicbackground.Thedataforbothenvironmentmeasuresandcorrespondingparticipants’ physiologi-calresponsemeasuresaresummarizedinTable1.
3. Methodologies
Acomprehensivesignalprocessinganddata-preprocessingframeworkwere proposedinorder toapplyselectmachine learningmethods. Fig.2 illustrates the framework anddescribeshow it wasused forinformationfusion andknowledge miningapproaches.Here,eiandriindicateithquantifiedevent(asampleinthequantifiedenvironmentaldata)andresponse
(asampleinthequantifiedphysiologicalresponsedata)respectively.Thevariablemjfor j∈
{
1,2,...,N}
indicatesthetotalnumberofsamplesbelongingtothejthparticipantpj.Theinformation,therefore,wasfusedinthreestages:
(a) Each participants’event-based data (e) are collectedfromfivesensors, whichwere re-sampled toa unique frequency andsampleswerealignedasperwithontheirtime(Fig.2,mark“A”).
(b)The environment andresponse data fromeach participantwere independently cleaned, filtered,and quantified. Each participants’ quantified eventandresponse datawere fused (paired) byassigning aquantified response ri to eventei
(Fig.2,mark“B”).
(c)Thepairedparticipants’datawerethenstacked(Fig.2,mark“C”).
Thethree-stageinformationfusionapproachproduced thecompileddataset,whichwasfed toselectmachinelearning techniques.Foreachmachine learningtechnique, thecompiled dataset(Fig.2,mark“C”)wasarrangedandconfiguredas perthetechniques’requirementsandobjectives.
3.1. Signalprocessing
3.1.1. Frequencyunification
Theenvironmental features soundanddust were collectedat0.4 Hzfrequency;while GPS position,temperature, hu-midity,andilluminance were collected at1 Hzfrequency (Table 1). Therefore, an up-samplingmechanism witha linear interpolationwasappliedtosoundanddustdata[5]tounifythefrequencies ofthegathereddata.Allfeatureswere then alignedto the same timestamp, whichwas crucialto ensure that all sensorvalues belong to an exact eventduring the study.
3.1.2. Signalfilteringandsmoothing
The physiological response data (EDA signals) were kept attheir original 4Hz frequencyto maintain the information required forarousal detection from the physiological data. With close inspection, we found that some participants EDA
Fig. 2. Information fusion and knowledge mining framework.
signalswereunusableandwerediscarded.Theremaining (accepted)EDAsignalswerefirstsmoothedandthenfilteredto removeartifactsasrecommendedinEDAliterature[6,8].
3.1.2.1. Physiologicaldataselection. TheEDAsignalsfrom30participantswereanalyzedbycomparingthevarious“profiles.”
TheEDAsignalsfromfourtypesofuncorruptEDAprofilesshowninFig.3(a)–(d)wereconsideredforthedataanalysis.The EDAsignalsbelongingtothetwoerroneousEDAprofiletypesillustratedinFig.3(e)and(f)werediscarded.Intotal10EDA signalswerediscarded.TheerroneousEDAsignaltypeswereclassifiedas:
Fig. 3. Signals in (a)–(d) are the most commonly found EDA signal profiles and considered for the analysis. Most commonly found error in signals are shown in (e) and (f).
(a) Type-1error,whenEDAsignalvaluesonlyfluctuatebetweentwovalues,i.e.,theEDAsignalbehavedlikeastepfunction, andthesignalmayalsocontainasignificantamountofsensorloss(nosensorresponserecord).
(b)Type-2error,whenthemajorityofthesamplevalueswerezero(significantsensorresponseloss),despitetheotherwise normalfluctuations(correctsensorresponse)inEDAsignal.
3.1.2.2.Stationarywavelettransformationbased smoothing. After selectingEDAsignals,they were smoothedby undergoing
aStationaryWaveletTransformation(SWT)andreverseSWT.Authorsin[8]suggestedanadaptivemethodforSWT-based smoothingforEDAsignalsrecordedforlongperiods(30h).Inourstudy,EDAsignalswererecordedfor25–29min. There-fore,we applied a one-level SWT and reverse-SWT for smoothing. Each EDA signal was transformed using “Haar” as a motherwavelet in the SWT [24]. A one-level SWT transformation was performedon each signal; and on the obtained waveletcoefficients,athresholdofvalue ± 0.001wasappliedtoeliminatelargerfluctuationinthesignal.Thatis,the val-uesofwaveletcoefficientsabove+0.001 andbelow−0.001were cutoff (Fig.4(a)).Finally,areverseSWTwasappliedto thetransformedsignaltoproduceasmoothedsignal(Fig.4(b)).
3.1.2.3.Truncation oftheunwanted signalfragments. SWTbased treatmentto theEDA signalseliminated thelarge
fluctu-ationsfrom thesignal. However, some sharp drops in signal (corrupt fragment) caused by artifact were not filteredout completely.Thus,the corruptfragmentsandparticipants’waitingtimefragmentsofEDAsignal weretruncatedfromboth original(raw)andsmoothEDAsignals.Fig.4(b)isanexampleofsuch truncation.ThisprocessproducedtwoEDAsignals: original(originalsignalwithfilteringonly)andsmooth(originalsignalwithbothsmoothingandfiltering).
3.2.Signalquantificationandlabeling
Signalquantificationinvolvedthreesteps:time-windowmarking,arousaldetection,anddatalabeling.Infact,theseare thecriticalstepsinthefusion oftheenvironmentaldataandthephysiologicalresponsedata.AsshowninFig.2,atfirst, physiologicaldatawerequantified,andthen,thetimestampinformationwaspassedtotheenvironmentaldataforits quan-tification.
Fig. 4. Stationary Wavelet Transform based smoothing. (a) Wavelet transform of an original EDA signal using Haar wavelet, and smoothing by applying a threshold over wavelet coefficient. (b) Original and smoothed EDA signal with filtering of corrupt and unnecessary fragments.
Fig. 5. (a) Timestamp is indicating Start and End of a participants’ walk during the study. It illustrates the approach to quantify a participant’s physiological response and environmental experience data (b) Timestamp and time-window marking for an EDA signals (physiological response) at every t seconds for the detection of arousal r pj
i for i = 1 to m j .
3.2.1. Timewindowmarking
EachEDAsignal’stimestamp informationwascompared withthetimestampsrecordedatvariousstagesduringa par-ticipants’ walk. Based onthe signalfiltering showninFig.4(b) andavailable timestampinformation,the signal fragment belonged tothe walkingduration—indicated by Start andEnd in Fig.5(a)—were markedwith aregular interval of time-windowsizetseconds.Suchatime-window markingwascrucialtoourdataanalysistoobserveparticipantsphysiological statesinrelationtotheirexperienceoftheeventsoccurringataregularintervaloftseconds(Fig.5(a)).
Therefore,foreachtime-window,eventepj
i fori=1tomjexperiencedbyparticipantpjisavectoroftheenvironmental
featuresandwascomputedbyaveragingthevaluesofsignalfragment(environmentalmeasurement)attheith correspond-ingtime-window.Ontheotherhand,theparticipantsphysiologicalresponserpj
i fori=1tomjuponexperiencingtheevent epj
i wascomputedby an arousal detectionmethoddescribed inSection 3.2.2. Additionally,theparticipants’ field-of-view
(Isovistdescriptors:area, perimeter,occlusivity,andcompactness)were computedatthestartofeachtime-window.Thus, participantquantified data pj hadan identicallyindependent vectorof environmentalconditions(event e
pj
i ) anda
3.2.2. Arousaldetection(EDA)
Thelevelof arousalrpj
i inan EDAsignal dependsonidentifyinga specific signature(pattern) calledskinconductance
response(SCR)orarousal[3,6,9,33,35].ThestateofarousalinanEDAsignalistypicallydefinedasapeakhavingaspecific signature[6].WeprocessedtheEDAsignalsusingaskinconductanceprocessingtoolLedalab[3].Ledalaboffersa continu-ousdecompositionanalysis(CDA)methodforanalyzinganEDAsignal.InCDA,anEDAsignalisdecomposedintotonicskin conductancelevel(SCL)andphasicdriversSCR.
We performedCDA oneach EDAsignal data—of each participant—by using therecommended settings inLedalab [3]. Thatis, thesignal’s optimizationprocedure was performedtwo times,which automatically determined the optimization parametersforevaluatingthenumberofsignificantSCR(nSCR)aboveadefinedthresholdof0.01μSiemenswithin a time-window.WeusednSCR,becausewecouldnot,inatheory-drivenmanner,definewhatstimulus(event)causedachangein participants“physiologicalarousalstate.” Thus,wereliedonadata-drivenapproachbyanalyzingphasicSCR,anon-specific fastchangingEDAmeasure; i.e.,thenumberofpeaksinphasic skinconductanceresponsemeasures nSCRtoanykindof eventforthegiventime-window.Therefore,thenSCRgaveusthemeasuresofrpj
i showninFig.5(b).
3.2.3. Datalabeling
Whenaggregatingallparticipantsdata(Fig.2,mark“C”),weobservethatnSCRvalueforatime-windowvaryfrom0to 12.AnnSCRvalue0indicatethat,inatime-window,aparticipanthadanormalphysiologicalcondition.Ontheotherhand, annSCRvaluegreaterthan0foratime-windowindicatesthataparticipantexperiencedastateofarousalatleastoncein thattime-window.Thus, forthelabeling ofeach time-window—ofeach participantdata—abinary-class labelindicatinga binarystateofphasicnSCRrpj
i canbeused,where
(a) class0is“normal” physiologicalresponse(“N”),i.e.,annSCRvalueequalto0;and (b)class1is“aroused” physiologicalresponse(“A”),i.e.,annSCRvaluegreaterthanto0.
Amulti-classclassification wasalsoused,in whichcase, arousedphysiologicalresponse, “A” hastwo categories:class “LA” indicatinglowarousalresponse,i.e.,0<nSCR<6andclass“HA” indicatinghigharousalresponse,i.e.,nSCR≥ 6.Atotal of6057samplesand9inputfeatureswereavailableinthecompileddatasetforatime-windowsizet(quantificationrate) of5-s.Inthecompileddata,3491samplesbelongedtothecategory“N” and2566samplesbelongedtothecategory“A,” i.e., approximately60%and40%ofthesamplesrespectivelybelongto“N” and“A.” Furthermore, in themulticlass classification, 2079sampleswerelabeled“LA” and487sampleswerelabeled“HA.”
3.3.Machinelearningmethods
3.3.1. Non-inferentialmodeling
Webuildapredictivemodelconsistingoftheenvironmentalfeaturesastheinputs,andbinary(andmulticlass)quantified arousallevel astheoutput usingREP-Tree, whichis adecision treelearner [29]. Ina decisiontree,a tree-likepredictive model is built, where the leaves represent the target (e.g., the class labels: “N” or “A”) and the branches representan observationfora feature(e.g.,soundlevel) ata node.REP-Tree isa methodappliedto reducethe sizeofadecisiontree, whereitkeepspruningsubtreesbyreplacing itwithaleaf(aclass label)aslongastheerrordoesnotincrease (i.e.,the accuracyofthemodeldoesnotdecrease).
WechoseREP-Treetobuildapredictivemodelbecausethealgorithmconstructsadecisiontree,whereeachnodedecides forafeature,anditsspecificvalueproducesaparticularclasslabel.Whilemakingapredictivemodel,REP-Treechoosesthe mostsignificantfeaturesbasedontheircontributiontothemodel’saccuracy,whichisadvantageousforthisproblemsince itisuncertainwhichenvironmentalfeaturesinfluencephysiologicalresponses.Forthevalidationofthemodel’spredictive performance, we choseten-fold cross-validation (10-foldCV). Section 4 describesthetest accuracies of10-fold CV based REP-Treetraining.
3.3.2. Inferentialmodeling
Contrarytonon-inferentialmodeling,inferentialmodelingexplainstherelationshipsbetweentheinputfeaturesandthe outputfeature.Afuzzyrule-basedinference systemiscapableof describinghowindependent environmentalfeaturesare relatedtothe dependentphysiological response(phasicnSCR)feature.Forthis, we applied FURIA,whichisa fuzzy rule-basedclassifier[17].
Unlikeconventional rule-basedclassifiers,FURIAgivesafuzzyrule[17].FURIAproduces fuzzyrules withoperators ≤ , =,and ≥ ;theoperators defineclearconditionsforafeature’s associationwithaclasslabel(e.g.,“N” or“A”).FURIAalso providesarange(e.g.,x→y)indicatingfuzzinessinfeature’scondition,whichmaybeconsideredasasoftboundarywhile associatingafeaturewithaclasslabel[17].Thisabilitywasparticularlyusefulinthisstudysincewewantedtoobservethe specificvalues rangeoftheenvironmental features thatcorrespondedto a participants’state ofarousal.Forinstance,we neededtodetermineforwhichparticularsoundlevelrange,aparticipantexperiencedastateofarousal.SinceFURIAfulfills thisrequirement, itwasselected asthetechniqueforinferential analysis.Interpretationoftheobtainedrulesisdescribed inSection4.
3.3.3. Featureselection
Feature selection is a process to determine the ability ofeach input feature to predict the output. Moreover, feature selection involves making a model using a subset of features and testing its predictive accuracy. We applied backward featureelimination(BFE)methodinthisresearchforitsabilitytoexamineallpossiblecombinationsoffeaturesubsets[22]. BFEstartswithallfeatures ina set(inthiscase, itbeginswith9features)tobuildandtest themodel.Subsequently,BFE iterativelyeliminatesfeaturesone-by-onewhilepropagatinghighaccuracyfeaturesubsetstothenextiteration.Finally,BEF givesalistofsubsetswiththeircorrespondingaccuracies,fromwhichasubsetcanbeselecteddependingontheaccuracy orthe numberoffeatures required.Inaddition toREP-Tree, MLP[16]andSVM [7]were used foramore comprehensive analysisinBFE.Therefore,the featureselection resultwasan assessmentofthreedifferentpredictors. Duringthefeature selection,ateachiteration,BFEused60%randomlyselectedsamplesfortrainingandtherest40%samplestotestthemodel.
3.3.4. Patterndiscovery
Ingeneral,theprimaryaimofself-organizingmap(SOM)istomapm-dimensionaldataontoa2-dimensional(2D)plane. The2DplaneofSOMconsistsofanetworkofneurons(nodes).Thenetwork’snodesacquiretheunderlyingpropertyofthe input data samples (e.g., eventsin the environmental data). Moreover, a SOM projects similar data samples to a cluster center(anodeinaSOM)asperthesimilarity(Euclideandistance)ofthedatasampletothenode[18,37].
SOM is an appropriate choice for thisproblem since it is tedious to define the numberof clusters, especially when problemshavecomplexrelationsbetweenthefeatures.SOMproducedclustersautomatically(seeSection4.4).Additionally, toanalyzepatternrelatedtothegeo-locations,geo-locationsreferencedmeanphysiologicalresponsermeani=
(
rxpi1,yi+rp2 xi,yi+ ...+rpN
xi,yi
)
/N across all participants wascomputedby matchingGPS location information(xi: latitude,yi: longitude) andaggregatingthesamples.Geo-locationreferencedmeanphysiologicalresponsesrmeaniwerecomputedtovisuallyunderstand
patternsinparticipants’physiologicalresponsesrelatedtotheactualmapoftheneighborhood,describedinSection4.4. 4. Results
4.1. Sensitivityanalysis(non-inferentialmodeling)
First,a classifier (REP-TreedescribedinSection 3.3.1)wastrained andtestedonthefive “time-resolved” datasets cor-responding to five quantification rates 25, 20, 15, 10,and 5 s, whose outputs were labeled asthe binary class: normal physiologicalresponse,“N” andarousedphysiologicalresponse,“A.” TheparametersettingsusedtotraintheREP-Tree mod-elsisinTableA.1.TheperformancesofthetrainedREP-Treemodelsareshownonareceiveroperatingcharacteristic(ROC) curveplot[26]inFig.6.
The model’s performance improved as the quantification rates decreased (Fig. 6). The model’s highpredictability for smaller quantification ratesis an indicator of the participants’strong sensitivitytowards the changes in the urban envi-ronment.Themodel’sperformance forsmoothedEDAdata(redsquare)wasbetterthan themodel’sperformance forraw EDAsignal(circles).Thus,thesmooth EDAdatamoreaccuratelydrawtheassociationbetweenachangeinenvironmental featuresandparticipants’physiologicalstatesofarousal.
Theresultsofthe10-foldCVtrainingoftheRET-Treeclassifierforbothbinaryandmulticlassclassificationforthedataset wheresmooth EDAdatawere quantifiedat5-stime-window asshowninTable2.Theclassifier’spredictiveaccuracywas foundtobe87%forthebinary-classclassificationand80%forthemulticlassclassification.
4.2. Sensitivityrangeanalysis(inferentialmodeling)
The non-inferential model indicates that the participants’ physiological responses are sensitive to the environmental changes.Therefore,webuild aninferentialmodeltounderstandhowenvironmentalfeaturesinfluenceparticipants’ physi-ologicalresponses.Afuzzyrule-basedinferentialmodelwasbuiltusingFURIAwhoseparametersettingsarementionedin
TableA.1.We adoptedabinary-class classificationofnSCR,wherenSCRs werecategorizedintotwoclasses: normal phys-iological response, “N” and arousedphysiological response, “A.” TheFURIA algorithm offered an average test accuracyof
Fig. 6. ROC graph of classification models on two categories of datasets represented in two different shapes: square and circles. Square represents dataset prepared with the output feature being the quantified smoothed EDA data; circles represent dataset prepared with the output feature being the quantified original EDA data.
70.23%aftera 10-foldCV training. Suchaccuracy isnotably highforthecomplex problemof understandingthe humans’ perceptionoftheirurbanenvironmentalconditions.
WeanalyzedthesetoffuzzyrulesgeneratedbyFURIAbysegregating therulesbetweentheparticipants’“N” and“A.”
Fig.7isavisualinterpretationoftheobtainedfuzzyrulesforbothclasses“N” and“A.” Weinterpretedandrepresentedthe FURIArulesinFig.7tofindthevalues(rangeofvalues)oftheenvironmentalfeaturesthat
(a) werelinkedtoclass“A,” whichindicatesparticipants’arousedphysiologicalstate; (b)didnotsignificantlyinfluencetheparticipants’arousedphysiologicalstate.
Tovalidatetheknowledgeobtainedfromthevisualinterpretationoffuzzyrules,distributionsoftheenvironmental fea-tureswereexamined throughhistogramsinFig.7(b),(d),(f),(h),(j), and(l). Thevisualinterpretation andsummarization ofthe rules forsoundlevel inFig. 7(a)and its corresponding distributionin Fig.7(b) indicate that theparticipants nor-malphysiologicalresponses matcha particularsoundlevel distribution.Forexample,thesound leveldistributionaround 60dB–66dB (Fig.7(b)) correspondnormalphysiologicalstate (Fig.7(a)). Furthermore,the participantshada tendencyto exhibitarousedphysiologicalstate whenexperiencedsoundlevelabove66dB.Thisresultindicatesthatloudsoundlevels correspondtoincreasedparticipantarousal.
Theresultwassimilarfortemperature,wheretemperaturedegreesgreaterthan21–22°Cwereassociatedwitharoused physiologicalstate (Fig.7(e)). However, itcan be observed that thesamples inthe datasetfortemperatures above 22°C werefewerthanforthetemperaturedegreesbelow22°C(Fig.7(f)),whichwecouldtakeasconfidencethatheatalonedid notcausethephysiologicalarousalofparticipants.In(Fig.7(i)),theparticipantsexhibitedphysiologicalarousalfordarker locations(illuminancelevelbelow580lx).
4.3.Simultaneousimpactofenvironmentalfeatures
Inferencemodelingprovidedthevaluesforenvironmentalfeatures thatwereresponsiblefornormalandaroused phys-iologicalstates.However,itisalsoessential todiscoverwhichoftheenvironmentalfeature(s)havethestrongestinfluence ontheparticipants’physiologicalresponses.Hence,weconstructedabackwardlinearfilterelimination(BFE)basedfeature selectionframeworkandanalyzedtheobtainedresultstobuildasignificancehierarchyoffeaturesubsets(Fig.8).Afeature subset’ssignificancewasestimatedonitsabilitytopredict“N” and“A” classeswithhighaccuracy.
Fig.8isasignificancehierarchytriangleofthefeaturesubsets,whereasubset’spredictabilityreduceswhenthenumber offeatures inthesubset decreases.Three predictorsprovided threefeature selectionresultsets. Fig.8isthe compilation ofthethree resultsets fromallthree predictors.The MLP,REP-Tree, andSVMagreed on thefeaturesubset temperature,
Fig. 7. Visual interpretation of the fuzzy rules. The color “red” indicates the range for which the fuzzy rules finds nSCR > 0, i.e., an indicator of aroused physiological state. The color “blue” indicates the range for which the fuzzy rules finds nSCR = 0, i.e., an indicator of normal physiological state. The color “white” indicates a range of fuzziness. The color “gray” indicates the range for which rules do not provide any conclusive information. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
humidity,illuminance,andIsovistarea,wheretheREP-Treehadthehighestaccuracy,followedbySVMandMLP.Therefore, temperature,humidity,illuminance,andIsovistarea,werenotedasthemostsignificantfeaturesetbutisamatterof trade-off betweenaccuracyandnumberoffeaturesasindicatedinhierarchytriangle(Fig.8).
4.4. Patternsofperceptualvariations
Thepredictivemodelingconfirmedthesensitivityofparticipants’physiologicalresponsestowardsdynamic environmen-talconditions.Thefuzzyrule-basedanalysisdescribedtherelationshipbetweentheenvironmentalfeatures andthe phys-iological response.Featureselectionindicated themostsignificantenvironmentalfeatures. However,patterndiscovery ex-plains:
(a) whichparticipantswereexperiencingasimilarenvironmentalconditionsandwhatweretheirresponse; (b) whethertheparticipants’physiologicalresponsesforcertainenvironmentalconditionsweresimilar; (c) thepatternsoftheenvironmentalfeaturesthatinfluencetheparticipantsphysiologicalarousal.
Thecompiled data(see Fig.2)wereanalyzedusingSOM.Fig.9isaresultofautomatic clusteringfromatrainedSOM, wherethe9-dimensionalinputdataweremappedontothe20× 20dimension2Dplaneconsistingofhexagonalnodes.Each nodeinthemapacquiredthepropertyofasetofsamples.Fig.9(a)showsthemapsoftheenvironmentalfeaturesonfeature
Fig. 8. Hierarchy of feature importance. The symbol I ∗appeared only in the REP-Tree based feature selection. The feature set {T, R, A, I} appear in all three predictor’s results.
Fig. 9. Trained SOM results; node value in the maps are indicated by color: the lowest value is shown in dark blue, and the highest value is shown in bright yellow. (a) U-matrix: SOM clustering map. (b) F-matrix: maps for environmental features, which were linearly scaled with a variance of 1.0 so that they have equal importance in clustering. (c) L-matrix: participant ID and participants physiological response state label (“N” and “A”) map. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
matrices(F-matrices). Ona feature matrix (F-matrix)of an environmental feature (e.g.,sound level), the features’ value assignedtoF- matrixnodesare corresponding tothe nodesonthe SOM’sunified distancematrix(U-matrix) inFig.9(b) andLabelmatrix(L-matrix)inFig.9(c).Hence,thepositionandvalueofthenodesinallthemaps(matrices)inFig.9are comparabletoeachother.Morespecifically,theU-matrixistheresultoftheF-matricesoftheenvironmentalfeatures,and theL-matrixisthe correspondingdominantlabelassociated withthenodes.Therefore,tomake senseofthe pattern,we needtocompareallmatriceswithoneanother.
TheU-matrix inFig. 9(b) showstheclustersofsimilar datapoints. The nodeswithsmalldifferences(in termsof Eu-clideandistance)areshownindarkblue,andthenodeswithhighdifferencesandareshowninbrightyellow.Inaddition, thepatches ofnodeswithsimilar colors,separatedby lighter colors,indicatethe clustersofdatasamples.Moreover, the datasamplescorrespondingtoaclusterintheU-matrixshareacommonality,anddissimilardatasamplesarefurtherapart. Itisthereforeimpliedthattheparticipants’IDlabelbelongingtoaclusterexperiencedsimilarenvironmentalconditions.
Fig.9(c)is anL-matrixwitheach nodewaslabeled withparticipantID andthestateof physiologicalresponse.White nodesindicateanormalphysiologicalresponseandbluenodesindicatethearousedphysiologicalresponse.Bycomparing thesematrices,onecandiscoverrelevantpatternsintheorganizationofthedataset.Thiscould carefullybeinterpretedas
Fig. 10. Geo-location referenced mean physiological responses across all participants. An animation of this graphic indicating real-time simulation is avail- able at [12] .
a “cause” (Fig.9(a)) and“effect” (consultwithFig.9(b)and(c)) ofthedynamicandsimultaneous environmentalfeatures withtheparticipants’physiologicalresponses.
OntheU-matrix(Fig.9(b))abrightyellowpatchseparatesitselffromalltheothernodesclusters.Thisdistinctlyavailable yellowspotistheresultofahighconcentrationofasetsimilarinputsamples,whichinthiscase,isduetotheconcentration highilluminance valuesasevident fromF-matrix forilluminance (Fig.9(a)). Fig.9(c) showsthatat theexact samespot, participants’hadarousedphysiologicalstate(mostofthenodesarecoloredblue)andnodeswerelabeledwithparticipants ID’s(8,13,23,and29)indicatingthatalltheparticipantsexposedtoextremelyhighilluminancealsoexperiencedanequal arousedphysiologicallystate.
Additionally,threeotherclustersofdarkblueexistontheU-MatrixinFig.9(b):oneatthebottom-left,oneatthe top-leftandoneatthetop-rightabovetheyellowpatch.InvestigatingtheF-matricesinFig.9(a),wecanfindthattheclusters atthebottom-leftandthetop-leftinFig.9(b)aretheresultsofhighvaluesofsoundandtemperatureandextremelylow values of illuminance. These clusters, when compared to L-matrix in Fig. 9(c), indicate that the majorityof participants respondedwithanarousedphysiologicalstate.Similarly,theclusteronthetop-rightisduetoacombinationoflowvalues ofdustandtemperature.ThecorrespondingL-matrixinFig.9(c)hasthemajorityofnodesindicatinganormalphysiological state. Further, the F-matrix for Isovist area in Fig. 9(a) showsthat the high value of Isovist area resultedin an aroused physiologicalstate,alsoevidentfromtheL-matrixinFig.9(c).L-matrixalsoindicatesthatparticipantIDs16,23,24,29,32, and35experiencedsuchahighIsovistareaandrespondedwithasimilarphysiologicalstate.
In patternanalysis, the mean physiologicalresponse across all participants wasmapped onto the geographiclocation alongthepath.Thegeo-locationreferencedmeanphysiologicalresponsewascomputedandnormalizedbetween0and1. The geo-locationreferencedphysiological responses highlighted specific locationson theneighborhood’s map where par-ticipantsexperienced arousedphysiologicalstate (Fig. 10).The locations, whereonaverageall participantsexhibited high physiologicalarousal responseare indicatedin redwhile lowphysiological arousalisindicated byyellow. Varyingsizeof dotsonthemapinFig.10isproportionaltothedegreeofparticipants’physiologicalarousal.
5. Discussion
Through this research,we extractedpatterns fromthe datagathered duringa controlled study, wherewe asked par-ticipantsto walk through an urban environment (Section 2.2). Ourdata analysismethods hadthe followingdimensions: signal processing, multi-sensorinformationfusion, andknowledge miningusing machinelearningtechniques.The sensor frequencyunificationandquantificationledtothepreparationofidenticallyindependentdatasamplesofeventsand corre-spondingphysiologicalresponse.Duringthedataprocessingphase,wecategorizedphysiologicalresponsedata(EDAsignals) intocleananderroneoussignals(Section3.1).EDAsignal recordingissusceptibletoartifactsandthesuggesteddefinition
identifiesanerroneousEDAsignal.Finally,thequantificationmethodsegmentedthecontinuoustemporaldataintoregular timeintervalsoft-seconds(time-windowsize)andthequantificationrateof5-swasmostefficient(Section4.1).
Weappliedboth supervisedandunsupervised machinelearningtechniques.ThisincludedtestingREP-Trees’predictive accuracyindeterminingthemodels’sensitivitytowardsfivedifferentquantificationrates:5,10,15,20,and25s.The pre-dictivemodelatt=5shadthehighestaccuracy(Fig.6).The highaccuracyoftheREP-Tree modelindicates itspredictive abilityoftheparticipants’normalandarousedphysiologicalresponsesstateforagivensetofenvironmentalcondition con-sistingofsoundlevel,dust,temperature,humidity,illuminance,andIsovistdescriptors.
The inference modeling,in addition,produced exact valuesof the environmentalfeatures and their influenceon par-ticipantsphysiologicalresponsestate (Section4.2). Also,the environmentfeatures withthelargestrangeof valuesinthe dataset(highestdistribution)weredirectlylinkedtonormalphysiologicalresponses(Fig.7).Inotherwords,theparticipants showeda“habitual effect,” andtheytendtoresponddifferentlytoachangeintheenvironmentalconditionfromthe pre-viousone (Section 4.2). Suchageneralization offuzzyrulesacrossall participantsislimitedbecauseoftheavailabilityof fewerdatasamplesandthevariationsincities’architectures.However,itisnecessarytomentionthatallparticipants en-gagedinthestudyondifferentdaysanddifferenttime-of-dayandweobservedahighaccuracyinthemodel’spredictability despitebeingappliedtosuchacomplexanddiversedataset.Forexample,afuzzyrule(Fig.7)indicatesthataparticipants’ arousallevelscorrespondtoextremelylowilluminance,orhightemperature, oralarge Isovistarea.Specifically,changein physiologicalarousalwasobservedforasmalltoalargeIsovistarea,i.e.,anentrytoacrossroadandpassingfromanarrow toawiderstreet(Fig.10).
Itwasdifficulttoidentifythefeatureswiththehighestinfluenceonthephysiologicalresponsefromtheinference mod-eling.Therefore,a backward feature eliminationmethodwiththree predictors(Section 3.3.3) helpeddetermine themost significantenvironmentalfeature(s)andispresentedinasignificancehierarchytriangle(Fig.8).Thefeatureselection pro-cess,however,haditstrade-off;whenreducingthenumberoffeaturesfromthefeatureset,italsodecreasestheaccuracy ofthepredictors.Aftera thoroughinspectionofFig.8,thepredictorssuggest thetemperature, humidity,illuminanceand theIsovistareaasthemostsignificantfeaturessetcomparedtoasetofanyotherfeaturescombination(Section4.3).
SOMwasemployedforautomaticclusteringtodiscoverpatternsinthedataset(Section4.4).Theparticipantswith sim-ilarenvironmentalconditionswereexpectedtohavea similarperception(physiologicalarousalstate)andexpectedtofall intothesameclusterornodeonthemap.Forexample,aclusterformedduetoextremelyhighilluminanceandanotherfor lowilluminanceconditions(Fig.9).Thisindicatesthataparticularenvironmentalconditioninfluencesmostofthe partici-pantsequallyandthemajorityofparticipantsrespondedtoasimilarphysiologicalresponsestatewhenexperiencingsimilar conditions.Furthermore,becausetheparticipantswalkedatdifferentspeeds,thenumberofquantifiedevents correspond-ingtoeach participantslightlyvaried.Therefore,thegeo-locationreferencednormalizedmeanoftheeventswasthebest methodtoshowthegeolocationoftheparticipants’averagephysiologicalresponsesonthemap(Fig.10).Thismapcanbe usedtovisuallyinspecttheimpactofurbanfeatures,suchasstreet-width,street-type,traffic,typeofarea(residentialand industrial)andtheirpotentialimpactontheparticipants’physiologicalresponse.
6. Challengesandopportunities
Themethods developedforthisinvestigationhelp revealpatternsfromcomplexhuman-environment interactions.The analysispredominantlyfocusedonimprovedquantificationmethodsforphysiologicalarousalleveldetectionandameansto correlatearousallevelwithenvironmentalstimuli.Thisapproachallowsustoobserveanincreaseinphysiologicalarousalin responsetospecific environmentalconditions(Section5).Theprimary challengeofthisstudywastheprocessofselecting theappropriate tuning parameters toquantify andevaluate the arousal label. Forexample, the accuracyof the methods (Fig.6)varieddependinguponthequantificationrate.Similarly,theaccuracyofthemethoddependsontheprocedureand thresholdadoptedforthenSCRslevel detection[6].Moreover,we captured9features ofareal-worlddynamicssituation. Hence,anincreasednumberoffeaturesmayfurtherimprovethepredictivemodel’saccuracy.
Futurestudies can utilize the presented experimental design and quantification methodology. For instance, it can be extended tocapture citizen’spublic transport commutingexperience (physiological response while walking,waiting, and riding),andfortrafficsafety,themethodcanbepotentially appliedtounderstandthephysiologicalarousalpatternof ve-hicleriderswhiletheyridethroughcities[10,34].Moreover,thedevelopedpredictivemodelcanbeusedtoextrapolatethe potentialcitizen’sarousallevelstoalargergeographicareawhencombinedwiththeisovistvaluesandmeasured environ-mentaldatabeyondtheselectedpath.
Inthis research, we recognized factors influencinghumans perception. Whereasto meet the refereedchallenges, our findingssuggest thatfurtheremployingvirtual realityset-up couldhelpreducing noisethat maybe inducedbyunknown factors.Additionally,ourfindingssuggestthatasubjectivethresholdingskinconductancecanalsobeemployedtomitigate thechallenges.
Moreover,inthefieldofurbanstudies,itiscrucialtounderstandhowthebuiltenvironmentinfluenceshumanbehavior andperception.Thisquestionhasbeencentral tothepractice andresearch eversince andposes afundamental method-ologicalproblemsinceitisespeciallydifficulttoa)objectivelymeasureperceptionandb)dealwiththemultitudedynamic environmental factors preventing to identify the effect of pure urban form on human perception. As an answer to this problem,thisresearch providesamajorcontributionbypresentingandempiricallytestinganovelresearchframeworkfor predictingandinferringtheeffectsofplanningdecisionsonhumanperception.Inessence,theframeworkprovidesinsides
patternsinthedataset:Thehighaccuracyofthenon-inferentialpredictivemodelwasevidenceoftheparticipants’ physio-logicalstatesensitivetothechangesinenvironmentalconditions.Thefuzzyrule-basedinferentialmodelingresultsindicate thattheoccurrenceof“normal” and“aroused” physiologicalconditionscorrespondstospecificvalues(andrangeofvalues) foreachenvironmentfeature.Itsuggestedthatthechangesintheparticipantphysiologicalarousalstateprimarilyoccurred duetothefluctuationsintheenvironmentalconditions.Featureselectionshowedthatsomeenvironmentalfeatures,suchas temperature,humidity,illuminance,andthe-filed-of-viewweremoredominantintheirinfluenceonparticipants’ physiolog-icalresponsethansoundlevelanddust.Patternanalysisfromself-organizingmapindicatedthat,primarily,theparticipants whoexperience similarenvironmentalconditionsresponded insimilar physiologicalarousalstate. Finally,thegeo-location referencingofaveragephysiologicalresponseacrossallparticipantsproducedameanstovisually inspecthowparticipants respondduringthe actual walk inrelation topermanent urban features. The proposed dataanalysisframework revealed patternsfromthecomplexspatial-temporalenvironmentalandphysiologicaldatathatimpact ourunderstanding ofurban settings.
Acknowledgments
Thisresearchwasfundedby SwissNationalScienceFoundation(SchweizerischerNationalfondszurFȵrderungder Wis-senschaftlichenForschung)projectno.100013L_149552titledundertheGermanResearchFoundation(Deutsche Forschungs-gemeinschaft)ResearchGrantno.DO551/21-01.Authorsarethankfultoalltheparticipantswhotookpartinthestudy. AppendixA
Table A.1
Parameter settings of the machine learning techniques.
Algorithm/Tool Parameter Definition/Purpose Value
Ledalab Analysis type Type of method for decomposing a signal Continuous decomposition Optimization time Number of times a signal is optimized 2
Window range 1 3.7 s
Smooth method Gaussian Smoothing wind 0.2 s REP-Tree #Leaf instances Minimum children per node. 2
Depth Maximum limit of tree depth/level. No limit Pruning Pruning of tree nodes. True FURIA Function Membership function for fuzzification Trapezoidal MLP Learning rate Convergence speed. 0.3
Momentum rate Influence of previous iteration. 0.2 Hidden Layer Maximum hidden layer nodes. 10 Iterations Maximum time for parameter optimization 10 0 0
LibSVM SVM kernel Type of function at a hidden node Radial basis function Epsilon Termination criteria for algorithm 0.001
SOM Map dimension Dimension of the 2D plane 20 × 20 Normalization Method of data normalized for SOM training Linear scaling Training mode Number of samples in an epoch of training. Batch Iteration Number of training epochs 25 Fine-tuning Number of fine-tuning epochs 20
References
[1] E. Alpaydin , Introduction to Machine Learning, MIT Press, Cambridge MA, 2014 .
[3] M. Benedek , C. Kaernbach , A continuous measure of phasic electrodermal activity, J. Neurosci. Method 190 (1) (2010) 80–91 .
[4] M.L. Benedikt , To take hold of space: isovists and isovist fields, Environ. Plann. B 6 (1) (1979) 47–65 .
[5] T. Blu , P. Thévenaz , M. Unser , Linear interpolation revitalized, IEEE Trans. Image Process. 13 (5) (2004) 710–719 .
[6] J.J. Braithwaite , D.G. Watson , R. Jones , M. Rowe , A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psycho- logical experiments, Psychophysiology 49 (2013) 1017–1034 .
[7] C.C. Chang , C.J. Lin , LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol. 2 (3) (2011) 27:1–27:27 .
[8] W. Chen , N. Jaques , S. Taylor , A. Sano , S. Fedor , R.W. Picard ,Wavelet-based motion artifact removal for electrodermal activity, in: 37th Annual Interna- tional Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2015, IEEE, 2015, pp. 6223–6226 .
[9] J. Choi , B. Ahmed , R. Gutierrez-Osuna , Development and evaluation of an ambulatory stress monitor based on wearable sensors, IEEE Trans. Inf. Technol. Biomed. 16 (2) (2012) 279–286 .
[10] C. Collet , C. Petit , S. Champely , A. Dittmar , Assessing workload through physiological measurements in bus drivers using an automated system during docking, Hum. Factors 45 (4) (2003) 539–548 .
[11] I. Empatica, 2018, https://www.empatica.com (Accessed on: 21.01.2018).
[12] ESUM, Human perception of the urban environment, 2018, https://www.esum.arch.ethz.ch , January.
[13] M. Garbarino , M. Lai , D. Bender , R.W. Picard , S. Tognetti , Empatica e3—a wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition, in: EAI 4th International Conference on Wireless Mobile Communication and Healthcare, IEEE, 2014, pp. 39–42 .
[14] D. Griego, S. Kuliga, M. Bielik, M. Standfest, V.K. Ojha, S. Schneider, R. König, D. Donath, G. Schmitt, C. Miltiadis, A. Forino, ESUM urban sensing handbook: component, assembly and operational guide: sensor backpack and 360 ° videos. zürich, ETH zürich, 2017a,
[15] D. Griego , V. Buff, E. Hayoz , I. Moise , E. Pournaras , Sensing and mining urban qualities in smart cities, in: IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), IEEE, 2017, pp. 1004–1011 .
[16] M.T. Hagan , M.B. Menhaj , Training feedforward networks with the marquardt algorithm, IEEE Trans. Neural Networks 5 (6) (1994) 989–993 .
[17] J. Hühn , E. Hüllermeier , FURIA: an algorithm for unordered fuzzy rule induction, Data Min. Knowl. Discov. 19 (3) (2009) 293–319 .
[18] T. Kohonen , The self-organizing map, Proc. IEEE 78 (9) (1990) 1464–1480 .
[19] A. Krause , D.P. Siewiorek , A. Smailagic , J. Farringdon , Unsupervised, dynamic identification of physiological and activity context in wearable computing, in: Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings, 2003, pp. 88–97 . October.
[20] A . Krause , A . Smailagic , D.P. Siewiorek , Context-aware mobile computing: learning context-dependent personal preferences from a wearable sensor array, IEEE Trans. Mob. Comput. 5 (2) (2006) 113–127 .
[21] S.F. Kuliga , M. Standfest , M. Bielik , S. Schneider , R. Koenig , D. Donath , G. Schmitt , From real to virtual and back a multi-method approach for inves- tigating the impact of urban morphology on human spatial experiences, in: The Virtual and the Real in Planning and Urban Design: Perspectives, Practices and Applications, Routledge, Taylor & Francis, 2017, pp. 151–169 .
[22] S. Maldonado , R. Weber , F. Famili , Feature selection for high-dimensional class-imbalanced data sets using support vector machines, Inf. Sci. 286 (2014) 228–246 .
[23] P. Mavros , M.Z. Austwick , A.H. Smith , Geo-EEG: towards the use of EEG in the study of urban behaviour, Appl. Spat. Anal. Policy 9 (2) (2016) 191–212 .
[24] B. Molavi , G.A. Dumont , Wavelet-based motion artifact removal for functional near-infrared spectroscopy, Physiol. Meas. 33 (2) (2012) 259–270 .
[25] V.K. Ojha , A. Abraham , V. Snášel , Metaheuristic design of feedforward neural networks: a review of two decades of research, Eng. Appl. Artif. Intell. 60 (2017) 97–116 .
[26] J. Pearce , S. Ferrier , Evaluating the predictive performance of habitat models developed using logistic regression, Ecol. Modell. 133 (3) (20 0 0) 225–245 .
[27] R.W. Picard , E. Vyzas , J. Healey , Toward machine emotional intelligence: analysis of affective physiological state, IEEE Trans. Pattern Anal. Mach. Intell. 23 (10) (2001) 1175–1191 .
[28] M.Z. Poh , N.C. Swenson , R.W. Picard , A wearable sensor for unobtrusive, long-term assessment of electrodermal activity, IEEE Trans. Biomed. Eng. 57 (5) (2010) 1243–1252 .
[29] J.R. Quinlan , Simplifying decision trees, Int. J. Man. Mach. Stud. 27 (3) (1987) 221–234 .
[30] M. Ragot , N. Martin , S. Em , N. Pallamin , J.M. Diverrez , Emotion recognition using physiological signals: laboratory vs. wearable sensors, in: International Conference on Applied Human Factors and Ergonomics, Springer, Cham, 2017, pp. 15–22 .
[31] P. Rani , C. Liu , N. Sarkar , E. Vanman , An empirical study of machine learning techniques for affect recognition in humanrobot interaction, Pattern Anal. Appl. 9 (1) (2006) 58–69 .
[32] I. Rodríguez-Fdez , M. Mucientes , A. Bugarn , FRULER: fuzzy rule learning through evolution for regression, Inf. Sci. 354 (2016) 1–18 .
[33] W.T. Roth , M.E. Dawson , D.L. Filion , Publication recommendations for electrodermal measurements, Psychophysiology 49 (8) (2012) 1017–1034 .
[34] W. Shuyun , Z. Jinxi , L. Xiang , C. Weilin , Traffic safety evaluation of asphalt pavement anti-sliding property based on time series analysis of drivers’ electrodermal activity, in: 2nd International Conference on Intelligent Computation Technology and Automation, ICICTA’09, 2009 (Vol. 3, IEEE, 2009, pp. 859–863 .
[35] S. Taylor , N. Jaques , W. Chen , S. Fedor , A. Sano , R. Picard , Automatic identification of artifacts in electrodermal activity data, in: 37th Annual Interna- tional Conference of the IEE Engineering in Medicine and Biology Society (EMBC), 2015„ IEEE, 2015, pp. 1934–1937 .
[36] F.J. Varela , E. Thompson , E. Rosch , The Embodied Mind: Cognitive Science and Human Experience, MIT Press, Cambridge MA, 2017 .
[37] J. Vesanto , J. Himberg , E. Alhoniemi , J. Parhankangas , Self-organizing map in MATLAB: the SOM toolbox, in: Proceedings of the MATLAB DSP Confer- ence, 1999, pp. 16–17 .
[38] X.W. Wang , D. Nie , B.L. Lu , Emotional state classification from EEG data using machine learning approach, Neurocomputing 129 (2014) 94–106 .
[39] I.H. Witten , E. Frank , M.A. Hall , C.J. Pal , Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, 2016 .
[40] V. Xia , N. Jaques , S. Taylor , S. Fedor , R. Picard , Active learning for electrodermal activity classification, in: IEEE Signal Processing in Medicine and Biology Symposium (SPMB), IEEE, 2015, pp. 1–6 .
