Audio technology and mobile human computer interaction: from space and place, to social media, music, composition and creation

DOI: 10.4018/IJMHCI.2017100103



Copyright©2017,IGIGlobal.CopyingordistributinginprintorelectronicformswithoutwrittenpermissionofIGIGlobalisprohibited. 

Audio Technology and Mobile

Human Computer Interaction:

From Space and Place, to Social Media,

Music, Composition and Creation

Alan Chamberlain, School of Computer Science, University of Nottingham, Nottingham, UK Mads Bødker, Department of IT Management, Copenhagen Business School, Copenhagen, Denmark Adrian Hazzard, School of Computer Science, University of Nottingham, Nottingham, UK David McGookin, Department of Computer Science, Aalto University, Helsinki, Finland David De Roure, Oxford e-Research Centre, University of Oxford, Oxford, UK Pip Willcox, Bodleian Libraries, University of Oxford, Oxford, UK

Konstantinos Papangelis, Computer Science and Software Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, China

ABSTRACT

Audio-basedmobiletechnologyisopeninguparangeofnewinteractivepossibilities.Thispaper bringssomeofthosepossibilitiestolightbyofferingarangeofperspectivesbasedinthisarea.Itisnot onlythetechnicalsystemsthataredeveloping,butnovelapproachestothedesignandunderstanding ofaudio-basedmobilesystemsareevolvingtooffernewperspectivesoninteractionanddesignand supportsuchsystemstobeappliedinareas,suchasthehumanities.

KeywoRdS

Design, Ethnography, HCI, Humanities, Interaction, Location, Mobile

1. INTRodUCTIoN

Thisarticlerepresentsaseriesofresearchprojectsandapproaches,whichexpanduponaninitial workshop,“AudioinPlace”thattookplacein2016(Chamberlainetal.,2016).Thekeythemeofthe workshopwasaudioanditsrelationshiptomobileHCI(Human-ComputerInteraction).Theapproach oftheworkshopwasnottorestrictsubmissionstooneparticulararea,buttoencouragepapersfrom amultiplicityofdomains.Theresearchpresentedinthispaperaimstoextendourunderstandingsof audio-basedmobileHCI,provideaplatformfordiscussionanddebate,extendingourunderstandings ofmobileHCIandencouragingapproachestounderstandingtechnologythatareattheforefrontof MobileHCI.

technologies,systemssuchastheArduinoandBeagleBone,arebuttwoofthemanysmallform-factorhardwaresystemsavailable.Oftenthesesystemsareadaptable,areeasilyrepurposedandcan beusedinconjunctionwithotherexistingtechnologiesmakingthemanattractivepropositionfor peoplewantingtodevelopportableexploratoryaudio-basedsystems.Forafullerexpositionsee McPhersonetal.(2016)andMorealeetal.(2017).

However,thetechnologyisonlyonepartofthestory,beingabletotaketechnologyoutinto theworldopensupaplethoraofinteractivepossibilities,differentwaystounderstandandinteract withone’ssurroundings,newwaystointeractwitheachotherandthechancetodevelopdifferent approachestounderstandsuchinteractions.Ofcourse,theworkinthisarticleprimarilyfocusesupon audiointeraction,butstill,indoingithasenabledpeopleofferawiderangeofperspectivesthatrange fromautoethnographicmethods(Eslambolchilaretal.,2016)formobilesystemdesign,throughto location-basedinteraction,anddesignapproachesthatusethepastaswaytodesign,re-imagineand developtechnologiestoday(DeRoureetal.,2016a).

2. ReCoRdING SoUNdS, MAKING MeANINGS

In‘AcousticTerritories’(2010),LaBellemakesacaseforunderstandingsoundsaspartofa“charged spatiality”(xviii);asocialandemotionaltopologyimplicatedintheemergenceofasense(-and politics)ofplace.Sounds,LaBellesuggests,areneverprivate,butmustbeunderstoodasarelational forcethatsomehowweavesaspacetogetherwhileatthesametime‘unhinging’itinunforeseenways byassociations,formingpersonalmemoriesandemotionsbutalsonetworksofsocialconnections, culturalmeaningsandsoon.Inthissection,wesuggestthatinordertoimagineandbuildmobile interactionsaroundlocalizedsound,weneedmethodstoaddresstheopportunitiesinthisfieldand thatareattentivetothepossibleemotionalandsubjectivevaluesinscribedintheconsumptionas wellasproductionoflocalizedsound.

Assuchitsuggeststhatpracticesofsoundrecordingonmobiledevices(aswellastheensuing useofrecordedsoundsforthepurposesofdistributing,sharingormapping)offersrelativelyuntapped modesofengagement.Particularlyasbothurbanandruralarchitectures,infrastructuresandmobile digitalmediasarebecomingintertwinedtoformmeaningandexperiences(underthegeneralmotifs ofubiquitouscomputing,InternetofThings,orambientintelligence),weseeaneedformethodsthat canhelpinexploringsomeofthepossiblewaysinwhichtangibleplacesand‘intangible’content suchassoundcaninteract.Beyond(orarguablybefore)anytechnicalchallengesassociatedwith buildingaccessibleandeasytousesystemstoallowpeopletorecordandsharesoundsusingamobile device,comesthequestionwhatvaluessuchsystemsmightfulfilorwhatpurposestheymightserve.

Wesuggestthatautoethnographiescanplayapartinanefforttoconnectsoundrecording tomobileHCI.Specifically,weexplorepossibleconnectionsbetweendigitalmedia,spaceand ‘meaningmaking’,suggestinghowautoethnographies(BødkerandChamberlain,2016)mighthelp indiscoveringandarticulatingdesignopportunitiesinthemergingofmobiledigitaldevicesand meaningfulplacemakingpractices.

3. AUToeTHNoGRAPHy

Usinganauto-ethnographicallyinspireddesignapproachtolocatemediagoesbeyondtheconventional “implicationsfordesign”(Button,2000),ismorethana“scenic”study(Dourish,2006),oran ‘imposed’analytic.Wesuggestthatitcanbeusedasmethodofimagination,asalensthroughwhich wecandiscoversocialaswellaspersonalpotentialsindesign.Itcanbeassociatedwithaself-design agenda…thatfocusesonthepotentialofself-experienceandarticulationforthedesignerswork.

narrativesthatfavouremotionalandaffectiveencounterswithourselvesengagedintheworld.As amethod,wefindthatwritingnarrativesinthestyleofanautoethnographyarecapableofteasing outsignificantandemotionallypoignantstoriesandlooseimplicationsthatareotherwisedifficult torecount.Theyarewaysofseeingourselvesengagedinaculture,andassuchtheycanbeways ofrecognizing(andrendering)experiencesthathaveparticularemotionalresonancesorstrong, embodiedandaffectiveimpactsthattraditional‘analytical’ethnographies(orindeedothermethods) mayneitherbereceptivetonorhaveaccessto.

Considerforinstancehowaworkshopparticipant(M)recountsaparticularsoundthattriggers asurgeofnostalgiaandvisceralmemory:

In my imagination the sounds affix themselves more readily to a broader childhood landscape of flat, overcast marshlands. Maybe driving through it, sitting in the backseat of my parents’ car, or to the repetitiveness of the wet landscape?

Hewishesthatthesound:

… could seep out of the ground there too, that other people might share in my experience of the short musical sequence in what has, for me, become the ‘proper’ setting. Perhaps the music is also meaningful to other people?

Theimaginedconnectionofasoundwithaplaceisarguablyprivate,butexperiencingsuch connectionsbetweensoundandplacecanbepositivelycommunal,anotiondeliberatedinTruax’work onwhathecalls‘acousticcommunities’(Truax,2001).Communalsounds“areusuallyacoustically richandmayevenhavemusicalvalue,andthereforetheyacquiretheirsignificanceinthesoundscape throughtheirabilitytomakeastrongimprintonthemind,animprintthatembodiestheentirecontext ofthecommunity”.M’srenderingoftheemotionaleffectofasoundseemsprivateatfirst,butlater pointstothepossibilityofsoundsaspartofasharedprocessofmakingmeaning.

Anothernarrativesuggestshowanotherworkshopparticipant,wantedtoshareasignificant sonicexperiencewithotherpeople:

I know it’s odd but I’d like to share those kinds of things about the place that I live now. Just leave an audio trail, a story or a thought that people might come across in the ether. There’s a Celtic hill fort next to the town that we walk over and I’d like to leave things for other people that I know to respond to, I have plans to use the defenses of the fort, they are layered like giant steps designed by some ancient architect.

Soundsemplacedandlocalizableinthelandscape,asimaginedintheaboveexample,mightcome toworkastracesofanemergentheritagelandscape,partofanongoingengagementinremembering ormakingsenseofplace.

Anumberoftoolsofvaryingcomplexityexisttodriverapidexperimentationwiththeconcepts thatemergefromautoethnographies.Usinglittle-bits(www.littlebits.cc),smallwearableandmobile prototypesthatincludesoundandavarietyofwirelesssensorscanbebuilt.RaspberryPimini-computers(https://www.raspberrypi.org)canbeusedtobuildsoundstreamingdevices(e.g.those oftenusedforthelocusonussoundmap,seehttp://locusonus.org/soundmap/051/).Recho(http://www. recho.org),acommercialiPhoneapp,allowsuserstoleavebehindshortsoundstiedtoalocation. PercussaAudioCubes(https://www.percussa.com/what-are-audiocubes/)thatallowausertotangibly interactwithdifferentsounds.Wehaveusedaudiocubesinworkshopstofacilitatediscussionsabout sound,tangibleinteractionandmapping.

Further,wemightsupplementthestandardautoethnographicmedium(writing)withmorefocused renderingsoftheexperienceoffieldrecordingorconsuminglocation-basedsoundsonmobiledevices. Thismaypointtofurtheralternativewaysinwhichusersofmobiledevicescan‘tunein’toplaces andgeneratenewcollectiveandemergentmeaningsinplace.

4. MUSICAL LANdSCAPeS: FoRMING ReLATIoNSHIPS BeTweeN THe PHySICAL ANd dIGITAL

‘Locativemedia’originatedinanefforttodistinguishtheemergingbodyofcreativeworkundertaken withnewmobiletechnologiesfromthecommercialimpetusoflocation-basedservices(Tuters 2004).Therearenumerousexamplesoflocativemediaexperiencesthatexplorearangeofdigital interactions,suchas:augmentedreality,wheredigitalmediaisoverlaid.Wenowturnourattention tolocativemedia,withaspecificfocusonlocativemusic.Thetermontotherealworld(Izadiet al.2002;Vlahakisetal.2002;ZimmermannandLorenz2005;Gildersleeve2015);mediascapes,a termcoinedbyHPLabstodescribesitespecificlocativeexperiences,asauthoredbytheirMscape tool(Behrendt2005;Reidetal.2005);context-aware,oftenusedtodescribeworkswherearangeof contextualdataisusedotherthanpurelypositioningtechnologies(Yuetal.2008;ElliottandTomlinson 2006;Abowdetal.1996);mixed-reality,todescribethecombinationofreal-worldactivitycombined withonlineactivity(BenfordandGiannachi2011;Crabtreeetal.2004;Galani2003;Flinthametal. 2003)andspecificallyinrelationtolocation-basedgames,thetermpervasivegames(Benfordetal. 2003;Chamberlainetal.2010;Waern,Montola,andStenros2009).

‘Mapping’constitutesthecentralfunctionofmostlocativemediaexperiences,“makingdata geographicallyspecificorplacingadigitalobjectinspace”(Hemment2006,350).Thelatterhalfof thisquoteacknowledgesFraukeBrehendt’splacedsoundexperiences–aclassfromhertaxonomyof mobilemusic(Behrendt2010).Placedsoundsdescribethetriggeringofdigitalaudioatpredefined locales,typicallyrealisedviaGPSpositioningandGPSenabledmobiledevices.Earlyadoptersof thesetechnologicaldevelopments(i.e.Rueb2002;Kirisitsetal.2008;Tanaka2008;Blistein2015; Collier2015;Gayeetal.2006;Bryan-Kinnsetal.2009;Reidetal.2005;Chamberlainetal.2010; Benfordetal.2003)were“askingwhatcanbeexperiencednowthatcouldnotbeexperienced before…anewunderstandingoftherelationbetweenphysicalanddigital”(Hemment2006).These practitionersnotedthatplacingnew,novelaudioexperiencesintospacesfunctionedasaproducerof contentandalsoastageuponwhichthecontentisdelivered(Tanaka2008).

Inthissection,weplaceaspecificfocusonformingrelationshipsbetweenphysicalspaceand theconstructionofdigitalmusicalsoundtracks;proposingthatthephysicalassemblyofachosen locationwithallisnaturallyoccurringandbuiltartefactscanserveasastructuralblueprintfrom whichamusiccompositioncanbecreatedandplaced.Wedrawuponakeyexampletoillustratethis approach,aspresentedinthefollowing.

openparkland.Sculptureexhibitsaredistributedthroughouttheparkandvisitorsarefreetoroam, exploreandinteractwiththemintheirowntime.ThissoundtrackwasaGPSdrivenexperienceusing smartphonesandheadphones,whereasoundtrackwascomposedandpresentedviaalargenumber ofmusicalcellsandlayersthatcouldbetriggeredbyGPSasvisitorsexploredtheparktocreateone singlecontinuousmusicalarrangement.Originalmusicwasusedthroughout,composedspecifically fortheexperience.Theintentionofthesoundtrackwassimilartothatofthefilmsoundtrack,whose roleistoprovideasupportiveaccompaniment,toreflectthenarrativeratherthandominateit.

Inordertocomposemusicforsuchalargelocation,withindeterminatedurationandnon-linear arrangement(i.e.dependentontherouteandwalkingpaceofeachvisitor)acoherentmethodto structurethemusicwasnecessitated.Toachievethiswesoughttoestablishrelationshipsbetweenthe physicalattributesoftheparkandkeymusicalandfeaturesinthesoundtrack.Scrutinizingthepark revealedthefollowingkeyattributes:(1)sculptureexhibitspacedinspecificlocales;(2)apatchwork ofphysicalregionsdemarcatedbytransitionsinlandscaping(e.g.formalgardenstoexpansesof grassedareastorollingfieldstoconstrainedpaths);(3)theplacementofwalkways,hedgerowsand trees;and(4)finallybuildings.Thisthreedimensionalassemblyofinterconnectedphysicalattributes wasusedtodirectlystructurethesoundtracksmusicalfeatures,atahighandatalowlevel.

First,wedescribethehigh-levelapproach,termedmusicallandscapes.Oneofthemainattributes oftheparkwasthispatchworkofphysicalregionsmadedistinctbydifferenttypesoflandscaping andattributesthatboundedthem.Inresponseoursoundtrackwasformedfromaseriesofmusical sections,ormovements,overlaiddirectlyontothispatchworkstructure.Thus,adistinctmusicalsection washeardwithintheboundsofeachphysicalregion.Figure1marksthecollectionofregions,as illustratedbydifferentblocksofcolourthatspannedthepark.ThewhiteareasinFigure1represent thelocationofsculptureexhibits.Inpractice,theboundarylinesoftheseregionsactedasGPStrigger zones,soavisitor’stransitionacrossboundarylineisaccompaniedbyatransitionfromthecurrently playingmusicalsectiontothatnewregion’smusicalsection.Therationaleforthesizeandshapeof theseregionsalongsidetheplacementofneighbouringboundarylinescanbedescribedinfourways:

• Constrained boundaries:Drawninreferencetofixedphysicalstructuresthatimpedetheuser access,i.e.,wall,fences,hedgerows,buildings,areathatausercannotcross;

• Implied boundaries:Drawninreferencetophysicalattributessuchasstructuresorobjectsthat canbetraversed,butimplyatransition,i.e.,gates,changesoflandscapesuchasformalgardens intoruralparklandorfromgrassedareasontoconcretewalkway;

• Negotiated boundaries:Boundariesarenotdefinedbyanyphysicalstructure,butratherin response,ornegotiationtoanotheradjacentnegotiatedboundary;

• Unconstrained boundaries:Notdrawntoanyphysicalattributeorinrelationtoanyotherboundary.

Themajorityoftheregionsareformedby‘implied’boundarytypes,thusdefinedinresponse toexistingphysicalattributes.

Fromamusicalperspective,asetofcompositionaltechniqueswerethenusedtorendereach musicalsectiondistinct,especiallythosewithneighbouringregions.Eachsectionwascomposedina differentbutcomplementarykeycentre(seeFigure2),newthematicideaswereemployedwithineach region,alongsidedistinctmusicaltextures(i.e.combinationofinstrumentsounds).Thismeantthatas visitorswalkedoverthesephysicaltransitionstheyweresimilarlyaccompaniedbymusicaltransitions.

Withmusicallandscapesdepictingourhighlevelmusicalstructure–transitionsbetweensections –wethenconsideredalowerlevelofmusicalinteractionthattakesplacewithintheregionsthemselves. Specifically,howtoaccompanywithmusicavisitor’strajectoryintoandoutofinteractionwith sculptureexhibits.Wetermedthisprocess,musicaltrajectories.WedrewinspirationfromFoshet al.,(Foshetal.,2013)who’s‘trajectoryofinteraction’urgesdesignersofsuchvisitorexperiencesto contemplatefourkeyphasesofinteractionwithartefactsorpointsofinterest,namelyapproach,arrive, engageanddepart.Inreplytothis,wecomposedmusicalinteractionsforourregionsthatfollowed thesefourphases.Figure2updatesourillustrationtoshowanumberofadditionaltriggerzonesplaced withthemajorityoftheregions.Typically,theyfanoutfromthelocationofthesculptureexhibitsto

theouterboundaryofaregion.Eachofthese‘internal’triggerzonesenactafurtheradjustmentto themusicplayedinthatregion.Thefollowingdescribesourmusicalapproach:

• Approach:Asavisitorwalkstowardsasculpturetheyanticipatetheforthcominginteraction; consequently,musicwascomposedtoforeshadowedthemomentofarrivalbygraduallyincreasing themusicalintensity(i.e.crescendo)uponentryintoeachsuccessivetriggerzone;

• Arrive:Thiscrescendoisthenresolvedatthemomentofarrival(entryintoafinaltriggerzone encompassinganexhibit).Resolutionistypicallyenactedbyakeychangeandafinalpushof musicaldynamic;

• Engage:Themomentofarrivalthenquicklygiveswaytoareductionofmusicalintensityand activitytoprovidespaceforthevisitortointeractwiththeexhibitwithoutdistraction;

• Depart:Asvisitorswithdrawfromtheexhibitandheadbackoutthroughthetriggerzonesa differentmusicaltreatmentispresented,onethatmaintainsalowlevelofmusicalactivityto preparethevisitorforthenextapproachphaseencountered.

Thetechniquesofmusicallandscapesandmusictrajectoriesenabledforthedesign,composition andauthoringofalocativemusicalexperiencethatexhibitscloseandintertwinedrelationshipsbetween thephysicalandthedigital.Theflowthroughthephysicalparkissynchronisedwiththecourseof theaccompanyingsoundtrack,atbothahighandlowlevelofmusicaldetail.Thisapproachenabled ustoconsiderthephysicalformoftheparkinanewlightwhilstgracefullyguidingandintegrating acoherentstructureintothecompositionofalarge-scalemusicalwork.Wehavenotenteredinto discussionregardstheexperientialimpactofmusicoraudiowhenplaceduponartefacts,orlocales. Rather,weexploredhowmultiplefragmentsofdigitalaudiocanbecomposed,arrangedandplaced toreflectandcomplimenttheinterconnectednessofattributescontainedataspecificlocation.The footingsofthisapproacharetransferabletothedesignoflocativeexperiencesthatdeliverrelationships betweenthephysicalanddigital.

5. MoVING BeyoNd LoCATIoN: SoCIAL MedIA ANd dATA AS CoNTeXT IN MoBILe AUdIo SySTeMS

Location-basedaudioexperiences,whereaudioisgeographicallylocatedwithintheworld,area commonandprevalenttechniquetopresentaudioinplace.Initsmostbasicform,adatabaseof audiofilesisassociatedwithasetofgeographiccoordinates,andpresentedtoauserwhenheorshe entersanactivationzone(usuallywithinafewmeters)ofthegeographicalcoordinates.Whilstother locationandproximitysensingtechnologieshavebeenused,thisbasicapproachhasbeenacommon techniquefromtheearliestwork(Bederson,1995)totheamongstthelatest(Ciolfi&McLoughlin, 2012).Significanttechnicaladvanceshavebeenmade,movingfromstraightforwardplaybackofaudio (Bederson,1995)totheuseofvirtualspatializedauditoryenvironmentsthatcanpresentmultiple overlappingsounds,andvarythesoundspresentedbasedonuserdistancefromthegeographicsource (Vazquez-Alvarez,Oakley,&Brewster,2011).However,theaudioitselfisoftenfixed(existingas aprocreatedfileontheuser’smobiledevice)andismanuallycuratedandfixedtoageographical location.Thismanualcreationmeansthatusershavea“canned”experience,wherethesameaudio istriggeredinthesamephysicallocation.Thisworkswellformanyofthe,usuallyculturalheritage, applicationswherelocation-basedaudioisemployedtoaugmentandenhanceexistingphysical places.AsSmith(Smith,2011)notes:“heritageisnotthehistoricmonument,archaeologicalsite, ormuseumartefact,butrathertheactivitiesthatoccuratthem”.Theembodiedexperience(Ciolfi, 2015)oflocation-basedaudioiswellplacedtofulfilthisrole,contextualisingthebuiltenvironment.

plunderingbuildingswereplayednearthebuildingsthereferredto).Suchworkassumesthatthe augmentedexperienceis“visited”,withtheuserinteractingwiththeaudio-augmentedenvironmentas theirprimarypurpose,beforeswitchingitoffanddoingsomethingelse.Thereisalsonoassumption thattheuserwouldrevisittheexperience(inmuchthesamewayasheorshemayonlyvisitamuseum exhibitiononce),givenitsstaticnature.

Withincreasingamountsofuser-generatedcontentbeingtaggedwiththelocationofitscreation, andanincreasingfocusoncommunityparticipationinthecreationofculturalheritage(Han,Shih, Rosson,&Carroll,2014),whereuserscreategeo-locatedcontentaboutthehistoryoftheirlocal area,weargueitisbothvaluableandnecessarytoconsiderhowlocation-basedaudiocouldactasa ubiquitousmeanstogainnewinsightintoeverydayenvironments.Throughcontentbeingdynamic, thereisvalueinmorecontinuoususe,withtheauditoryenvironmentbecomingsecondarytothe user’smainactivity.Allowing,forexample,theusertogainunderstandingofthe‘heritage’ofplace (Smith,2011),asaby-productofanotheractivity.Forexample,goingforajogcouldbeawayto learnaboutwhatisgoingoninthecommunityviathesamesortoflocation-basedaudiosystemas studiedbyReidetal.(Reidetal.,2005).

Doingsohoweverraisesnewissuesandchallenges.Webrieflypresenttwoexamplesofourown workthataredesignedtobeusedinthisway.PULSE(McGookin&Brewster,2012a):anauditory displaytopresentgeotaggedtwittermessagesinsitu;andExplore:aculturalheritage‘app’that incorporatesseasonal(winterandsummer)variationsintoitscontent.Weoutlinehowreflectionsof bothhighlightchallengesdynamicaudiodisplayofcontenttohelpcontextualisetheenvironment. 5.1. Pulse

PULSE(McGookin&Brewster,2012b)wasdesignedtoallowuserstogaina‘vibe’-anintrinsic understandingofthepeople,placesandactivitiesaroundtheircurrentlocation-derivedfromthe Twittersocialmediaservice(www.twitter.com).PULSEranasan`app’onaniPhone,withapassive userinteraction:interactionwasdrivensolelybyauser’sphysicalmovementthroughtheirenvironment. Asusersmoved,PULSEdownloadedpublicTwittermessages(‘tweets’)generatedbyanyuserinthe currentlocation.Periodically,PULSEwouldselecttheclosesttweet,synthesizeitusingahighquality Text-to-Speechengineandinsertitinavirtual3Dauditoryenvironment.Thususersheardtweets aswhisperedconversationsinpassing.Theseaudiotweetswereaugmentedwithmeta-data(suchas therateofmessagegenerationortrendsinmessagetopic)basedontheimmediategeographicalarea. Thesewereeitherpresentedtousersimplicitly(suchasmappingthetimebetweenspokenmessage presentationstotheratetweetswerebeingcreatedinthearea),orexplicitly(bymappingtheseonto anambientwatersoundscape.Openearbudswereusedtoallowmergingbetweenthevirtualand realauditoryenvironment.WeevaluatedPULSEoveratwo-weekperiodinEdinburgh,UK.Agroup offivelocalresidentswereaskedtouseeachofthetwoversionsofPULSEastheywishedovera five-dayperiod.In-linewithourearlierdiscussion,PULSEwasusedasasecondaryactivity,being integratedwithinotheractivities,ratherthanparticipantssettingtimeasidetouseit.

5.2. explore

andasanaturalexperienceofvisitingtheplace.Explorepresentsstandardnotificationsasauser comesneargeo-locatedcontent(historicalimages,audiovignettesorvideos).Explorealsosupports seasonalityofcontent,asthetemperaturecanvarybetween-20to+20degreescentigrade.Twosets ofcontent,presentingawinterandsummerperspective,couldbeswitchedbetweenbyparticipants. Afieldstudyinvolvingforty-fivevisitorstotheislandwasconductedtoevaluateExplore.

5.3. Future Challenges

Basedontheresultsofbothstudiesweoutlineissuesandchallengesthatshouldbefurtherconsidered whenapplyinglocation-basedaudiomorewidely.

5.3.1. The World Changes

InExploreparticipantscouldswitchbetweeneitherthecurrentseason(summer)oroutofseason content(winter),whilsttheusersofPULSEheardtweetsgeneratedovertheprevious24hours.The temporalrelationshipbetweenwhenthecontentwaslocatedandwhenitwasheardwasimportant. WhilstparticipantswereopentoviewingvisualcontentinExplorethatreflectedthecontentoutof season,suchasthecurrentenvironmentinthewinter,audiocontent,suchasvignettesofhistorical individualsjarredwhenheardoutofseason.Similarly,withPULSE,participantsidentifiedtweetsthey heardthatdidn’tcorrelatewiththecurrentenvironment.Forexample,atweetgeneratedaboutgoing totheparkbecauseitwasasunnyday,whentheenvironmentwassunny,jarredwithaparticipant hearingitintheparkafewhourslaterwhenitwasraining.Yetothermessageswithasimilargap betweencreationandhearingdidnotdothis.Asaudioisalwaysheardasanintegrationintothe existingenvironment,ratherthanbeingclearlydifferenttoit(e.g.viewingvisualcontentonascreen), itmaybeparticipantsaremoresensitivetoaudioincongruence.Howthedynamicelementsofthe environment(morningtoevening,weatherconditionsetc.)affecttherelevanceofcontent,andhow thisshouldbeincorporatedintomobileaudioworkisanunderdevelopedarea.

5.3.2. Filtering and Prioritising Content

InExplore,participantsfounditannoyingwhenahighdensityofnotificationswerepresentedin quicksuccession.Thiswascommoninareaswheretherewasalotofcontent.Intheworkdescribed byHanetal.(Hanetal.,2014),itislikelythatsomeplaceswouldhavehighlevelsofcontent, whilstotherswouldbelower.Howtoprioritiseandfilterthiscontenttoavoiduserannoyanceis important.However,ourworkinPULSEshowsthatcontentviewedasmundane,orlowpriority,can beinterestingbecauseitispresentedinsituviaaudio.Withseveralinstancesofmundane,everyday messageshighlightedbecauseofthis.Whilstnotoverloadingtheuserisimportant,audiocanchange thenatureofwhatausermayconsidermostinsightful.

5.3.3. Considering Interaction as a Secondary Task

5.3.4. The Concurrency of Places

ParticipantsinPULSEwerelocal,usingitinfamiliarlocations.However,inmanycasesitsupported thediscoveryofnewplacesandeventsinfamiliarenvironments.Inparticular,awarenessofsocial activitiesorgroupsofpeoplewithdivergentintereststotheirown:“You’vegotthe‘GodI’vesucha hangoverthismorning’ones.Whichareabitboring,butalsointerestinginaway,causeitreminds youofasocialgroupofpeople”.AnothernotedamessagefromaperformerintheEdinburgh festivalthatwastakingplaceduringourstudy:“Thisguywhosaiditwastwoandahalfhourstill hisshowstarted.Thatwasquiteinteresting.Seeingitfromtheotherside.”Inthisway,PULSEwas abletoillustratehowdifferentcommunities(or`places’)couldexistconcurrentlyinthesamespace. Userunderstandingofthesefamiliarenvironmentswaschangedandprovoked,highlightingother communitiesofpractice,ratherthanthembeing“perceptuallyhidden”duetofamiliarity.Ouruseof apassiveauditorydisplayallowedthistooccurserendipitously,withparticipants‘stumbling’upon newinsightsduringtheirregularactivities,ratherthanhavingtoexplicitlyconsiderlookingforthem.

6. eXPeRIMeNTING wITH THe PAST: FICTIoNAL FUTUReS AS MATeRIAL FoR CoMPoSITIoN ANd deSIGN

Wehaveconductedaseriesofexperiments,underthebannerof“numbersintonotes”,inwhich themathematicsoftheearly19thcenturyisusedtogeneratemusicthatisguidedbyhumanand machineinteraction.Ourapproachplaysatonceintogenerativedesignandintoalternativehistories ofalgorithmsandmechanisms.Wedescribeascenarioinwhichdevicesplacedinaninteractivespace blurthetraditionalboundariesofcomposer,performerandaudience,andsimultaneouslyquestion thecreativeboundariesofhumanandmachine.

Ourworkisre-imaginingprospectiveandtheoreticaltechnologiesofAdaLovelace’stimetoday, inspiringnewresponsesanddiscoveriesrelatingtomusicpracticeandthephilosophyandhistoryof technology.Byco-designingdigitalanddigital-physicalartefactstointerpretthewritingandtimes ofLovelaceandBabbage,weenablecriticalreflectionandre-interpretation.Weframethisapproach asexperimentalhumanities:throughmaking,throughprototypingandco-design,weclose-readthe thoughtprocessesLovelaceandBabbagerecorded,andwecanpointtopathsinthedevelopmentof computingandprogrammingthatwerenottaken,digitallyreachingbeyondwhatwaspracticablein thenineteenthcentury.

6.1. Historical Context

CharlesBabbagegaveaseminarabouthisproposedsteam-poweredcomputer,theAnalyticalEngine, attheUniversityofTurinin1840.OriginallytranslatedintoFrenchforpublication,in1842Ada LovelacetranslateditintoEnglishandaddedextensivenotes.Famouslythesenotescontainthefirst algorithmdesignedforexecutionbyageneral-purposecomputingmachine,whichisoftencelebrated asthefirstcomputerprogram.Theyalsocontaintwoimportantideaswhichhavebeentakenupin subsequentacademicdebateandinformourwork:

• “Supposing,forinstance,thatthefundamentalrelationsofpitchedsoundsinthescienceof harmonyandofmusicalcompositionweresusceptibleofsuchexpressionandadaptations,the enginemightcomposeelaborateandscientificpiecesofmusicofanydegreeofcomplexityor extent.”(noteAinMenabrea,1843).Thisisoftencitedwithinthecomputersandmusicdomain; • “TheAnalyticalEnginehasnopretensionstooriginateanything.Itcandowhateverweknowhow

6.2. experiments

December2015sawthe200thanniversaryofLovelace’sbirth,andamajorsymposiumwasheldto marktheoccasion.Thisincludedthediscussionofathoughtexperiment:hadLovelacelivedlonger, andhadBabbagesuccessfullybuilttheanalyticalengine,whatmighthavehappenedinpursuitof Lovelace’smusicalobservation?(HowardE.andDeRoureD.,2015)

Aspartofthisexperimentwehypothesizedaworkflowinvolvingtheanalyticalengine:the machinerunsaparameterizedprogramtogenerateanumbersequence,andpartsofthissequence arethengiventodifferentinstruments.Wedemonstratedthiswithanemulatorandthendeveloped aninteractiveWebAudiotoolsothat,withoutspecialistknowledge,anyonecansimplymapinteger sequencesformusicalexpression.Theinteractivetool,asinglepagewebapplication,providesseveral algorithmswhichillustratethemathematicsoftheearly19thcentury(http://demeter.oerc.ox.ac.uk/ NumbersIntoNotes).InspiredbytheuseofpunchedcardsintheJacquardloomandtheproposed analyticalengine,wegeneratedvirtual‘pianorolls’andaMIDIoutputthancanbeimportedinto othertoolsforeditingorperformance.

Thenextpartofourthoughtexperimentwastoask“whatwouldLovelacedotoday?”The proposedanalyticalenginearchitecturewasimpressiveevenbytoday’sstandards;forexample,167 binarybitswouldberequiredtodaytoachieveits50decimaldigitarithmetic.However,precision aside,todaythealgorithmsrunreadily(andinrealtime)ontheprocessorsofopensourceArduino hardware,suchasan8bitmicrocontroller,or32bitARM.

OurfirstexperimentwastobuildtheArduinoanalyticalengine.Withaslaveaudioprocessor (VS1053[1])and3Wamplifiertodrivespeakers,wereplicatedtheNumbersintoNotestoolasa smallstandalone“musicengine”.

Fiveofthesedeviceswereconstructed,withavarietyofsensorstoinfluencetheparameters ofthealgorithms:ourprototypesusedinfraredremotecontrolsandultrasoundproximitymeasures. Thesecanbelocatedinaninteractivespace.Hereweexploitamusicaloutcomeofourearlierwork: multiplesequencesplayedalongsideeachothergenerateharmonies,andthiseffectisricherwhen sequencesareplayedatdifferentbutgeometricallyrelatedtempos(e.g.120,60,30beatsperminute). Theresultingharmonysequencesareexperiencedaccordingtothelocationandorientationofthe listenerrelativetothedevices.

Themultipledeployeddevicesareeffectivelycoupledbythehumanssituatedinthespace,and theresultingcompositionisemergent.Wecouldapproachthatashumansenteringthespaceand composingan“experience”forotherhumanswhofollowthem,goinginandinteracting—creating thelocativeaudiointeractivelyandasynchronously.Theexperienceisthereforesociallyand mathematicallyco-constituted.NowwecanreturntoLovelacenoteGandaskwhetherthecomputer originatesthemusic?Furthermore,inourdemonstrationatMobileHCI(DeRoure&Willcox,2016) werecognizedthatnospaceisfreefromsignals,asweexperiencedtheinteractionsofmultipleaudio andinfrareddevicesinthesamephysicalspace.

Whatwouldhappenwithmorearduinosandhowmighttheybeconfigured?Toexplorethis wecreatedasimulationusingthenetLogoagent-basedsimulationenvironment.Thesimulation enablesustoexplorenewdesignfeaturesandincreasingscale.Forexample,thesimulatedarduinos (turtles)can‘hear’eachotherandrespond,therearemultipletypesofarduino,andtheycanbe programmedtomove.

6.2.1. Closing Thoughts –Mobile Music-Boxes

7. CoNCLUSIoN

ThispaperhaspresentedaseriesofperspectivesthatfocuseduponmobileHCIandaudio.Eachof theseperspectivesalthoughdifferenthashighlightedthebreadthofresearchinthisgrowingarea,each withtheirownchallengesandissues.Asmobiletechnologiesandcommunicationinfrastructure,and mobileaudioinput/outmechanismsdevelopnewopportunitiesbecomemoreapparent.Asmobile HCImovesinto‘new’territoryitbecomesapparentthatwhatareneededarenewapproachesand methodsthatallowresearcherstobothunderstandanddesigninteractivesystems.Oneofthekey lessonslearntfromtheworkshopwasnottounderestimatetheresearchareasandfieldsthatare lookingforwaystodevelopandunderstandnewwaysofusingaudio-basedmobiletechnologies. Withthisinmindthefutureofmobileinteractionwillofferanexcitingrangeofopportunitiesacross manyfieldsofresearch.

ACKNowLedGMeNT

ReFeReNCeS

Abowd,G.D.,Atkeson,C.G.,Hong,J.,Long,S.,Kooper,R.,&Pinkerton,M.(1996).Cyberguide:AMobile Context-AwareTourGuide.Baltzer Journals, September 2.

Bederson,B.B.(1995).Audioaugmentedreality:aprototypeautomatedtourguide.InConference companion on Human factors in computing systems.NewYork,NY,USA:ACM.doi:10.1145/223355.223526

Behrendt,F.(2005).FromCallingaCloudtoFindingtheMissingTrack:ArtisticApproachestoMobileMusic.

Proc. of 2nd International Workshop on Mobile Music Technology(Vol.5).

Behrendt,F.(2010).Mobile Sound: Media Art in Hybrid Spaces[PhDDissertation].UniversityofSussex. Benford,S.,Anastasi,R.,Flintham,M.,Drozd,A.,Crabtree,A.,Greenhalgh,C.,&Row-Farr,J.et al.(2003). CopingwithUncertaintyinaLocation-BasedGame.IEEE Pervasive Computing,2(3),34–41.doi:10.1109/ MPRV.2003.1228525

Benford,S.,&Giannachi,G.(2011).Performing Mixed Reality.TheMITPress.

Blistein,J.(2015).SwedishBandMakeAlbumAvailableExclusivelyintheWoods.Rolling Stone,May28. Boden,M.A.(1990).The Creative Mind: Myths and Mechanisms.London:WeidenfieldandNicholson. Bødker,M.,&Chamberlain,A.(2016,June12-15).AffectTheoryandAutoethnographyinOrdinaryInformation Systems.Proceedings of 2016 European Conference on Information Systems ECIS ‘16,Istanbul,Turkey. Bryan-Kinns,N.,Airantzis,D.,Angus,A.,Fencott,R.,Lane,G.,Lesage,F.,Marshall,J.,Martin,K.,Roussos,G andTaylor,J.(2009).SensoryThreads:PerceivingtheImperceptible.InIntelligent Environments(pp.404–410). Button,G.(2000).Theethnographictraditionanddesign.Design Studies,21 (4),319–332.doi:10.1016/S0142-694X(00)00005-3

Chamberlain,A.,Bødker,M.,Hazzard,A.,&Benford,S.(2016).AudioinPlace:Media,MobilityandHCI- CreatingMeaninginSpace.Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. Mobile HCI ‘16.ACMPress.

Chamberlain,A.,Rowland,D.,Foster,J.,&Giannachi,G.(2010).RidersHaveSpoken:ReplayingandArchiving PervasivePerformances.Leonardo,43(1),90–91.doi:10.1162/leon.2010.43.1.90

Ciolfi,L.(2015).Embodiment and Place Experience in Heritage Technology Design.TheInternationalHandbooks ofMuseumStudies.doi:10.1002/9781118829059.wbihms319

Ciolfi,L.,&McLoughlin,M.(2012).DesigningforMeaningfulVisitorEngagementataLivingHistoryMuseum.

Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design(pp. 69–78).NewYork,NY,USA:ACM.doi:10.1145/2399016.2399028

Collier,D.(2015).Bluebrain–MobileMusic.Retrievedfromhttp://davidcollier.ie/bluebrain-listen-to-the-light/ Crabtree,A.,Benford,S.,Rodden,T.,Greenhalgh,C.,Flintham,M.,Anastasi,R.,&Tandavanitj,N.et al.(2004). OrchestratingaMixedRealityGame’ontheGround.Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp.391–398).ACM.

DeRoure,D.,&Willcox,P.(2016).Numbersinplaces:creativeinterventionsinmusicalspace&time.

Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - Mobile- HCI ’16(pp.1059–1063).ACM.doi:10.1145/2957265.2964199

DeRoure,D.,Willcox,P.,&Chamberlain,A.(2016a).ExperimentalDigitalHumanities:Creativeinterventions inalgorithmiccompositiononahypotheticalmechanicalcomputer.DMRN+11:DigitalMusicResearch Network,December.

Dourish,P.(2006).Implicationsfordesign.Proceedings of CHI 2006,ACMPress.

Eslambolchilar,P.,Bødker,M.,&Chamberlain,A(2016).WaysofWalking:UnderstandingWalking’s ImplicationsfortheDesignofHandheldTechnologyviaaHumanisticEthnographicApproach.Human Technology: An Interdisciplinary Journal on Humans in ICT Environments.

Flintham,M.,Benford,B.,Anastasi,R.,Hemmings,T.,Crabtree,A.,Greenhalgh,G.,&Row-Farr,J.et al. (2003).Whereon-LineMeetsontheStreets:ExperienceswithMobileMixedRealityGames.Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp.569–576).ACM.doi:10.1145/642611.642710 Fosh,L.,Benford,S.,Reeves,S.,Koleva,B.,&Brundell,P.(2013).SeeMe,FeelMe,TouchMe,HearMe: TrajectoriesandInterpretationinaSculptureGarden.Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp.149–158).ACM.doi:10.1145/2470654.2470675

Galani,A.(2003).MixedRealityMuseumVisits:UsingNewTechnologiestoSupportCo-VisitingforLocal andRemoteVisitors.Museological Review, 10.

Gaye,L.,Holmquist,L.E.,Behrendt,F.,&Tanaka,A.(2006).MobileMusicTechnology:Reportonan EmergingCommunity.Proceedings of the 2006 Conference on New Interfaces for Musical Expression(pp. 22–25).IRCAM—CentrePompidou.

Gildersleeve,Z.(2015).AnExplorationofMobileAugmentedReality.Retrievedfromhttp://sww.zachgildersleeve. com/downloads/finalReport.pdf

Greg,E.T.,&Tomlinson,B.(2006).PersonalSoundtrack:Context-AwarePlaylistsThatAdapttoUserPace.In CHI’06ExtendedAbstractsonHumanFactorsinComputingSystems(pp.736–741).ACM.

Han,K.,Shih,P.C.,Rosson,M.B.,&Carroll,J.M.(2014).EnhancingCommunityAwarenessofand ParticipationinLocalHeritagewithaMobileApplication.Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing(pp.1144–1155).NewYork,NY,USA:ACM. doi:10.1145/2531602.2531640

Hazzard,A.,Benford,S.,&Burnett,G.(2015).SculptingaMobileMusicalSoundtrack.Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems.ACM.

Hazzard,A.,Benford,S.,Chamberlain,AandGreenhalgh.C.(2015).ConsideringMusicalStructureinLocation-BasedExperiences.Proceedings of NIME ‘15.

Hemment,D.(2006).LocativeArts.Leonardo,39(4),348–355.doi:10.1162/leon.2006.39.4.348

Howard,E.,&DeRoure,D.(2015).Turningnumbersintonotes”inAdaLovelaceSymposium2015-Celebrating 200YearsofaComputerVisionary.Proceedings of the Ada Lovelace Symposium ’15.ACMPress.

Izadi,S.,Fraser,M.,Benford,S.,Flintham,M.,Greenhalgh,C.,Rodden,T.,&Schnädelbach,H.(2002).Citywide: SupportingInteractiveDigitalExperiencesacrossPhysicalSpace.Personal and Ubiquitous Computing,6(4), 290–298.doi:10.1007/s007790200030

Kirisits,N.,Behrendt,F.,Gaye,L.,&Tanaka,A.(2008).Creative Interactions–The Mobile Music Workshops 2004-2008.DieAngewandtePress.

LaBelle,B.(2006).Background Noise: Perspectives on Sound Art.ContinuumBooks. Maanen,J.(1988).Tales of the Field.Chicago:ChicagoUniversityPress.

McGookin,D.,&Brewster,S.(2012a).PULSE:thedesignandevaluationofanauditorydisplaytoprovideasocial vibe.Proceedings of the CHI 2012(pp.1263–1272).NewYork,NY,USA:ACM.doi:10.1145/2207676.2208580 McGookin,D.,&Brewster,S.(2012b).PULSE:thedesignandevaluationofanauditorydisplaytoprovidea socialvibe.Proc. CHI 2012(pp.1263–1272).Austin,TX,USA:ACM.doi:10.1145/2207676.2208580 Mcgookin,D.,Tahiro,K.,Vaittinen,T.,Kyt,M.,Monastero,B.,&Vasquez,J.C.(2017).ExploringSeasonality inMobileCulturalHeritage.ProceedingsofCHI‘17.ACMPress.

McPherson,A.,Chamberlain,A.,Hazard,A.,McGrath,S.,&Benford,S.(2016,June4-8).Designingfor ExploratoryPlaywithaHackableDigitalMusicalInstrument.Proceedings of Designing Interactive Systems, DIS’16,Brisbane,Australia.ACMPress.doi:10.1145/2901790.2901831

Menabrea,L.F.(1843).SketchoftheAnalyticalEngineinventedbyCharlesBabbage,Esq.InR.Taylor(Ed.), ScientificMemoirs(Vol.3).London,UK:RichardandJohnE.Taylor.

Morreale,F.,Moro,G.,Chamberlain,A.,Benford,S.,&McPherson,A.(2017).BuildingaMakerCommunity AroundanOpenHardwarePlatform.Proc. of CHI’17,Denver,USA.

Reid,J.,Cater,K.,Fleuriot,C.,&Hull,R.(2005).‘Experience Design Guidelines for Creating Situated Mediascapes’. Mobile and Media Systems Laboratory.HPLaboratoriesBristol.

Reid,J.,Geelhoed,E.,Hull,R.,Cater,C.,&Clayton,B.(2005).Parallelworlds:immersioninlocation-based experiences.ProceedingsofCHI‘05(pp.1733–1736).Portland,USA:ACM.

Rueb, T. (2002). Sonic Space-Time: Sound Installation and Secondary Orality.Consciousness Reframed. Retr ieved from http://locative.ar ticule.net/wp-content/uploads/2013/06/ TRuebSonicSpaceTimeSoundInstallationAndSecondaryOralitypdf

Smith,L.(2011).TheDoingofheritage:heritageasperformance.InPerforming Heritage: Research, Practice and Innovation in Museum Theatre and Live Interpretation(pp.69–81).

Tanaka,A.(2008).VisceralMobileMusicSystems.InTransdisciplinaryDigitalArt.Sound,Visionandthe NewScreen(pp.155–170).Springer.doi:10.1007/978-3-540-79486-8_15

Truax,B.(2001).Acoustic Communication(2nded.).Westport,CT:GreenwoodPress.

Turing,A.M.(1950).ComputingMachineryandIntelligence.Mind,49(236),433–460.doi:10.1093/mind/ LIX.236.433

Tuters,M.(2004).The Locative Utopia.TransculturalMappingLocativeReader.

Vazquez-Alvarez,Y.,Oakley,I.,&Brewster,S.(2011).Auditorydisplaydesignforexplorationinmobileaudio-augmentedreality.Personal and Ubiquitous Computing.doi:10.1007/s00779-011-0459-0

Vlahakis,V.,Ioannidis,M.,Karigiannis,J.,Tsotros,M.,Gounaris,M.,Stricker,D.,&Almeida,L.et al.(2002). Archeoguide:AnAugmentedRealityGuideforArchaeologicalSites.IEEE Computer Graphics and Applications,

22(5),52–60.doi:10.1109/MCG.2002.1028726

Waern,A.,Montola,M.,&Stenros,J.(2009).‘TheThree-SixtyIllusion:DesigningforImmersioninPervasive Games’.InProceedings of the SIGCHI Conference on Human Factors in Computing Systems,1549–1558.ACM. doi:10.1145/1518701.1518939

Yu,Z.,Zhou,X.,Yu,Z.,HyukPark,J.,&Ma,J.(2008).iMuseum:AScalableContext-AwareIntelligentMuseum System.Computer Communications,31(18),4376–4382.doi:10.1016/j.comcom.2008.05.004

Alan Chamberlain is a Senior Research Fellow in the Mixed Reality Lab, Computer Science at the University of Nottingham. He has published numerous papers on many aspects of Human Computer Interaction. His research interests relate to real-world problems as articulated by various communities of practice and relate to Human Computer Interaction, Ethnography, Action Research, Participatory Design, Digital Arts/Humanities and User Engagement. Mads Bødker is Associate Professor at the Department of IT Management, Copenhagen Business School. His work combines human-computer interaction and user experience with ideas from the arts, philosophy, and anthropology. Adrian Hazzard is a Research Fellow in the Mixed Reality Lab, School of Computer Science, University of Nottingham. Adrian has a background in music performance and composition. The main focus of his current research is in interactive musical experiences from both a compositional and experiential viewpoint.

David McGookin is an Assistant Professor at Aalto University Finland in Human-Computer Interaction. His work focuses on multi-sensory interaction in mobile and nomadic environment. In particular, he focuses on how individuals engage with geo-tagged media in their immediate environments through different senses.

David De Roure is Professor of e-Research and Director of the Oxford e-Research Centre. He has strategic responsibility for Digital Humanities at Oxford and directed the national Digital Social Research programme for ESRC, for whom he is now a strategic adviser. His personal research is in Computational Musicology, Web Science, and Internet of Things. He is a frequent speaker and writer on digital scholarship and the future of scholarly communications.