Daniela and KUBITZA, Thomas
Available from Sheffield Hallam University Research Archive (SHURA) at:
This document is the author deposited version. You are advised to consult the
publisher's version if you wish to cite from it.
RISSEEUW, Martin, CAVADA, Dario, NOT, Elena, ZANCANARO, Massimo,
MARSHALL, Mark, PETRELLI, Daniela and KUBITZA, Thomas (2016). An authoring
environment for smart objects in museums : the meSch approach. In: ARDITO,
Carmelo, BELLUCCI, Andrea, DESOLDA, Giuseppe, DIVITINI, Monica and MORA,
Simone, (eds.) Smart Ecosystems cReation by Visual dEsign 2016. CEUR-WS,
Copyright and re-use policy
Sheffield Hallam University Research Archive
An Authoring Environment for Smart
Objects in Museums: the meSch
Abst r a ct
The m eSch proj ect addresses t he challenges of creat ing a personally m eaningful, sensorily rich, and socially expanded m useum visit or experience t hrough t angible and em bodied int eract ion w it h digit al cont ent . I t is of param ount im port ance t hat cult ural herit age
professionals are direct ly involved in t he design of t hose experiences. The m eSch approach is t o em pow er cult ural herit age professionals w it h t ools t hat guide t hem t hrough a do- it - yourself process of creat ing or adapt ing digit ally augm ent ed experiences for t heir ow n m useum spaces, t herefore r educing t he barriers of int roducing I nt ernet of Things t echnology in cult ural herit age spaces.
Au t h or Ke y w or ds
Tangible and Em bodied I nt eract ion; Aut horing Tool; Cult ural Herit age.
ACM Cla ssifica t ion Ke y w or ds
H.5.m . I nform at ion int erfaces and present at ion ( e.g., HCI ) : Miscellaneous; See
Past e t he appr opr iat e copyr ight / license st at em ent her e. ACM n ow suppor t s t hr ee differ ent publicat ion opt ions:
• ACM copyr ight : ACM h olds t he copyr ight on t he wor k. Th is is t he hist or ical appr oach.
• License: The aut hor ( s) r et ain copy r ight , but ACM r eceives an exclusive publicat ion license.
• Open Access: The aut hor ( s) w ish t o pay for t he wor k t o be open access. The addit ional fee m ust be paid t o ACM.
This t ext field is lar ge enough t o h old t he appr opr iat e r elease st at em ent assum in g it is single- spaced in Ver dana 7 point font . Please do not change t he size of t h is t ext box.
Each subm ission w ill be assigned a unique DOI st r ing t o be in cluded her e.
M a r t in Risse e uw
WAAG Societ y
Am st er dan, Net her lands
m ar t in@w aag.or g
D a r io Ca v a da
eCTRL Trent o, I t aly
cavada@ect rlsolut ions.com
Ele n a N ot
M a ssim o Za n ca na r o
FBK – Fondazione Bruno Kessler Trent o, I t aly
M a r k T. M a r sha ll D a n ie la Pe t r e lli
Sheffield Hallam Univer sit y Sheffield, UK
m .m email@example.com
Th om a s Ku bit z a
Univer sit y of St ut t gar t St ut t gar d, Ger m any
firstname.lastname@example.org ut t gar t .de
Copyright is held by the author/owner(s).
I n t r odu ct ion
Wit hin t he Mat erial Encount ers w it h Digit al Cult ural Herit age ( m eSch1) proj ect [ 3] , w e are w orking on t echnology t o support t he w hole m useum visit ing experience by fost er ing a vision w here m at erialit y and rich sensory int eract ion add t o t he cognit ive aspect s for a personally m eaningful experience. There is a w ide recognit ion t hat such approach m ay great ly im prove bot h t he visit and also visit ors’ appreciat ion for t he m useum ’s cult ural values [ 1] .
Figure 1 A visit or int er act ing wit h a sm art replica in t he At lant ic Wall exhibit ion [ 2] ;
The idea behind m eSch is t aking obj ect s out of t heir cases ( or using replicas like in [ 2] ) t o let visit ors experience obj ect s’ shape and w eight w hile learning about t hem ( figure 1) , t o engage t he visit ors at a physical level and put infor m at ion in cont ext w it hout t he dist ract ions induced by int eract ing w it h ext raneous devices, apps, and t ouch screens.
As an exam ple, t he m eSch plat form has been
dem onst rat ed at t he At lant ic Wall exhibit ion at Museon in The Hague, The Net herlands2. The At lant ic Wall w as
1 ht t p: / / m esch- pr oj ect .eu
2 ht t p: / / www .m useon.nl/ en/ exhibit ions/ hague- and- at lant ic- w all
t he Germ an defense line along t he European w est coast erect ed during Wor ld War I I t o block t he allied invasion. The exhibit ion w as m eant t o reflect on how t he cit y of The Hague and it s cit izens had been affect ed. The m eSch plat form has been used t o enrich t he display of hist orical art efact s w it h voices from t he past , such as t hose of t he Germ an soldiers, t he Dut ch civilians and t he civil servant s3. Visit ors could play t hese narrat ives by using ‘sm art replicas’, reproduct ions of real m useum obj ect s ( e.g. a reproduct ion of a surrogat e t ea bag, a 3D- print ed replica of a Delft blue m ug, and so on) enhanced w it h sensors. By placing t heir sm art replica on hot spot s on int eract ive cases visit ors could list en t o narrat ives and see hist orical phot ographs and video clips t hus enriching t he original m useum obj ect s on display4. From April t o Novem ber 2015, m ore t han 14,000 visit ors int eract ed w it h t he m eSch sm art replicas at t he hot post s.
The act ive involvem ent of cult ural herit age professionals in t he preparat ion of t his kind of experiences is of param ount im port ance, even if t hey oft en lack t echnical skills [ 3] . The m eSch approach consist s in fost ering reuse over creat ion from scrat ch. A cat alogue of reusable narrat ive and int eract ion
st rat egies is provided as st art ing point s t hat can be adapt ed and refined.
Th e in gr e die n t s for a d igit a lly a u gm e n t e d
e x pe r ie n ce
The approach t o experience creat ion and
personalizat ion in m eSch is based on a clear separat ion
3ht t p: / / m w 2016.m useum sandt hew eb.com / glam i/ t he- hague-
and-t he- aand-t lanand-t ic- w all- w ar- in- and-t he- ciand-t y- of- peace/
of cont ent from int eract ion, and aim s at facilit at ing t he preparat ion and t he reuse of ( i) narrat ive t hreads t hat can be adapt ed t o different visit ors and t ypes of experience and ( ii) int eract ion st rat egies t hat describe how cont ent should be released in cont ext w hile experiencing t he obj ect s and t he space.
Figure 2. The m eSch Magazine: t he st art ing point for br owsing and adapt ing digit ally augm ent ed experiences
We define a declarat ive for m alism , t he experience
schem a t hat support s t he descript ion of t he cont ent
it em s, t he available int eract ions, t he sem ant ic annot at ions and t he condit ional rules t hat govern t he adapt ive com posit ion of t he experience t hat is delivered t o visit ors at runt im e [ 4] . This is done t hrough a 4-layer dat a st ruct ure including: t he narrat ive, t he
appliance, t he device and t he int eract ion script : ( i) t he narrat ive: a set of carefully select ed digit al cont ent
it em s, annot at ed int o alt ernat ive t hem at ic t reads and levels of det ail t hat revolve around t he obj ect s and places of an exhibit ion; ( ii) t he appliance: a declarat ive specificat ion of t he capabilit ies of t he t echnology em bedded in t he m useum prem ises ( such as, for exam ple, t he possibilit y t o det ect t he visit ors’ proxim it y
t o a specific exhibit ) ; ( iii) t he device: t he descript ion of t he act ual hardw are required for deploying t he syst em and it s t echnical requirem ent s; ( iv) t he int eract ion
script : t he int eract ion rules t hat govern w hen
present at ions should be st art ed or how t he syst em behaviour should be adj ust ed t o personalize t he experience t o visit ors’ cont ext ( e.g., a set of rules t hat act ivat e t he proj ect ion of t he m ost appropriat e cont ent about an exhibit w hen visit ors are in it s vicinit y) .
This m ult ilayer definit ion of t he experience schem a concept support s t he follow ing propert ies: ( i)
reusabilit y: an experience schem a defines a skelet on of
experience t hat can be reused w it h different cont ent and in different physical set t ings t o creat e m any experience inst ances; ( ii) m odularit y: part s of an experience schem a can be replaced t o creat e a new experience schem a. For exam ple, t he int eract ion script can be changed t o int roduce different rules for firing t he present at ion of t he sam e cont ent in cont ext . Anot her exam ple is t he possibilit y t o exper im ent t he sam e t ype of experience w it h an alt ernat ive hardw are set - up; ( iii) com posit ionalit y: different exper ience schem as can be com posed t oget her t o creat e m ore art iculat ed augm ent ed exhibit ions; ( iv) r educt ion of
com plexit y: aut hors can concent rat e on t he preparat ion
low level sensor logs, t hus facilit at ing t he
experim ent at ion of different t echnological solut ions.
Th e m e Sch sce n a r io of a u t h or in g a digit a lly
a u gm e n t e d visit e x p e r ie n ce
A cat alogue of reusable narrat ive and int eract ion st rat egies w it h st ep- by- st ep inst ruct ions on how t o inst ant iat e t hem for a specific m useum and t he t ype of visit personalizat ion t hat t hey support is provided t o assist curat ors in t heir t ask. This cat alogue is offered in t he form of an elect ronic m agazine of digit ally
augm ent ed m useum experiences already t est ed in m useum s or out door cult ural herit age sit es ( see figure 2) . Curat ors m ay brow se t he m agazine and t ake inspirat ion from w hat ot her m useum s have successfully adopt ed for t heir visit ors.
Figure 3. The sem ant ic dim ensions of t he experience schem a for t he At lant ic Wall exhibit ion: changing t he dim ensions is an easy appr oach t o adapt t he cont ent st r uct ure when port ing t he experience schem a t o a new exhibit ion.
Once t hey find a digit ally augm ent ed experience t hat m ay suit t heir needs, curat ors m ay replicat e t he “ recipe” of t he experience ( w hich consist s in it s experience schem a) and it er at ively adapt it t o t heir specific needs.
The first and t he sim plest st ep consist s in adj ust ing t he nam es of t he sem ant ic dim ensions used t o annot at e t he cont ent net w ork ( e.g. t hem at ic t hreads, out put languages, t ypes of audience) t o t he specific dom ain of t he new exhibit ion ( Figur e 3) .
The follow ing st ep consist s in replacing t he cont ent m at erial: t he narrat ive part of t he experience schem a. Figure 4 show s how a curat or can define new cont ent ( a narrat ive it em ) for a specific t hem e, point of int erest and language. During t he phase of cont ent preparat ion, curat ors m ay use t heir propriet ary cont ent or benefit from t he cont ext ual search facilit ies t hat support t he search in Europeana ( a virt ual European library w it h t he goal t o m ake Europe's cult ural herit age accessible5) of cont ent it em s t hat are relevant for t he current experience edit ing t ask.
Figure 5. The m apping bet ween t he sem ant ic cat egories t hat annot at e t he cont ent and t he devices t hat sense t he int eract ion cont ext ( for exam ple an NFC t ag at t ached t o t he sm ar t r eplica is associat ed t o language and t hem e preferences of t he visit or )
The int eract ion script s m ay be adapt ed as w ell t hough t hat m ay require higher t echnical skills. Yet , a curat or can cust om ize an int eract ion script t o t he specific needs of a new exhibit ion by set t ing t he script param et ers: for exam ple, t he dist ance range for recognizing proxim it y and t he m apping bet w een t he sem ant ic cat egories t hat
5ht t p: / / www .europeana.eu/
annot at e t he cont ent and t he sensor devices used t o capt ure t he visit or’s int eract ion ( Figure 5) .
When t he edit ing is com plet e, t he package of t he cont ent it em s t hat build up t he narrat ive and t he rules t hat govern t he cont ext - dependent play during t he visit are t ransferred t o t he onsit e server t hat m anages t heir applicat ion.
Pr e lim in a r y Eva lu a t ion
The aut horing t ool w as developed t hrough an it erat ive design- evaluat ion- redesign process in st rict
collaborat ion w it h cult ural herit age professionals. A prelim inary evaluat ion cycle t ook place over a period of t hree w eeks in t he form of a dist ribut ed expert
This form at ive evaluat ion great ly influenced t he follow ing st age of part ial redesign and act ual
im plem ent at ion of t he aut horing int erface. The second high- fidelit y prot ot ype w as t hen t est ed for usabilit y by ot her t hree cult ural herit age professionals w ho had t o perform in aut onom y a cont rolled t ask of recipe reuse, result ing in furt her am eliorat ions. The t hird version of t he syst em w ill now be used in open w orkshops ( “ aut horing feast s” ) t o engage pot ent ial users of t he syst em in creat ing t heir ow n sm art obj ect s.
Con clu sion
I n t his paper, w e described a research prot ot ype for aut horing t angible and em bodied int eract ion w it h digit al cont ent by cult ural herit age professionals. The
approach is t o em pow er cult ural herit age professionals w it h t ools t hat guide t hem t hrough a do- it - yourself process of creat ing or adapt ing digit ally augm ent ed experiences for t heir ow n m useum spaces. We propose a form alism , t he experience schem a, t hat is based on a clear separat ion of cont ent from int eract ion, and aim s at facilit at ing t he preparat ion and t he reuse of narrat ive t hreads t hat can be adapt ed t o different visit ors and t ypes of experience and int eract ion st rat egies t hat describe how cont ent should be released in cont ext w hile exper iencing t he obj ect s and t he space.
Ack n ow le d ge m e n t
This w ork w as done as part of t he Mat erial Encount erS w it h digit al Cult ural Her it age ( m eSch) proj ect . The proj ect ( 2013- 2017) receives funding from t he
European Com m unit y’s Sevent h Fram ew ork Program m e ‘I CT for access t o cult ural resources’ ( I CT Call 9: FP7-I CT- 2011- 9) under t he Grant Agreem ent 600851.
Re f e r e n ce s
1. Dudley, S. ( 2010) . Museum m at erialit ies: Obj ect s, sense and feeling. I n S. Dudley ( ed.) . Museum Mat erialit ies: Obj ect s, Engagem ent s,
I nt erpret at ions. Rout ledge.
2. Marshall, M.T., Dulake N., Ciolfi L., Durant i D., Kockelkorn H., Pet relli D. ( 2016) Using Tangible Sm art Replicas as Cont rols for an I nt eract ive Museum Exhibit ion. I n Proceedings of TEI 2016
3. Pet relli D., Ciolfi L., van Dij k D., Hornecker E., Not E., Schm idt A. ( 2013) I nt egrat ing m at erial and digit al: a new w ay for cult ural herit age. ACM int eract ions m agazine 20 ( 4) , 58
4. Zancanaro, M., Not E., Pet relli D., Marshall M., van Dij k T., Risseeuw M., van Dij k D., Vent urini A., Cavada D., Kubit za T. ( 2015) Recipes for t angible and em bodied visit experiences. I n Proceedings of