Strathprints Institutional Repository
Dunlop, Mark and Komninos, Andreas and Durga, Naveen (2014)
Towards high quality text entry on smartwatches. In: CHI 2014,
2014-04-26 - 2014-05-01. , http://dx.doi.org/10.1145/2559206.2581319
This version is available at http://strathprints.strath.ac.uk/47209/
Strathprints
is designed to allow users to access the research output of the University of
Strathclyde. Unless otherwise explicitly stated on the manuscript, Copyright © and Moral Rights
for the papers on this site are retained by the individual authors and/or other copyright owners.
Please check the manuscript for details of any other licences that may have been applied. You
may not engage in further distribution of the material for any profitmaking activities or any
commercial gain. You may freely distribute both the url (
http://strathprints.strath.ac.uk/
) and the
content of this paper for research or private study, educational, or not-for-profit purposes without
prior permission or charge.
Any correspondence concerning this service should be sent to Strathprints administrator:
Towards High Quality Text
Entry on Smartwatches
Abstract
Smartwatches now provide users with access to many applications on smartphones direct from their wrists, without the need to touch their smartphone. While applications such as email, messaging, calendar and social networking provide views on the watch, there is normally no text entry method so users cannot reply on the same device. Here we introduce requirements for smartwatch text entry, an optimised alphabetic layout and present a prototype implementation together with preliminary user feedback. While raising some problems, the feedback gives indicates that reasonable quality and speed is achievable on a smartwatch and encourages our future work.
Author Keywords
Text entry; smart watches; mobile usability
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.
Introduction
Text entry is a key component of many smartphone applications, ranging from adding simple diary entries through social network postings to writing complex business emails and documents. The recent release of smartwatches has met considerable interest, but without
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
CHI 2014, Apr 26 - May 01 2014, Toronto, ON, Canada ACM 978-1-4503-2474-8/14/04.
http://dx.doi.org/10.1145/2559206.2581319
Mark Dunlop
Computer and Information Sciences University of Strathclyde
Glasgow G1 1XH, Scotland UK [email protected]
Andreas Komninos
Computer and Information Sciences University of Strathclyde
Glasgow G1 1XH, Scotland UK [email protected]
Naveen Durga
KeyPoint Technologies Ltd
Raja Praasadamu Junction, Botanical Gardens Road Kondapur, Hyderabad 500081, India
Figure 1: Prototype smartwatch
text entry the interaction is frustratingly limited. One can see posts, short-messages and emails but one can't reply on the same device. In this work-in-progress we outline a text entry approach for smartwatches and also describe our initial prototype (Fig. 1) and an initial small-scale usability study.
Text entry on small devices
Before the widespread adoption of touch screen smartphones, 12-key physical keypad phones were the most common text entry method on small devices (sometimes much smaller than the current relatively large screen phones). Predictive technologies (e.g. [4, 10, 11]) interpreted the ambiguous keys (usually three or four letters per key) into words. This approach was shown to achieve around 10 words per minute (wpm) for novices and around 20-25 wpm for experts in controlled studies [12]. We investigated using this approach with reduced number of keys - initially targeting watches [3], but due to technological restrictions our system was implemented on a touchscreen handheld, and later, to reduce movement, on physical key phones [6]. While in theory ambiguous predictive text quality was very high (over 90% accurate), each key sequence could match many different words. Some of these sequences included pairs of common words that caused particular problems (e.g. on a standard phone keypad he and if were typed on the same keys, as were good and home). The early models of prediction were based on simple unigram dictionary models where the most common word matching a sequence was suggested. Nowadays, phones have much more power and memory so can easily support more complex prediction models, which greatly reduces the impact of ambiguity by taking the context of the words into account.
Alternative approaches for input on small devices include handwriting with a stylus, but this has been shown to be relatively slow at under 20 wpm [16], fast but difficult to learn chord keyboards (e.g. [15]), specialised alphabets (e.g. [8, 19]) and sending handwritten image messages instead of text [13].
One system of particular note is ZoomBoard [17] that miniaturises the QWERTY keyboard onto a watch, with the user first zooming into an area then picking a letter. ZoomBoard achieved approx. 10wpm in studies and our work is essentially investigating whether predictive based approaches can surpass this using single taps per letter. Many domestic appliances such as TV guides and games consoles use a date-stamp inspired method, where the user scrolls through the alphabet and picks letters on a 2D line or 3D grid. In their early work on small device text entry Bellman and MacKenzie [1] showed this to be a slow entry method that was not particularly helped by dynamic optimisation.
Speech input is an obvious alternative but is still prone to problems with background noise, spoken accents and is less private and discrete than typing.
Criteria for smartphone style text entry on
smartwatches
For appropriate interaction on a smartwatch we
constrained our design by the following criteria, derived from the literature and our own experience:
Entry must be based on finger use: for casual interaction using a stylus is inappropriate.
Entry must target a small display (e.g. 25x25 mm) with simple touchscreen interaction.
Figure 2: Backs Word Toggle Nume Table 1: G entry layout pace completion e capitalisation eric punctuation m Gesture definitions
Initia
We de as this Althou 20]) h felt th watch into se mode Limited dynam smartwatches h with much proc phone. Providin can challenge t devices. Entry should fo texts but shoulo Comm in upp o Numbe o Comm o Comm social Should support movement and For accurate ta should be reaso Interaction sho are easy to lea users are alrea taps and simple are independen
al Design
ecided to focus on s is the quickest s ugh word based ge have shown great hat simple taps on environment. As even zones as sho
ic feedback - curre have limited on de cessing handled on ng dynamic graphi the communication ocus on small mess d support a wide r mon Western Europ er and lower case ers
mon punctuation; mon symbols (e.g.
networking symbo t simple editing of d backspace). apping with fingers
onably large, e.g. ould include simple rn and remember dy familiar with, e e short unidirectio nt of their location
taps for the prime imple interaction t esture input techn benefits for mobil large keys are mo such we segment own in Figure 2. Zo ent evice processing n the paired ical feedback n between
sages and short range of input: pean alphabets
;
widely used ols).
text (e.g. caret s touch targets >= 7 mm. e gestures that
and which e.g. on-screen onal swipes that
on screen.
e input method, to perform. niques (e.g. [14,
e phones, we ore suited to the ted the display
ones 1 to 6 form large the curren While ther keyboard l maintain a usability. F 1…6 with t entry syste Interaction 1. A nu re di nu 2. A w su 3. Sw pu ar av us 4. A m lo ex While we d ordering, t alphabet. T here: amb e ambiguous keys t input text and ac re has been consid
layouts (e.g. [2, 5 a standard alphabe For alphabet entry the input being dis em (running on th n was defined as f
tap on an ambigu umber and update eflect the most like isambiguation eng umber-key sequen first tap on the ce ith subsequent tap uggestions that ma wipes are defined unctuation and nu re replaced with a vailable, a rightwa sed for word comp long press on the mode to allow move
ong press on the a xtended character decided to restrict there are, of cours There are two com biguity of the layou
while the centre z cts as a space bar derable work on op 5, 9, 18]), here we etical layout to aid y the user will type sambiguated by th he paired smartpho follows:
uous key enters th es the current wor ely word from the gine based on the
nce.
entral zone adds a ps rotating round atch the ambiguou as shown in table
meric modes the lternative layouts. ards swipe gesture
pletion.
e centre zone ente ement of the care lphabetic keys wil rs for that key (e.g
the design to an a se, many ways to mpeting optimisati ut vs movement d zone shows r. ptimised e decided to d pick-up e on keys he text one). hat key rd display to current a space, alternative us entry. e 1. In zones 1…6 . Where e can be rs edit t while a l show g. à, å, ç). alphabetic split the on criteria istance.
To reduce ambiguity errors the best assignment of letters to keys would result in the lowest ambiguity of keystrokes by separating letters that can commonly cause confusion when in the same location in a word (e.g. putting a and e on the same key would be
problematic as common words such as bed and bad, for example, are only differentiated by this pair). Arranging the splits can help minimise the distance a user has to move his/her finger when entering text by putting commonly co-occurring letters on the same key. In the
extreme case putting all 26 letters on one key would minimise the amount of movement of the fingers while typing, but at a massive cost to ambiguity.
We analysed the 53,131 possible alphabetic
arrangements using a normalised ambiguity score based on badgrams frequencies for English and distance based on bigram data (using same data as [5]). The least ambiguous keyboard was abcd efgh ijklm nop qrs tuvwxyz while the keyboard with least travel for the finger was abcdefghijklmnopqrstu v w x y z. Figure 3 shows the distribution of the layouts (both axes are scaled to the range 0…1, where 0 is the worst we found and 1 the best). Because distances are small on a watch, we felt it more important to select a keyboard layout that minimised ambiguity rather than movement. Hence, to select a keyboard we took a weighted average giving disambiguation score more weight than distance score. The best compromise keyboard was selected as abcd efghi jklmn opqrs tuv wxyz which is very high ranking in disambiguation score and the highest distance scored keyboard on the plateau in Figure 3 (this keyboard is shown in red (top centre)). For reference the traditional phone keyboard is shown in orange in Figure 3 (top left) - showing our 6-letter-key layout performs very close to the 8-letter-key phone layout in terms of raw ambiguity of layout. However, as discussed above, prediction technology has improved considerably since physical phone predictive text so we expect much higher prediction accuracy in practice.
Initial Implementation
Building on the OpenAdaptxt [7] framework provided us with a powerful disambiguation engine that gives contextually based word suggestions, word completion and next word prediction. Our implementation was built
Figure 4: Interaction s to enter Hav time. re tap while represents a right swipe. using smart has a smart For ou 2 of o and co contex using our in enteri
Preli
We co enteri prelim the lay layout quite two ca watch In nee found watch same to be tappin using vibrat recogn users with a before sequence ve a good presents a left to OpenAdaptxt runn tphone paired to a 30 x 25 mm scree tphone, where the ur initial prototype ur design from ab ompletion gestu xt of their entries OpenAdaptxt Eng itial implementatio ng a short phraseminary User St
onducted prelimina ng standard short minary, our users dyout and had little t. They also found natural and found ategories of issues
and screen layou ed of careful inves some problems w appears unrespo area. The watch u
overly sensitive to ng, registering the
the in-build gestu ion as tactile feed nised - this appea when wrist-worn a phone. We will co e further studies. ning on a Sony Xp Sony SmartWatc en linked by Bluet bulk of processing e we implemented bove along with the
res. As users type was used to steer lish dictionaries. F on and Figure 4 a
.
tudies
ary user studies w t phrases. While cl did appeared to qu e trouble finding le the word complet prediction quality s were raised: sen
t design. stigation before a l with sensitivity. In nsive to fast repea used in the studies o movements of th se as swipes inste ure events. Finally, back on taps and rs to be harder to rather than hand-onduct tests on th peria Android h 2. This watch ooth to the g is done. elements 1 and e backspace ed, the previous r disambiguation Figure 1 shows
storyboard of
ith four users early
uickly adapt to etters in the new
tion gesture y high. However, sitivity of the arger study we particular the at taps on the s also appears he finger while ead of taps when
, we give a brief swipes being notice for some -held as normal hese issues The curren - this need version. G bar may al The space ambiguous indication users were than the la swap back watermark application
Discussio
In this wor optimised that maint disambigua distance lo that users the predict interaction investigatio We are now smartwatc have outlin extensive interesting completion OpenAdap word-by-w language c these in thnt layout has a fair ds to be vertically
iving some visual lso help, as this w between the uppe s key also needs t
given that the gro e initially trying to arge ambiguous bu k to (or perhaps sh k) the context of te n will also be inves
on and Future
rk-in-progress we layout for ambigu tains alphabetic so ation quality and ow. In our prelimin
can adapt easily t tion quality high, b n and visual proble
on.
w developing our ch text entry meth ned here. Once co user studies to an g opportunity is to n and next word s ptxt - these can pro word basis for regu
constructions. How he tight space on t
rly narrow central expanded in the r indication that thi was not obvious on er and lower letter o be reduced and oup forms a single tap the actual let uttons. An ability t how in miniature /
ext entry from the stigated.
Directions
have introduced a uous smartwatch t
orting while attaini keeping finger mo nary studies we ha to the key layout a but we also found ems that need furt
code into a fully f hod following the c omplete we will con nalyse potential sp
more fully exploit uggestion features ovide very fast en ularly used phrase wever, adding con the watch will be c
"space bar" revised s is a space n first use. rs on each more button, as tters rather to quickly / e original an ext entry ing high ovement ave seen and found some ther unctional criteria we nduct peed. One t the word s of try on a s or trols for challenging.
Acknowledgements
Thanks to our test users and to Chaitanya Krishna and Srinivas Chintagunta of KeyPoint Technologies for their help building upon OpenAdaptxt.
References
[1] Bellman, T., and MacKenzie, I. S. A probabilistic character layout strategy for mobile text entry. In
Proc. Graphics Interface '98, CIPS, 1998.
[2] Bi, X., Smith, B. A. and Zhai, S. Quasi-qwerty soft keyboard optimization. In Proc. CHI 2010, ACM, 2010.
[3] Dunlop, M. D. Watch-Top Text-Entry: Can Phone-Style Predictive Text-Entry Work With Only 5 Buttons? In Proc. MobileHCI 04, Springer LNCS, 2004.
[4] __ and Crossan, A. Predictive text entry methods for mobile phones. Personal Technologies, 4(2), 2000.
[5] __ and Levine, J. Multidimensional Pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In Proc. CHI '12, ACM, 2012.
[6] __ and Masters, M. M. Investigating five key predictive text entry with combined distance and keystroke modelling. Personal Ubiquitous
Computing, 12(8), 2008.
[7] __, Montaparti, S., Dona, P., Durga, N. and Meo, R. D. OpenAdaptxt: An Open Source Enabling
Technology for High Quality Text Entry. In Proc. CHI Workshop on Designing & Evaluating Text Entry, 2012.
[8] Goldberg, D. and Richardson, C. Touch-typing with a stylus. In Proc. INTERCHI93, ACM, 1993. [9] Gong, J. and Tarasewich, P. Alphabetically
constrained keypad designs for text entry on mobile devices. In Proc. CHI '05, ACM, 2005.
[10] Grover, D. L., King, M. T. and Kushler, C. A.
Reduced keyboard disambiguating computer, Tegic Communications, Inc., Patent US5818437, 1998. [11] Hasselgren, J., Montnemery, E., Nugues, P. and
Svensson, M. HMS: A Predictive Text Entry Method Using Bigrams. In Proc. Workshop on Language Modeling for Text Entry Methods at EACL, 2003. [12] James, C. L. and Reischel, K. M. Text input for
mobile devices: comparing model prediction to actual performance. In Proc. CHI'01, ACM, 2001. [13] Kienzle, W. and Hinckley, K. Writing handwritten
messages on a small touchscreen. In Proc. MobileHCI'13, ACM, 2013.
[14] Kristensson, P. O. and Zhai, S. Learning shape writing by game playing. In Proc. CHI '07 Extended Abstracts, ACM, 2007.
[15] Lyons, K. M., Starner, T. E., Plaisted, D., Fusia, J. G., Lyons, A., Drew, A. and Looney, E. W. Twiddler Typing: One-Handed Chording Text Entry for Mobile Phones. Georgia Institute of Technology, 2003. [16] MacKenzie, I. S., Nonnecke, R. B., McQueen, C.,
Riddersma, S. and Meltz, M. A comparison of three methods of character entry on pen-based
computers. In Proc. Factors & Ergonomics Soc. 38th Meeting, 1994.
[17] Oney, S., Harrison, C., Ogan, A., and Wiese, J. ZoomBoard: a diminutive qwerty soft keyboard using iterative zooming for ultra-small devices.
Proc. CHI '13, ACM, 2013.
[18] Oulasvirta, A., Reichel, A., Li, W., Zhang, Y., Bachynskyi, M., Vertanen, K. and Kristensson, P. O. Improving two-thumb text entry on touchscreen devices. In Proc. CHI '13, ACM, 2013.
[19] Wobbrock, J. O., Myers, B. A. and Kembel, J. A. EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion. In Proc. UIST'03, ACM, 2003.
[20] Zhai, S. and Kristensson, P. Shorthand writing on stylus keyboard. In Proc. CHI '03, ACM, 2003.