With the Online Experience Index, we aim to identify which ecommerce
sites are leading the way in providing a powerful customer experience.
In May we reviewed appliance e-commerce sites and found variation
in the levels of usability between the sites, as well as identifying some
recurring trends and issues.
Following on from our review of appliance retail websites using our
Online Experience Index, this report reviews the websites of five mobile
phone providers: Three, EE, O2, Virgin Mobile and Vodafone. Reviews
were performed in the week of 4th November 2014.
In addition to the main Index, we have also included scores for
accessibility, and persuasion, emotion and trust (PET) techniques, to
explore a broader scope of the online experience.
Online Experience Index
Headline findings
• Most websites presented their brand and purpose well, although there was room for improvement in
navigation around the website.
• As expected for this sector which needs personal and banking information to purchase, the checkout process was lengthy and websites varied in how well they communicated what information was needed and why. • Filters were generally well implemented for selecting
phones, featuring a comprehensive range of criteria. • Most websites offered a search function with search suggestions but there was room for improvement in presentation of search results.
• Use of PET techniques was low, with only two websites implementing several techniques in areas of social proof and trust.
Methodology
This is the methodology we applied in order to gain a robust index of e-commerce sites:
In order to score each site’s overall experience rating, expert UX professionals assessed the sites in the context of a core user journey of browsing for a new mobile phone. To replicate a natural user experience, the examiners conducted the review whilst browsing for a new mobile phone and contract plan that included at least 2Gb data allowance, then continuing through checkout as a new online customer.
Sites were rated on over 120 key touchpoints, designed to address multiple aspects of the user journey, from
homepage through to checkout. The touchpoints were systematically weighted to reflect their relative importance. From this, individual ratings for each facet, as well as overall usability scores for each site, were determined, providing us with a comprehensive picture of the online experience across the four sites.
An extensive inventory of techniques was considered when scoring the sites for their use of persuasion, emotion and trust. These are discussed in more detail in their respective sections.
The Index, by facet
Taking each facet of the user journey individually, we found variation in the usability of each website. This
variation is apparent when looking at the individual ratings. The following diagrams illustrate the best and worst
individual ratings by facet of the user journey.
Homepage
Most mobile providers had a “store” homepage that communicated their brand and purpose with engaging content. All sites offered a wide range of products and services, each competing for space on the homepage. Virgin best communicated website purpose with each of their products displayed as sections of the main
navigation.
Navigation
Most sites implemented navigation well, with the top-level menu maintaining a consistent position throughout the user journey. Moving forward in the user journey, most sites also provided breadcrumbs and clearly titled pages to support users knowing where they were within the site. On Three, the store navigation is lost once the user leaves the homepage and the user journey jumps around the website which may be confusing for users.
Search
A search function was available on four out of the five websites but missing from Virgin. When included it was located in a clear and consistent position throughout and sites offered search suggestions. However, on Three, these were absent from searches originating on the search page. In general the search functions covered the whole website, however this returned a large number of results and the presentation of search results was generally poor
compared to other sectors. Sites did not provide
information on the order of results or enable users to refine or reorder them and few reiterated search terms.
Filters
Most websites performed highly when it came to filtering options of products, with O2 including additional ability to filter on phone models.
Observed:
Three, EE, O2, Virgin Mobile and Vodafone
Product page
The process of buying a mobile phone differs from other sectors because it is a composite purchase of mobile phone and network plan. Three websites, Three, Virgin and EE, enabled users to select both phone and network plan from a single product page which reduced the steps needed, however, this increased the complexity of user journey through the page. Vodafone and O2 split the steps onto separate pages which simplified the user journey but this meant that users could not see the full cost of the package as they selected the phone.
Checkout Flow and Layout
The checkout flow for mobile phone is longer than other sectors because network providers gather personal information to perform a credit check and generally require payment upfront in addition to setting up a direct debit. EE provide clear guidance at the start of the
checkout process to explain what information is needed and why. While other websites included some of this information, they did not draw attention to it which many mean that users miss it. O2, Three and Virgin provide a progress bar through the checkout process that allowed the users to see main stages of the journey.
PET – persuasion, emotion & trust
We also looked out for ways in which the sites had incorporated persuasion, emotion and trust (PET)techniques into the user journey, based on an extensive list which covered areas such as social proof, salience and gamification.
Unlike the main OEI scores, PET scores are not given as a percentage; instead, the scores refer to how many instances of PET are found on each site. This is because a score of 100% would not necessarily be a good thing, as too many PET techniques can be overwhelming and distract from the purpose of the site. Similarly, PET
techniques can be applied inappropriately, in which case sites were deducted one point for each instance.
The diagram below shows the overall PET scores for each website, along with their strongest area. The area shaded orange identifies the range of scores which are likely to be effective whilst not overwhelming the user.
The scores ranged from 8 to 17 which indicates a low uptake of use of PET techniques in this sector. O2 made the most use of social proof by including reviews, ratings, testimonials and social media links on the product pages.
PET Scores
1. Appliances Online: 39.5 2. Currys: 31 3. Argos: 30 4. Appliances Direct: 27
EE made the most use of trust techniques by clearly explaining the use of personal information. Compared to other sectors, mobile phone providers make little use of PET techniques, indicating opportunities to improve. For example they could highlight scarcity with ‘popular’ or ‘low-stock’ flags on phones, utilise framing techniques to highlight particular network plans, and increase salience by offering complimentary products such as phone cases on the product pages or in the basket.
Accessibility
Some points within the main Index also identify potential accessibility issues. This score is not intended to provide any in-depth assessment of accessibility, but gives a general idea of how well these sites are providing an accessible experience.
Some items from the OEI identified as being important in terms of accessibility were fully implemented by all four sites. For example, in the checkout process long forms are broken down into manageable stages and most sites included alt tags on images.
On the other hand, there are aspects which sites need to improve on, such as providing clear and concise product descriptions on product pages and presenting the large amount of information needed in a clear structured way on the page.
Accessibility
1. O2 74% 2. Three 67% 3. Virgin 64% 4. EE 58% 5. Vodafone 59%The Index visualisations
The following diagrams draw together the individual ratings to visualise, by mobile phone provider, the user experience across all facets of the user journey.
Three
Users may become lost in the website when they browse a specific phone and are taken out of the “Store” section into the “Discover” section of the website
Three
EE
Users are given clear information about the checkout process and what they will need in order to complete it.
O2
The product page provides clear headline information about the phone, ability to view different images and change colour and includes several forms of social proof with rating, reviews and a testimonial.
EE
Total index score: 81%
O2
Virgin
The basket page is a different style from the rest of the website and does not provide the user any information about the checkout process. Further, the user is
automatically opted-in to the insurance and may be confused because the quantity fields are not functional.
Vodafone
When choosing a monthly plan, users are able to compare headline details of the plans but details of extras are complex and confusing.
Virgin
Total index score: 74%
Vodafone
In Summary
In this, our third application of the Online Experience Index, we’ve continued to score sites across usability, accessibility and factors around persuasion, emotion and trust.
In the next quarter, we’ll be addressing the online experience of furniture retailers.
For further information around how we’ve
constructed the Index, and specifically these new factors, please don’t hesitate to contact us – [email protected] or on 0113 350 8155.
