3. Methodology
6.10. Evaluation, Third Iteration
The website is presented to the stakeholders who originally participated in this user-centred design process. Through a questionnaire, their opinion on the website is obtained. Feedback and recommendations are provided to the company as possible adjustments in future website development. First, the structure of the questionnaire is explained, followed by the results obtained. Then, the researcher reflects on the usability and accessibility of the developed website. At the end of this section, the addressed facilitators are highlighted.
6.10.1. Questionnaire Structure
The questionnaire consists of five questions per webpage. Each page receives the same question. The stakeholders are asked to evaluate the homepage, research page, pricing page, about us page, FAQ page and contact page. The questionnaire is in Dutch and is included as Appendix U. The translated questions are:
1. The page is clear-structured (linear scale 1-5, required question)
2. The page contains relevant information (linear scale 1-5, required question) 3. The page appeals to me (linear scale 1-5, required question)
4. Are there items you miss or can be improved in the future? (open question, optional) 5. Additional comments (open question, optional)
6.10.2. Questionnaire Results
Three of the four original interviewees answered the questionnaire. The interviewees comment on some technical difficulties found within the website. These are communicated to the developers, but not further discussed in this thesis. Each webpage is discussed describing the opinion of the interviewees obtained through the questionnaire.
The homepage, or landing page, is generally clear-structured and contains relevant information. Two interviewees respond that the page appeals to them, and one interviewee thinks this can be improved. They mention that the bottom part of the page contains a lot of text, making it difficult to scroll it quickly and scan through the information. Moreover, the icons presented under “How Reducept works” seem clickable, but are not. When continuing to read the page it makes more sense, but this causes confusion when the interviewee first arrives on the webpage.
The opinions of the interviewees on the research page are very divided: where two indicate the research page is clear-structured, contains relevant information and is appealing, one interviewee rates all these aspects low. They comment there is too much information on this webpage.
The pricing page is rated well on being clear-structured, but one interviewee indicates the information and the appeal of this page can be improved. First, the interviewees miss information on how much the actual VR headset costs if purchased through Reducept. Second, the terms used on the page can be confusing: how do practices know if they are small, medium or large? Moreover, the reimbursement nog only seems for healthcare organisations, whereas it also mentions reimbursement for patients. This needs to be explicitly stated.
The about us page is rated well on being clear-structured, containing relevant information and its appeal to the interviewees. Other than technical issues, this page did not receive suggestions for improvements.
The FAQ is also rated well on the three first questions. One interviewee does not see the different categories presented above the FAQ. For them, the questions are all opened and, therefore, the overview is lost. This could be due to the browser, but is communicated to the developers.
The contact page is rated diversely, with overall good ratings. However, one interviewee mentions that the text of the page runs through each other, possibly the result of the browser they are using. Nonetheless, this needs to be thoroughly examined.
In general, the pages were rated well. There are some technical errors that need to be fixed. In order to gain a better understanding of the improvements necessary for this website, it should be tested with a larger test group with a diverse background.
6.10.3. Usability Evaluation, Third Iteration
For the third usability evaluation, the researcher checks the usability requirements of Section 6.2.2. Only the requirements that are not met are stated below:
1. Links are identifiable as links (mentioned, underlined, different colour)* 18. Colour is not the only indicator of cue*
19. Contrast between colour is sufficient (WCAG 2.1. standards)* 23. Textual description of video provided***
These usability requirements are also equal to accessibility requirements that are not met, as described in the following Section 6.10.4.. If the links in the header would be underlined or otherwise indicated as links or clickable items, the first two usability requirements can be met. The contrast of color needs to be increased for the buttons. Moreover, a small textual description of the video should be provided. The video contains the same information as the infographic on the homepage, but the user cannot make this connection since the video and infographic are on separate webpages.
6.10.4. Accessibility Evaluation, Third Iteration
The list of requirements of Appendix T should be implemented in the website. The researcher checks all these requirements if they are applicable to the website. The results of the accessibility check can be found in Table 7.
Accessibility Guidelines Not MetMet/ Explanation
Info and relationships programmed (A) Not Met Buttons have the role of link Meaningful sequence programmatically determined
(A) Not Met Not all headers are programmatically determined Information not only provided by sensory
characteristics (e.g. color, shape, etc.) (A) Met All information is supported with multiple characteristics Orientation: both landscape and portrait compatible
(AA) Met Both landscape and portrait mode is supported
Contrast: minimum 4.5:1 (AA) Not Met The buttons’ text contrast is not
high enough with the background Resize text: able to resize text up to 200 percent (AA) Met Text can be increased without
information being lost Reflow: vertical or horizontal scrolling (AA) Met The user never has to use
horizontal scrolling on the website
Non-text contrast: 3:1 (AA) Not Met The icons in the button’s contrast
is not high enough
Text spacing: programmatically determined (AA) Met There is enough space between text lines
Accessibility Guidelines
Page titled: every page has a title that describes
purpose (A) Met
Titles are descriptive Multiple ways: more than one way to locate a
webpage within a set of webpages (AA) Not Met
As stated in Section 6.9.1. Language of parts: programmatically determined (A) Met Language determined in code Consistent navigation: consistent order of menus (AA) Met Menu’s are repeated in the same
order on multiple webpages Consistent identification: same functionality has same
name (AA) Not Met
As stated in Section 6.5.1. Parsing: end tags programmatically determined (A) Met Correct usage of tags Name, role, value: programmatically determined (A) Not Met Expanded texts are not
programmatically determined Non-text content: all non-text content has an
alternative text (A) Not Met Alt text provided does not describe media content Video: text alternative available (A) Not Met Text alternative not available Images of text: images of text do not convey
important information (AA) Met
Images do not contain text Color is not the only visual means of conveying
information (A) Not Met
The header links are only indicated through color
Link purpose: link describes purpose (A) Not Met Target not indicated Identify input purpose programmatically determined
(AA) Not Met
Not explained what is the input purpose in correct language Label in name: name is text visually (A) Not Met Labels different language Error identification: automatically detected (A) Met Errors are detected
Labels or instructions: provided (A) Met Labels are provided
Error suggestion: suggestions for corrections made
(AA) Met E-mail suggestions are made when invalid e-mail is entered Status messages: programmatically determined (AA) Met Message when mail has been send Keyboard: operable through keyboard interface (A) Not Met Search function is not operable
through keyboard No keyboard trap: switching focus through keyboard
possible (A) Met No keyboard trap
Bypass blocks: mechanism available to bypass blocks
or repeated content (A) Met Skip to content integrated
Focus on order: focus must be in logical order of
meaningful sequences (A) Not Met Logical order (left to right) has not been followed consistently Focus visible: focus through keyboard is visible (AA) Met Focus is visible
On focus: receiving focus does not initiate change (A) Met No actions triggered Met/
Not Met Explanation
16 out of 33 accessibility requirements are not currently met in this website, of which 10 are level A requirements. Therefore, this website is not accessible. The website needs to be iterated again in order to implement the changes necessary to make this website accessible. Six of these requirements are design related, of which two were conscious choices. However, alternatives or compromises should be considered if this website will be fully accessible. The other ten requirements need to be altered in the code of the website and were not detectable prior to this evaluation.
6.10.5. Facilitators Addressed on Website
As described in Section 5.4., creating and endorsing facilitators can help the implementation of VR E- health technologies in healthcare institutions. It is therefore important that these facilitators are addressed and highlighted in the communication towards practitioners. Therefore, as much facilitators as possible are integrated in the content of the website. In Table 8, an overview of the included facilitators is provided.
Facilitator Implemented Location
Evolving of Technology
No Some references made in the whitepaper, but not explicit
Training Yes Infographic, FAQ
Technical Support Yes Infographic, FAQ
Organisational Support n.a. n.a.
Knowledge Transfers (KT) Yes Infographic, FAQ
Better Treatment Yes Infographic, video, FAQ
Independence of Patients Yes Infographic, video, FAQ
Motivation of Patient Yes Infographic, video, FAQ
Participation in Research Yes Research
Recommendations Yes Experiences, FAQ
Novelty Yes Infographic, video, FAQ
Collaboration in Design Yes Whitepaper
Stronger Evidence Base Yes Whitepaper, Further Development
with Partners
Affordable Hardware/Licenses No Prices are known but not cheap
Community No Future project
Raise Awareness Yes Website
Compatibility of Content Yes Infographic, video, FAQ
Background Company Yes About us, Whitepaper
Gather Data Yes Infographic, FAQ
Decrease of Medicine Yes Whitepaper
Facilitator
Table 8 shows that many facilitators are addressed. The ‘Evolvement of Technology’ can be highlighted more on the website. The prices of the application are known, but they are not cheap. Therefore, the facilitator of the ‘Affordable Hardware/Licenses’ are not met, but this facilitator is also rated as least cost-efficient. Therefore, no additional effort is made to include this facilitator.
The facilitators ‘Community’ and ‘Smooth Distribution’ are not explicitly addressed on the website. However, establishing a smooth distribution is a current project of RelieVR BV, as they are designing special cases and a starter kit for practitioners. Once this distribution has been established, it is a good idea to mention it explicitly on the website. The company is also working on creating a community. Perhaps, the website could form a medium for this community. This is something that can be explored in future iterations of the website.