Chapter 8 Evaluation Results
8.2 Heuristic Evaluation
8.2.1 Additional Comments from the Heuristic Evaluation
In addition to the critical examination using heuristics, the evaluators made additional comments. An informal focus group was also formed, using three of the experts, to provide insightful feedback. The complete list of all expert comments and issues raised in the informal focus group discussion can also be found in Appendix C
(I: page 211). Below is a summary of the key points made by the experts, relating to each heuristic.
Flexibility and Efficiency of Use (the system should be able to deliver its functionality – i.e. the presented links are rightly functional and acceptable – either to experienced, or inexperienced users, and allow users to perform their task)
• The prototype was common for Web users. The users could therefore use the system to perform their task without great difficulty.
• The system provided more functionality and the tools were user-friendly. They reacted immediately with input. However, more clarification about how to use the system would be essential and useful for users.
Easy to Comprehend (the user should find it easy to understand the interaction with the system i.e. interacting with the links interfaces is easy to understand)
• Personalised Links Assistant was a very good idea and easy to use. Inquiry Assistant interface was a good notion but the presentation of results was less clear than it could have been. Definition of relationship types for the domain ontology was unclear to every user. Occasionally, the given links from the
Inquiry Links tool were not easy to understand.
• Appearance and disappearance of links definitely satisfy users.
Easy to Remember (the user should not have to remember instructions in order to interact with the system)
• It was easy to remember, apart from the definition of relationship types for the
Inquiry Assistant interface.
• The proposed concept was simply implemented. The design of the system was not too complex. The assistant tools were designed and placed on top of the top navigation menu. Users could remember these tools and where to find them. The “Select Text” button at the bottom of each page could warn users that there was a help function to assist users in finding more explanation when they wanted one. However, users needed to learn how to use the system before using it.
Pleasant to Use (the system should provide user-friendly interfaceand the user should be enjoyable interacting with the system )
• There should be a “Back” button to point back to the search’s result tree from the Inquiry Links tool (or any previous page about what a mistake was made). • The design of the “Select Text” button in each page could be made more
flexible with the design of the left mouse click.
User Control and Freedom (the user should be able to choose the system functions (i.e. links presentation and personalisation) and have control and freedom in interacting with the system)
• It was good that the system allowed users to select which dimensions of links to appear and it saved time for users to be able to get rid of irrelevant contents (links).
• The system was preferable to users because it provided functions such as
Personalised Links Assistant to present users with something users could decide to see at a time. “No link” option could make the experienced users or expert users with a non-link insertion version. “All links” selection could make inexperienced users see all presenting links.
• With the Personalised Links Assistant tool, one expert expressed that he felt in control (except when the system crashed).
Few Errors (the system should be error free or generate few errors i.e. the system should deliver links rightly according to its function and interface)
• Links all seemed to be relevant. However, sometimes the system crashed while browsing the documents, which resulted in the invisibility of links when there were supposed to be some links displayed.
• There were few errors from the system.
Consistency (the use of language and format of the system should be consistent)
• The system was consistent. All pages used same format, position, etc. However, some technical terms should be replaced, as they were unknown to some inexperienced users.
• Follow Links failed to find a link on a keyword such as “dough” but one existed.
Aesthetic and Minimalist Design (the system should provide a modest design and not contain irrelevant information)
• The design of the system was good and it was easy to handle and follow. The reaction time was short. The Personalised Links Assistant was good. The results from Inquiry Assistant in the textual form were a bit difficult to use and see. It would be easier to see and read if the results were displayed in the visual graph. In addition, the button for the Follow Links function should be placed with the other assistant tools at the top of the navigation menu.
• Linking in body was good but still I found many keyword links repeating themselves e.g. “dough”.
• It would be beneficial to look at some text processing techniques to parse the content before adding links because there was still some irrelevant links e.g. “rope” in a word “property”.
Match between the System and the Real World (the system should speak user’s language rather than system-oriented terms)
• It might be difficult for some users to understand how the Inquiry Links and the
Follow Links tool would help in their navigation. Maybe more background information was needed.
• The Inquiry assistant interface failed, otherwise it appeared fine.
•
It would be better to replace some technical wording in relationship types with simpler meanings.Other comments
• The design of the assistant tools could be improved to make it more usable. For instance, it would be useful to know which linkbase the links were appearing from, or which dimension the word came from e.g. different colour per linkbase.
• An issue such as giving too much assistance could take time for users to configure the setting that they want, needs to be taken into consideration. This is
because it could cause users to discontinue using the system and the tools provided.