Page Not Found error appears when user types URL of uploaded site into web browser

user in every instance (except in the instance of error 2).

Table 3.15. Participant errors during the expert usability test. Error

1. Users were not clear on how to edit a site in Dreamweaver when a mistake was made during site definition

2. Participant uses spaces when naming a new folder within the public_html folder 3. Users do not differentiate between the two different student URL formulas

4. Page Not Found error appears when user types URL of uploaded site into web browser

Table 3.16. Expert users’ error frequency during testing.

Participant EU1 EU2 EU3

Error # Error frequency

1 X X 2 2 X 1 3 X X 2 4 X X 2 Participant error totals 2 3 2

Expert Error Tolerance Analysis

Just as with the novice and intermediate testing, error severity levels are used to quantify errors’ effects on usability. In Table 3.17, I assign a severity level and a recommended design change to address each error and to work toward future error prevention. At this level of testing, there was one Level 3 error, two Level 4 errors, and one computing error. These error levels demonstrate that not only did expert participants identify the fewest errors during testing of all, but the severity of the errors is much more akin to the levels seen from the novice testing (see Table 3.5). Both the expert and novice errors were relatively mild in the aggregate compared to the more profusely severe errors intermediate users identified (see Table 3.11). A comparison of the error tables shows that total number of errors made decreases as the testing level increases. Accordingly, novices committed 12 total errors, intermediates made seven total errors, and experts identified four errors in all. The fact that errors per user are associated with increased task completion time is also evident by comparison—novices committed the most total errors and had the longest average testing time; intermediates committed fewer errors than novices and had a faster average testing time than that group of users; and experts made the fewest errors per user of all and also had the fastest average testing time of all. Furthermore, fewer errors

committed per user suggests a greater error tolerance. Of all the user groups studied, expert users were the most adept at recognizing and correcting mistakes as they worked to complete tasks.

While the recommended design changes for the novice interface mainly require slight modifications related to document design such as altering certain chunks of text to make them more prominent (see Table 3.5), and the design change recommendations for the intermediate interface are mainly related to navigation (see Table 3.11), the design change recommendations

for the expert D8T involve incorporating a greater amount of contextualization to the

documentation. Although Carroll (1990) and his colleagues believe contextualizing information at the novice level leads to a deeper encoding of new knowledge into memory, the user

experiences I observed during testing at different levels lead me to believe the opposite may be true in this situation. The deviation errors users made suggests a connection between the volume of new information presented in a given interface and the tendency of users to deviate from the D8T’s instructions when attempting to complete tasks. For example, 40% of both novice and intermediate users committed deviatory errors during testing (see Table 3.4, error numbers 9 and 10 and Table 3.10, error number 7). If this percentage were extrapolated to all novice users in a classroom setting, it follows that nearly half of students in both user groups (potentially a substantial number) would make the same type of error when presented with the vast amount of new information contained in the tutorial. Additional information, especially additional verbal as opposed to visual information, might only perpetuate the problem of deviation errors, which is why most of the design change recommendations do not specify including much more than what already exists.

The expert users, on the other hand, did not commit any deviatory errors. Expert users were also presented with the least amount of new information of all user groups; their status as experts required the conveyance of a minimal amount of documentation to allow them to successfully complete tasks. However, expert errors 2 and 3 suggests this group of users may benefit from the addition of information pertaining to file management external to Dreamweaver. At the expert level, users should already be familiar with the concepts of URL formulas

(although not necessarily Georgia State student URL formulas) and file management via new folder creation. Including information pertinent to why these concepts are important to

successful website development and maintenance could potentially be advantageous to this group of users who is equipped to meaningfully encode such information by virtue of the prior knowledge they bring to the classroom situation. Adding context to the expert D8T in these areas would address the errors 67% of users made during testing, and so could prevent the same errors from occurring in the future either by allowing to users to more readily recognize and correct errors before they would require assistance or by facilitating a deep understanding such that an error would be avoided completely.

As it stands, the expert D8T is the most error tolerant of all since users made only four total errors of comparatively low severity. The fact that experts identified the fewest errors of all user groups and committed the fewest errors per user is a strong indication that the D8T

functions as it was designed to in terms of error tolerance increasing as the levels increase. However, it is very possible for the interface to be even more error tolerant, and, at best error preventative, if the changes discussed above are incorporated.

Table 3.17. Errors, error severity levels, and recommended change from the expert testing.

Error Error severity

level/type

Recommended change 1. Users were not clear on

how to edit a site in Dreamweaver when a mistake was made during site definition

3 1. Include brief instructions for editing a site in Dreamweaver. Create a link to this information at the top of the expert page

2. Participant uses spaces