Analysis of interview results

We analyzed the interview results in view of the same meaning categories applied to the quantitative data analysis (Figure 26):

1) Competent use (and assessment) of self-learning strategies 2) Definition of learning goal(s)

3) Specification and use of relevant information 4) Assessment of information relevance

Furthermore, I included those aspects in our analysis that the quantitative study failed to answer:

6) Prior CING experience

7) Learners’ neglect of the simple past content pages 8) Learners’ problems with the CING material’s salience 9) Learners’ problems with the CING feedback

10)Problems with the overall CING experience

The complete, categorized details of the interview questions and answers are given in Appendix C6 and an analyzed version in C8. This analysis does not include the interpretation or discussion of the findings. This will be done in chapter 5. It must be noted that I do not understand the interview results to capture the entire reality of the learner behavior in question. My deductions and assumptions in this research are to be seen as tentative with room for adjustment. As Gläser and Laudel (2004) point out, subjects often provide information on parts of the set of analysis categories but not on the entire set. This was also true in my study and it was thus not possible to compare cases in every category.

Comparison of causes

Self-Learning Strategies

In the interviews, we first translated the abstract concepts of learning strategies into learners’ actual learning strategies for a grammar task like the one they had been given for their CING work (e.g., I have to look for information on the Present Perfect)and then asked learners to comment on their learning behavior in the CING. Most of the subjects reported that they normally use grammar books to study grammar rules and complete practice exercises (260406, 270406, 080506, 150506, 110406). Others cited language examples and practice exercises (170206), a combination of rules and examples (190406, 210406), or only examples (200406) as their usual learning material. In every case, subjects followed their usual learning strategies when working in the CING. This was supported by the navigational logs as well as the interviews.

These results show that existing learner strategies developed with traditional learning tools such as books can be applied in the CING. Some strategies, however, such as “note taking” (mentioned by 170206, 080506, 110406) are not accommodated by the CING (which

does not include a “notepad” function). It does, however, leave learners free to decide if they want to skip the topics they already know (260406, 080506) or to follow the given content structure until everything has been covered (170206).

One aspect of a successful learning strategy is that learners are able to overcome difficulties they encounter in their learning. Difficulties with the CING that our subjects directly reported included the unclear feedback system (170206, 270406) incomprehensible vocabulary (210406), inability to locate the back button for navigation (080506), failure to find their way out of a particular content area (120406), and the inability to read a computer screen (150506).

Except for 170206, 270406, and 120406, all subjects autonomously found a solution to their problems. Two of the subjects (170206, 270406) unable to solve their difficulties on their own were in the “strong” group of navigators.

Definition of Learning Goal

Before their CING work, learners were provided with a clearly defined learning task, which they were then asked to translate into learning goals. All except one of the subjects (170206) stated their learning goal to be the preparation for a test on the simple past and present perfect, but they differed in their understanding of their larger learning goals. Some considered it important to improve their knowledge as well as learn to apply the two grammar topics correctly (260406, 120406). Others, however, focused their learning on the improvement of knowledge of the task topics only (270406, 190406) or merely on the application of the grammar (080506, 150506) in the grammar test (200406) without aiming to comprehend the theory. Yet another approach was described by interviewees 170206 and 210406 who reported being more focused on their individual learning needs in that they integrated their own problem areas with the grammar into the test preparation. Most subjects considered the present perfect more problematic than the simple past. One subject reported that “Continuous” pages as well as “Perfect” pages were important for his test preparation, but the continuous is neither related to the present perfect nor the simple past.

Most learners’ CING behavior exhibited a CING sequence navigation from the tense/aspect topic area to the present perfect content pages (170206, 150506, 200406, 210406), with some also including the intro page (170206, 150506). Subjects' learning goals were very task-focused, except for the subject who considered “Continuous” pages relevant to the learning task (110406).

Specification and use of Relevant Information

The specification of relevant information (information specification IS) involves learners’ learning goal, the given learning task, as well as learners’ knowledge of their own learning needs on the topic.

Many subjects considered their own knowledge level on the simple past and present perfect to be average (170206, 080506, 110406, 120406, 200406, 210406), while a few considered their knowledge level to be between average and good or good (260406, 150506) or between average and weak or weak (190406, 270406).

The correct application of the present perfect and simple past was considered to be particularly important by most subjects (150506, 110406, 200406, 210406), with the simple past being the easier of the two topics (170206, 080506).

Many learners considered the content page Use of the Perfect (270406, 080506, 150506, 190406, 200406, 210406) to be the most relevant to their learning; followed by pages on the present perfect (170206, 150506, 110406).

Subjects evidently followed the existing CING structure on the Content Menu to reach the content they were looking for. Overall, the navigational data confirmed that the subjects followed the existing CING structure on the Content Menu to reach the content they were looking for. The simple past content pages, however, were not easily found by subjects as they did not expect to find them under the content title Continuous Forms (120406, 210406).

Assessment of relevant information

Learners' assessment of information relevance of the found CING pages took place via the content titles and content (260406, 080506, 110406, 190406, 200406) and the title’s key words (“present perfect” and “simple past”) (170206, 120406, 210406). The content also helped learners (200406, 210406, 120406) to identify irrelevant pages, particularly in cases where they initially thought the page was relevant. The keywords “past” and “perfect” were often correctly considered to be clear indications of page relevance (270406, 150506, 110406, 200406). As noted above, the pages on the present perfect were considered most relevant to the subjects’ task (170206, 260406, 080506, 120406, 190406, 200406), while some learners took a more holistic approach and found relevant information on various pages (270406, 210406), including those covering the past (110406).

Of the page types the CING provides, the majority of subjects selected the Explanation pages (080506, 120406, 190406, 210406) while some combined the Explanation

with the Exercise pages (150506, 110406) or chose only Exercise pages (150506). The interviews revealed evidence that the time in the CING was too short and that learners therefore did not visit certain page types (Discovery [080506] and Exercises [150506]).

Understanding the CING’s Hypertext Structure

In the interviews, some subjects did not correctly assess the task relevance of page titles, some had difficulties finding navigational tools (such as the back button) or tried to click the example of the Content Menu on the tool’s introduction page (170206).

Only one learner (170206) recommended reading the introduction page before proceeding to the Tense/Aspect menu to choose the most relevant pages, while the majority of subjects suggested either the menu (Tense/Aspect) as the first step in the search for relevant pages (260406, 270406, 080506, 150506, 110406, 120406, 190406, 210406) or the Search option (260406, 080506, 150506, 110406, 120406, 190406).

Other navigation aids learners suggested by subjects for CING navigation included the Toplinks (170206, 260406, 080506, 200406), the links on top of each content pages to related close by topic pages (150506) and the back button (270406, 150506, 120406). Subjects also advised skipping irrelevant information (120406), orienting along the CING list of content pages (190406), and ensuring that the page visited is relevant (150506).

Learner problems with the CING

The difficulties that learners have with a learning situation are often related to various aspects of the context. This can involve learners' prior knowledge, learning preference or learning strategies or the learning material, its structure, its type of presentation, or even the time they have available.

The discussion of the reasons for subjects' difficulties with the CING will therefore be discussed in chapter 5 to cater for the need to view the topic from various viewpoints. In the following chapter I will discuss these findings at greater length, and then turn to suggestions for ways to improve the CING in chapter 6.

4.5 Summary and conclusion

This research design was made to produce findings that help to investigate the CING’s success as a learning tool and its usability with first year students of English at the Chemnitz University of Technology.

With the help of the grammar pre and posttest data it was indicated that learners improved their performance on a grammar gapfilltest after having worked in the CING by one point. The majority of learners had an intermediate level of English, had been mainly instructed in the language at school and were more extrinsically motivated than intrinsically to learn and apply the English grammar correctly. Furthermore, they had never used the CING before the research session.

Questionnaire data on learners CING page title knowledge showed that most of the CING Simple Past and Present Perfect content pages listed were comprehended correctly by learners, but that some pages (e.g. Speech Time and Reference Time) were miscomprehended, especially if the titles lacked keywords to both topics. Correlations of learner profile information with title knowledge data showed that learners with better knowledge in English comprehended more of the grammar titles correctly. Nevertheless, the CING content page Simple Past entails clear topic keywords but still visited by only a few subjects regardless of learners’ language level. Finally, the interview sessions helped to explain this finding through learners’ comments that they would have never expected (or were surprised to find) the Simple Past page under the content link Continuous Forms and therefore did not visit the page.

CING experience and subject profile

As basis for our usability investigation our CING experience questionnaire revealed that the majority of subjects negatively experienced autonomous learning with the CING, the metalanguage in the tool’s grammar material (i.e. metalanguage), its page titles as well as the language examples which failed to show grammar structures in a salient way to them. The overall vocabulary in the language examples was experienced negatively by half of the subject group and about a third reported negative experience with the CING’s feedback. Correlations revealed a weak relation between English language level and learners’ experience with the CING’s vocabulary, content page titles and metalanguage. The better the