Data Analysis

The data collected for this study were students’ abstract drafts before and after using the tool, their responses to a questionnaire, and transcripts of semi-structured interviews. In the following, the analysis approach is described to show how each research question is answered.

3.6.1.6.1. Students’ abstract drafts before and after using the tool

For students’ abstract drafts before and after using the tool, two steps of analysis were conducted. First, I calculated the word count and sentence count for both drafts. I also manually annotated moves for each sentence, and documented the existence and

correctness of lexical bundles and verb categories. For this step, I analyzed all the first drafts and then the second drafts to avoid confusion.

Then, I compared the analysis from the first step to investigate the changes in word counts, sentence counts, moves, lexical bundles, and verb categories. The word counts and sentence counts in both drafts were compared. As for moves, I recorded the changes in terms of retention, addition, and deletion of sentences and corresponding moves between students’ first and final drafts. I also documented the changes in terms of addition and deletion or lexical bundles and verb categories. Descriptive statistics were calculated. The analysis of students’ abstract drafts before and after using the tool provided evidence to answer RQ 1, 2, 4, and 5.

3.6.1.6.2. Transcripts of semi-structured interviews

Transcripts of the semi-structured interviews were analyzed using Microsoft Word by applying a two-cycle coding method, as suggested by Salda a (2009). To retain originality and avoid inexperienced interpretation, in vivo coding (p. 74) and themeing the data (p. 139) were the two methods used for the first cycle of coding. In vivo coding uses original words from the participants as the codes; and themeing the data groups data into units and provides a code for each unit in a phrase or sentence. After the data were analyzed by the first cycle of coding methods, they were grouped into categories determined after reading all first-cycle codes.

The second cycle of the coding method is focused coding (Salda a, 2009, p. 155). This method takes the categories to a higher level—theme. Applying this method could help the researcher to be open to the data results and group the data into appropriate themes. This means if some information is unexpected, but occurs frequently, then the

researcher would be able to notice and decide whether to present it in the analysis. The analyses of these data provided evidence to answer RQ 3, 4, 5, 6, 7, and 8.

In the end of the analysis, 95 codes were generated. These codes were grouped into 19 themes, and these themes were placed into seven categories (see Table 3.23). The excerpts reported in the results were in Chinese and translated by me. To ensure the correctness of the translation, a second reviewer, who is a native Chinese speaker and currently studying in an American university, read all excerpts presented in this study and discussed changes of the translation with me.

3.6.1.6.3. Responses to a questionnaire

The responses of the six-point Likert-scale questionnaire were saved in an Excel spreadsheet. Descriptive statistics were calculated for each Likert-scale question in the spreadsheet. The analyses of these data answer RQ 1, 2, 3, 7, and 8.

3.6.1.6.4. Summary of data and research questions

In this study, the research questions were answered by relying on three data sources: (1) students’ abstract drafts before and after using the tool, (2) their responses to the questionnaire, and (3) transcripts of semi-structured interviews. The data analysis was presented in previous sections. Table 3.24 provides a summary of the correspondence between research questions and data sources. Additionally, it shows that multiple data sources serve as evidence to answer each research question.

Table 3.23

Categories and themes revealed from the analysis of the interview transcripts (N = 13)

# Categories Themes Number of codes

1 Students tried to write an abstract with a complete schema (16)

Adding sentences 4

Revising sentences 6

Changing the sequence of the sentences

2 Four moves in the end 4 2 Students tried to add lexical bundles

to their abstracts to better express the moves (13)

Reported successfulness 6 Reported

unsuccessfulness

7 3 Students’ attitudes toward the AWE

tool (25)

Positive attitude 13

Positive adjectives to describe the AWE tool

12 4 Willingness to recommend the

AWE tool to others (10)

Willing to recommend 10 5 Whether or not students encountered

difficulties when using the AWE tool (10)

Easy to use 4

Free and publicly available

2 Need for instructions 2 Inconvenience of the loss of Internet 1 Inconvenience of no copy-and-paste function 1 6 The appropriateness of this AWE

tool (9)

Appropriate for their current needs

8 May not use it because

of a missing function

1 7 Writing English abstracts is an

authentic task (12)

For journal article submission

4 For conference

application

1

For Master’s theses 8

3.6.2. Results

Following Chapelle’s CALL evaluation framework (2001) to assess the

effectiveness of the AWE tool by investigating learners’ performance, this study reports the results from six dimensions of the framework, namely Language learning potential,

Table 3.24

Correspondence between research questions and data sources

# Research question Data source Data

type 1 Language Learning Potential – What evidence

suggests the feedback provided by the AWE tool leads to students’ noticing and focusing on the use of lexical bundles?

Students’ abstract drafts before and after using the AWE tool

Students’ responses to the questionnaire

Qual & Quan

2 Language Learning Potential – What evidence suggests the feedback provided by the AWE tool leads to students’ noticing and focusing on the use of verb categories?

Students’ abstract drafts before and after using the AWE tool

Students’ responses to the questionnaire

Qual & Quan

3 Learner Fit – What evidence suggests the AWE tool for assisting abstract writing is appropriate for students with the characteristics of the intended learners?

Students’ responses to the questionnaire Transcripts of semi- structured interviews Qual & Quan 4 Meaning Focus - What evidence indicates the

students focus on the meaning of the moves of abstracts?

Students’ abstract drafts before and after using the AWE tool Transcripts of semi- structured interviews Qual & Quan

5 Meaning Focus - What evidence indicates the students focus on the meaning of lexical bundles?

Students’ abstract drafts before and after using the AWE tool Transcripts of semi- structured interviews Qual & Quan

6 Authenticity – How similar is this task to a task that students would do in their academic

career?

Transcripts of semi- structured interviews

Qual

7 Impact – What evidence suggests that students’ perceptions toward the use of the AWE tool are positive?

Students’ responses to the questionnaire Transcripts of semi- structured interviews Qual & Quan 8 Practicality – What evidence suggests the

AWE tool is sufficiently useable in this context to allow this task to succeed?

Students’ responses to the questionnaire Transcripts of semi- structured interviews Qual & Quan

Learner fit, Meaning focus, Authenticity, Impact, and Practicality. Overall, the results reported from each dimension are considerably positive, indicating that the AWE tool

designed for assisting Taiwanese engineering graduate students’ abstract writing is quite appropriate and useful.

3.6.2.1. Language learning potential: Evidence suggesting the feedback provided by the