• No results found

Enhancing critical analysis and problem-solving skills in undergraduate psychology: An evaluation of a collaborative learning and problem-based learning approach

N/A
N/A
Protected

Academic year: 2021

Share "Enhancing critical analysis and problem-solving skills in undergraduate psychology: An evaluation of a collaborative learning and problem-based learning approach"

Copied!
27
0
0

Loading.... (view fulltext now)

Full text

(1)

Karantzas, Gery C., Avery, Michelle R., Macfarlane, Susie, Mussap, Alexander, Tooley, Gregory,Hazelwood, Zoe, & Fitness, Julie (2013) En-hancing critical analysis and problem-solving skills in undergraduate psy-chology : An evaluation of a collaborative learning and problem-based learning approach. Australian Journal of Psychology, 65(1), pp. 38-45. This file was downloaded from: http://eprints.qut.edu.au/61222/

c

Copyright 2013 The Australian Psychological Society

Notice: Changes introduced as a result of publishing processes such as copy-editing and formatting may not be reflected in this document. For a definitive version of this work, please refer to the published source:

(2)

RUNNING HEAD: Critical Analysis and Problem-Solving

Enhancing Critical Analysis and Problem-Solving Skills in Undergraduate Psychology: An Evaluation of a Collaborative Learning and Problem-Based Learning Approach

Gery C. Karantzas1, Michelle R. Avery1, Susie Macfarlane1, Alexander Mussap1, Gregory Tooley1, Zoe Hazelwood2, Julie Fitness3

1

Deakin University, Australia 2

Queensland University of Technology, Australia 3

Macquarie University, Australia

Acknowledgments

The preparation of this manuscript was supported by a grant from the Office of Learning and Teaching (ALTC ID11-2005) and a Strategic Teaching and Learning Grant from Deakin University.

All correspondence concerning this manuscript should be addressed to Gery C. Karantzas, School of Psychology, Deakin University, 221 Burwood Highway, Burwood, Victoria, Australia, 3125. Ph: +61 3 9244 6959; Fax: +61 3 9244 6858;

(3)

Abstract

Critical analysis and problem-solving skills are two graduate attributes that are important in ensuring that graduates are well equipped in working across research and practice settings within the discipline of psychology. Despite the importance of these skills, few psychology undergraduate programs have undertaken any systematic development, implementation and evaluation of curriculum activities to foster these graduate skills. The current study reports on the development and implementation of a tutorial program designed to enhance the critical analysis and problem-solving skills of undergraduate psychology students. Underpinned by collaborative learning and problem-based learning, the tutorial program was administered to 273 third year undergraduate students in psychology. Latent Growth Curve Modelling revealed that students demonstrated a significant linear increase in self-reported critical analysis and problem-solving skills across the tutorial program. The findings suggest that the development of inquiry-based curriculum offer important

opportunities for psychology undergraduates to develop critical analysis and problem-solving skills.

Keywords: collaborative learning, critical analysis, graduate attributes, problem-solving, problem-based learning

(4)

Within the literature on graduate attributes, critical analysis and problem-solving skills have been espoused as two fundamental skills that should be developed in university undergraduate students (Barrie, 2006; Moore, 2004). These skills are thought to enhance graduates’ abilities to make connections between learning and practice (Thomas, 2011), and their capacity to deal with novel and ill-defined problems (Boud & Falchikov, 2006).

Moreover, skills such as critical analysis and problem-solving are important in enhancing students’ ability to think and act as global citizens in an ever-changing world (Cranney & Dunn, 2011; Morris, Cranney, Jeong, & Mellish, this issue).

As a result, critical analysis and problem-solving have become graduate attributes that are endorsed by many higher education institutions and professional accreditation bodies (Barrie, 2006). In psychology critical thinking and problem-solving constitute two of the six core graduate attributes endorsed by the Australian Psychology Accreditation Council (APAC, 2010; Haw, 2011), namely, ‘critical thinking in psychology’ and ‘learning and the application of psychology’.

While critical analysis and problem-solving can be regarded as distinct constructs (Bailin & Seigel, 2003), these concepts are highly related. Some contend that problem-solving may reflect a particular sub-category of critical analysis which is more procedural than epistemological (Hammer & Green, 2011), while others argue the concepts are nested, suggesting that critical thinking is the cognitive process that yields problem-solving and decision-making (Facione & Facione, 1993). Regardless of issues surrounding the conceptualisation of graduate attributes (for reviews see Barrie, 2006; Hammer & Green, 2011), skills such as critical analysis and problem-solving are deemed important in the discipline of psychology. In a study of the key graduate skills underpinning psychology, O’Hare and McGuiness (2004) found that critical thinking (which encompassed critical analysis and problem-solving) emerged as one of the key graduate skills. This is not

(5)

surprising, given that the work of psychologists – whether academic or professional practice – requires carefully evaluating opinions and biases, generating and testing hypotheses to answer perplexing theoretical and applied questions, and applying knowledge to mental health problems experienced at a population or individual level.

Despite the rhetoric surrounding the importance of graduate attributes such as critical analysis and problem-solving in psychology, Cranney et al. (2008) highlight that psychology has been slow to apply education principles regarding the assessment and curriculum

implementation of graduate attributes in undergraduate programs. Yet calls have been made for undergraduate programs to develop and implement evidence-based teaching practices to foster students’ professional development and consolidation of graduate attributes (Dunn, Saville, Baker, & Marek, this issue).

Despite these calls, there exist few studies within psychology that: (1) implement teaching and learning approaches underpinned by sound pedagogy; (2) evaluate the development of students’ graduate attributes progressively over time; and (3) assess the development of graduate attributes using measures other than surface curriculum mapping approaches such as checklists or single response items (Barrie, Smith, & Hughes, 2009).

In an attempt to address these issues, we developed and implemented an innovative tutorial program grounded in existing and emerging principles of teaching and learning with the aim of enhancing the critical analysis and problem-solving skills of undergraduate psychology students. We also assessed students’ change in these skills through the use of a multi-item self-report instrument specifically designed to measure critical analysis and problem-solving skills.

Collaborative learning approaches (CLA) to tutorial programs

Despite the emphasis on graduate attributes by tertiary institutions and professional bodies, undergraduate subjects and programs do not always provide systematic opportunities

(6)

for the development of critical analysis and problem-solving skills. A primary context in which these skills can be cultivated is the tutorial classroom. In this environment, students have the opportunity to work for discrete periods of time in small groups to complete subject-relevant activities and tasks. Nevertheless, studies of small-group activities (whether they are highly structured or less structured such as in problem-based learning) note that students often ‘progress in parallel’ (Dillenbourg, 1999) rather than interacting with group members on tasks. Other studies have found that students de-construct activities into divisible parts so that each group member works on a separate part of the activity. Thus, students often engage in co-operative rather than collaborative learning processes (Dillenbourg, 1999). While the terms co-operative and collaborative learning are often used interchangeably, they are distinct. Co-operative learning relates to small-group work often involving closed-ended tasks with particular answers (Rockwood, 1995). Moreover, the teacher is the central

authority figure in the class. In contrast, collaborative learning takes a social constructionists perspective such that students acquire knowledge through interaction and discussion on a more open-ended, complex task in which the teacher relinquishes their authority and encourages the small groups to drive the learning (Rockwood, 1995).

While traditional tutorials can often include collaborative activities, the efficacy of such tasks is significantly enhanced if learning opportunities are designed with collaborative learning in mind (Blunden, 2001). The collaborative learning approach (CLA) involves the “mutual engagement of participants in a co-ordinated effort to solve the problem together” (Roschelle & Teasley, 1995, p.69). Moreover, socio-cultural and shared cognition approaches to collaborative learning emphasise that collaborative learning opportunities need to be structured such that group members not only work together, but develop a shared

understanding of the problem, and engage in social interaction skills to regulate each other’s understanding and cognitive change toward deeper learning (e.g., Resnick, Levine, &

(7)

Teasdale, 1991). Roschelle and Teasley (1995) derived the concept of the “Joint Learning Space” in which students are asked to explicitly agree on joint goals, methods and solutions to a given problem and to gather and analyse knowledge to establish a shared meaning. In a comprehensive review of the outcomes of CLA, Prince (2004) presents strong evidence for the effectiveness of collaborative learning in increasing academic achievement and skills linked to graduate attributes.

Problem-based learning (PBL) approaches to tutorial programs

Barrie et al. (2009) argue that the development of graduate attributes requires movement away from traditional content-transmission teaching models to inquiry-based learning curricula, pedagogies, and assessment practices. Similarly, we argue that curriculum design that embeds the development and assessment of graduate attributes through inquiry-based approaches, such as problem-inquiry-based learning (PBL), is required.

PBL is regarded as an instructional method of learning in which students are exposed to case-based or project-based tasks that centre on a complex problem that does not have a single correct answer (Hmelo-Silver, 2004). Within this instructional framework, students collaborate in small groups to: (1) identify the key issues pertaining to the problem, and (2) engage in self-directed learning to gather resources and information to assist with the development of strategies to solve the problem. The role of the tutor within this learning approach is to facilitate student learning rather than transmit content knowledge. Therefore, akin to the goals and learning outcomes of CLA, PBL is designed to facilitate (amongst other things) the development of critical analysis and problem-solving skills.

To date, PBL has been widely investigated and is regarded as an effective way to develop graduate attributes such as critical analysis and problem-solving (e.g., Johnson, Johnson, & Smith, 1998; Prince, 2004). Despite the plethora of studies investigating the

(8)

efficacy of PBL (e.g., Prince, 2004) no Australian research has applied and evaluated PBL in the development of graduate attributes in undergraduate psychology.

Research aims

Drawing on CLA and PBL, our aims were to: (1) develop and implement a CLA and PBL-based tutorial program to foster development of critical analysis and problem-solving skills in undergraduate psychology students, and (2) to assess change in students’ self-reported critical and problem-solving skills over the tutorial program. It was hypothesised that students would report significant improvements across the tutorial program in their critical analysis and problem-solving skills.

Method Participants

The sample comprised of 273 third year undergraduate students (229 women, 44 men; M age = 22.16 years, SD = 4.72) from Deakin University, Melbourne, Australia. All students were enrolled in the elective psychology subject – the Social Psychology of Relationships. A total of 78% of students enrolled in this subject participated in the study. Approximately 65% of students were undertaking a major in psychology, 21% were undertaking a major in arts, 7% were undertaking a major in sciences or health sciences, while another 7% were

undertaking a double major (e.g., arts/commerce, arts/law). Students received no financial re-imbursement or course credit for taking part in the study.

Materials

Self-efficacy was assessed at baseline using the 57-item Learning Self-Efficacy Scale (LSES, Zimmerman, Kitsantas, & Campillo, 2005). This measure assesses self-efficacy across five dimensions including reading (11 items, α = .86), study (14 items, α = .79), test preparation (11 items, α = .88), note-taking (12 items, α = .77), and writing (9 items, α = .85]. Items across all subscales were summed to create a total learning self-efficacy score (α = .97).

(9)

Items are rated on an 11-point scale ranging from 0% (definitely cannot do it) to 100% (definitely can do it).

Intrinsic academic motivation was assessed using the intrinsic motivation items of the shortened version of the Academic Motivation Scale (AMS-C 28, Vallerand et al., 1992). AMS-C 28 consists of a total of seven factors of which three factors (12 items) reflect intrinsic motivation (i.e., intrinsic motivation – knowledge [4 items, α = .85], intrinsic

motivation – accomplishment [4 items, α = .84], intrinsic motivation – stimulation [4 items, α = .86]). These items were summed to create a total intrinsic academic motivation score (α = .93).

Learning style was assessed using the 21-item Revised Study Process Questionnaire (R-SPQ-2F, Biggs, Kember, & Leung, 2001). This measure assesses two broad learning styles – namely – surface approach (SA, 10 items, α = .77) and deep approach (DA, 10 items, α = .79) to learning. Items are rated on a 5-point scale ranging from 1 (this item is never or only rarely true of me) to 5 (this item is always or almost always true of me).

Critical analysis and problem-solving skills were measured using a 7-item measure specifically developed for the study (see Appendix 1). For the purposes of this study, two versions of the measure were developed – one to measure students’ general perceptions of their critical analysis and problem-solving skills, and another to measure students’

perceptions of these skills in relation to the tutorial activities. Maximum Likelihood

Exploratory Factor Analysis with oblique rotation of both measures revealed an identical one factor solution. In both versions of the measure, the one factor model explained a total of 55% of the variance. Item loadings in both measures ranged from .60 to .78. Given the unidimensional nature of the measures, scores on items were averaged and summed to create a mean total score at each time point (αs ≥ .88).

(10)

Procedure

Description of tutorial program. Drawing on CLA and PBL, the tutorial program was created to foster the development of critical analysis and problem-solving skills in undergraduate students. The program was inspired by the Choose Your Own Adventure (CYOA) novel series (Chooseco, LLC), in which readers assume the role of the main character of the tale, and are required to make a series of decisions throughout the story – decisions which affect the story outcome. In this same vein, students assume the role of a relationship psychologist and are presented with an unfolding story involving a novel and ill-defined problem pertaining to the “Robertson family”. Students are required to make multiple decisions and choose the best course of action to assist the family at different points of the story. Each option is associated with a different story narrative – thus the decision-making of students influences subsequent aspects of the story.

Each tutorial involves a different case study pertaining to the Robertson family. All case studies are presented to students in a comprehensive tutorial manual – the “CYOA Workbook” that supportsstudents’ tutorial activities across the program. The tutorial program consisted of four two-hour tutorials. The first tutorial served as an introduction that oriented students to the tutorial program including its aims and structure. Four weeks after the introductory tutorial, students took part in three fortnightly tutorials during which time the CYOA tutorial program was implemented. In each of these three tutorials, students were instructed to form groups of five to six, select a group leader and to open their CYOA workbooks and commence reading the background information for the given case study for the week. Students then used their workbook to progress through the case study which ultimately resulted in the development of a case formulation.

The workbook consisted of activities such as reading case notes, reflective

(11)

synthesise ideas, and a case formulation to assist with student’s critical analysis and problem-solving skills. The workbook tasks also directed students to subject materials including lecture notes, learning modules and readings to ensure that the practicum aligned with subject content. Most importantly, to foster collaborative learning, the workbook comprised a series of questions that guided the groups in developing shared meanings, goals and strategies in working through the case study associated with each tutorial.

At the end of the tutorial, each group was required to make a five minute class presentation regarding their case conceptualisation and course of action. Groups were required to provide a justification for their solution and audience members (students and the tutor) were given the opportunity to ask questions of the group.

Data collection. The CYOA tutorial program was administered to all students irrespective of their participation in the study and formed part of their regular tutorial program. The

evaluation of the tutorial program was approved by the University Human Ethics Committee. During the first tutorial (conducted during week 2 of the subject), tutors advertised the study as part of their introductory address to the students. Students who were interested in participating in the study were given a copy of the plain language statement, consent form, and a questionnaire booklet that included demographic questions (e.g., age, gender, course enrolment), measures of learning self-efficacy, learning style, academic motivation, and critical analysis/problem solving skills. Student participants completed the consent form at the conclusion of the first tutorial and submitted these materials to their tutor. Upon receipt of the consent forms and questionnaires, the tutor randomly shuffled the student consent forms and sealed them in an envelope. The questionnaire booklets were then stored in a secure location separate from the consent forms. All questionnaire booklets contained a unique random identification number to ensure that student responses could be de-identified

(12)

immediately after submission of the questionnaires to their tutor. Students were required to note down their number for their records for inclusion on subsequent questionnaires.

At the conclusion of each of the three CYOA tutorials – tutorials two, three, and four – participants again filled in the critical analysis and problem-solving skills questionnaire to track their progress along these graduate attributes. Thus, students who participated in the study completed assessments at four time-points. During the first tutorial (T1), baseline assessments of critical analysis and problem-solving skills as well as the individual difference variables –learning self-efficacy, academic motivation and learning style were collected. The administration of these measures took students approximately 30 minutes to complete.

During tutorials two to four (T2-T4) participants were again administered the critical analysis and problem-solving questionnaire, which took students five minutes to complete.

Results

Approximately 30% of the sample had missing data on the critical analysis and problem-solving skills measure at one of the four time points. Despite this, there was no participant attrition. A missing value analysis was conducted and revealed that the data were missing at random (MAR). The missing data were replaced using multiple imputation. All distributions were univariate normal (absolute skewness falling < 2.0 and kurtosis < 7.0) and multivariate normal (Mardia’s multivariate kurtosis = 7.08, p >.05). The means and standard deviations for all measures are shown in Table 1.

< Insert Table 1 here >

The data were analysed using Latent Growth Curve Modelling (LGCM). LGCM is a form of structural equation modelling that uses repeated assessments of an outcome variable to estimate the trajectory of change over time within and between individuals (Duncan & Duncan, 2004). The development of an LGCM requires that two latent variables are created – one to estimate baseline levels of a given phenomenon termed the intercept – and the other to

(13)

estimate pattern of change in a given phenomenon termed the slope (Duncan & Duncan, 2004). Each of these latent variables is then modelled to load onto the assessments of the outcome variable at each respective time point (see Figure 1). Given that the intercept reflects baseline levels of the outcome variable and is thus considered constant, all intercept loadings are fixed to 1 (see Figure 1). In contrast, the loadings associated with the slope are fixed to values that capture change over time in the outcome variable. In the example shown in Figure 1, linear change is represented by the slope factor loadings, 0, 4, 6, 8, in which, 0 represents baseline, and loadings 4 to 8 reflect weeks post baseline when students were assessed on critical analysis and problem-solving skills after participating in the CYOA tutorials. We tested the model with and without learning self-efficacy, academic intrinsic motivation and learning style (surface approach and deep approach) to determine the effects of these individual difference variables on change in critical analysis and problem-solving skills. Research to date suggests that learning self-efficacy, intrinsic motivation and deep approaches to learning are important predictors of academic performance and skill

acquisition (e.g., Duff, Boyle, Dunleavy, & Ferguson, 2004; Vallerand et al., 1993). Thus, if we were to determine the efficacy of the CYOA tutorial program in enhancing self-reported critical analysis and problem-solving skills, it was important that we account for these individual difference variables. Thus, these individual difference variables were modelled as part of these LGCM by regressing the individual difference variables onto both the intercept and slope variables.

LGCM was estimated using Maximum Likelihood χ2ML. Model fit was evaluated using Hu and Bentler’s (1999) combination approach with Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) ≥ .95, Root Mean Square Error of Approximation (RMSEA) ≤ .05, and Standardised Root Mean Residual (SRMR) ≤ .05 indicative of very good fit. We

(14)

evaluated whether an LGCM with individual difference variables fit the data significantly better than one without by conducting a chi-square difference test (∆χ2).

The LGCM reflecting a linear change in students’ critical analysis and problem-solving skills was of excellent fit χ2(2) = .31, p < .05; CFI = 1.00; TLI = 1.02; AIC = 24.31; RMSEA = .000; SRMR = .000 (model 1). The latent mean variance for the intercept

suggested that students showed significant deviation from the sample mean (Varintercept = .68,

t = 2.19, p < .05), however, no such deviation was found in the slope (Varslope = .01, t = 1.82,

p > .05). This suggests that students varied significantly in their baseline self-reported skill levels on critical analysis and problem-solving, but did not show any discernable deviation in their rates of increase in these self-reported skills (Varslope = .01, t = 1.82, p > .05). The LGCM reflecting linear change with the inclusion of individual difference variables was also of excellent fit χ2(10) = 13.52, p > .05; CFI = .994; TLI = .984; AIC = 81.52; RMSEA = .036; SRMR = .032 (model 2) but not significantly better in fit than the model without the

individual difference variables. Therefore, the individual difference variables did not significantly influence students’ baseline scores or change in critical analysis and problem-solving skills ∆ χ2(8) = 13.21, p > .05, (∆TLI = .036; ∆AIC = 27.21; model 2 – model 1).1

Discussion

By and large, it appeared that our CYOA tutorial program had a positive effect with students reporting an enhancement of their critical analysis and problem-solving. Moreover, our program seemed to achieve shifts in students’ self-reported critical analysis and problem-solving skills over a relatively short period of time – a total of 10 weeks.

Our findings suggest that having students engage in activities that are underpinned by CLA and PBL shows promise in fostering critical analysis and problem-solving skills in undergraduate psychology students. This is an important finding, as our study represents one of only a handful in psychology to evaluate the efficacy of a tutorial program designed to

(15)

enhance students’ critical analysis and problem-solving skills (Haw, 2011; Williams et al., 2003). Furthermore, the current study is the first to our knowledge to evaluate a program explicitly underpinned by CLA and PBL within the discipline of psychology.

While our research is novel, our findings are in line with various studies and reviews in other disciplines that have found support for the use of these teaching and learning

approaches in enhancing students’ graduate attributes (Johnson et al., 1998; Prince, 2004). In particular, it is suggested that these approaches structure the learning environment in a way that it canalises the student to evaluate and test their own assumptions, use logic to unpack the premises associated with arguments, and apply their knowledge and skills in a manner to solve problems (e.g., Hmelo-Silver, 2004).

For instance, drawing on CLA, the CYOA tutorial program requires students to develop a shared understanding of the case study during a given week’s tutorial. In order to develop a shared understanding, individuals must develop awareness regarding their own biases and that of others; as well integrate these perspectives into a coherent representation of the issue – aspects of CLA which pertain to the use of critical analysis. In drawing on the PBL literature (e.g., Hmelo-Silver, 2004; Prince, 2004) we designed the CYOA tutorial program such that students were presented with a problem (i.e., the case study) that was initially somewhat ill-defined. This required students to discernibly choose between resources (i.e., therapist case study notes, transcripts from therapist-client sessions, and psychological test scores) that would assist in developing knowledge about the problem. In doing so, our aim was to ensure that the tutorial case studies provided students an experiential and investigative learning context in which they could: (1) synthesise and evaluate material, (2) determine relevant facts and issues pertaining to the problem, and (3) conceptualise the problem in a manner that would provide a platform for generating a solution – characteristics central to the PBL approach (Hmelo-Silver, 2004).

(16)

Furthermore, as part of our PBL approach, we were mindful of creating case studies that exposed students to novel problems. The novelty of a given problem is regarded

Willingham (2007) as an important facet that distinguishes critical analysis from problem-solving. That is problem-solving does not necessarily require critical thinking if a student has experienced a similar problem in the past. When a problem is deemed familiar, Willingham contends that students will merely apply the solution used in the past to the current problem. Consequently, critical analysis is unlikely to be engaged to any great extent. Providing students with novel case studies as part of each CYOA tutorial was important in facilitating a learning environment in which critical analysis could be practiced.

It is important to point out that while our implementation of PBL captured important facets of this learning approach, the development of the CYOA workbook resulted in a more directive PBL approach than is traditionally associated with this method. PBL often

encompasses students working on an ill-defined problem at their own pace and in a self-directed manner. Within the CYOA tutorial program, we purposefully minimised these aspects of PBL. In many ways, the CYOA tutorial program represented what is sometimes termed ‘hybrid’ PBL (Hossam, 2008). That is, our PBL approach provided a more structured learning experience for students in which students were specifically asked to make decisions, and contingent on these decisions, were provided with limited resources. Furthermore, students were prompted throughout the workbook to incorporate information from lecture notes and readings as part of their case formulation. This structured/directive nature of our PBL approach may have enhanced students’ self-reported critical analysis and problem-solving skills. Research has found that PBL approaches that are more structured enhance students’ learning and graduate attributes to a greater extent than approaches in which problems are less defined (Albanese & Mitchell, 1993; Prince, 2004). These more directive approaches are thought to provide students with sufficient guidance and resources to keep

(17)

them focused on working through the problem, while still ensuring that students develop their own conceptualisation of the problem and engage in decision-making regarding the

examination of resources and in developing solutions (Prince, 2004). Limitations

While our findings suggest that our CYOA tutorial program enhances students’ critical analysis and problem-solving skills, there were some limitations with the current study. Firstly, we assessed students’ self-reported change in critical analysis and problem-solving skills. Future research would need to cross-validate students’ self-reported changes in these graduate attributes with assessment on ability tasks of critical analysis and problem-solving.

Secondly, our tutorial program evaluation did not include a control group. Such an addition to the study design would have been important in determining whether the CYOA tutorial program yields self-reported increases in critical analysis and problem-solving skills that significantly differ from more traditional tutorial programs. Having said this, assigning students who have enrolled in a subject of study into different learning conditions can be deemed problematic and unethical from both a pedagogical and equity and access standpoint. Students enrol in subjects under the assumption that they will receive comparable learning opportunities to all other students. Providing some students with enhanced learning

opportunities relative to others violates this assumption. While a waitlist control design may be viewed as an alternative design option, a waitlist control design is problematic at the subject level. Specifically, subject content has to be delivered to students in a different sequence depending on group assignment to an intervention group and waitlist control. This is often impractical and can violate good pedagogy whereby subject content would no longer bestructured in a manner in which early content scaffolds learning for later material.

(18)

While the omission of a control group limits our ability to establish that changes in students’ self-reported critical analysis and problem-solving skills were entirely due to the program, we reiterate that the critical analysis and problem-solving questionnaire surveyed students’ development of these skills within the tutorial activities. Thus, students’ reports of change were in reference to the tutorial program. Furthermore, our inclusion of various individual difference variables provides us with confidence that changes in critical analysis and problem-solving skills were unlikely to be due to factors such as students’ learning style, learning self-efficacy, and academic intrinsic motivation.

Thirdly, we cannot make conclusions about whether the CLA or PBL aspects of the tutorial contributed equally or differentially to students’ development of critical analysis and problem-solving skills. This is an empirical question that requires examination in future research by manipulating the degree to which these two approaches are implemented as part of tutorial programs.

Conclusion

In closing, our CYOA tutorial program was developed in response to the lack evidence-based teaching practices associated with the development of students’ graduate attributes within the discipline of psychology. As result, we have attended to the calls regarding the application of the Scholarship of Teaching to the field of psychology (Gurung this issue). Our tutorial program evaluation provides support for the notion that designing programs underpinned by teaching and learning approaches such as CLA and PBL can yield important outcomes regarding students’ self-reported development of critical analysis and problem-solving. In light of our findings, we invite others to develop and evaluate similar programs. A concerted effort of this kind can help ensure that students enrolled in the discipline of psychology develop the graduate attributes necessary to effectively meet the challenges of our profession well into the future.

(19)

References

Albanese, M. & Mitchell, S. (1993). Problem-Based Learning: A Review of Literature on Its Outcomes and Implementation Issues. Academic Medicine, 68, 52–81.

Australian Psychology Accreditation Council (APAC). (2010). Rules for accreditation and accreditation standards for psychology courses. Retrieved from

http://www.psychologycouncil.org.au/Assets/Files/APAC_Rules_for_Accreditation_&_ Accreditation_Standards_for_Psychology_Courses_Ver_10_June_2010.pdf

Bailin, S., & Seigel, H. (2003). Critical thinking skills. In N. Blake, P. Smeyers, R. D. Smith, & P. Standish (Eds.), The Blackwell guide to the philosophy of education (pp.180-193). Malden, MA: Blackwell Publishing.

Barrie, S.C. (2006). Understanding what we mean by the generic attributes of graduates. Higher Education, 51, 215–241.doi: 10.1007/s10734-004-6384-7

Barrie, S., Smith. C., & Huges, C. (2009). The national graduate attributes project: Key issues to consider in the renewal of learning and teaching experiences to foster graduate attribute. Retrieved 26 October, 2012, from http://www.olt.gov.au/resources

Biggs, J., Kember, D., & Leung, D.Y.P. (2001). The revised two-factor Study Process

Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133–149. doi: 10.1348/000709901158433

Boud, D., & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment Evaluation in Higher Education, 31, 399-413. doi: 10.1080/02602930600679050 Cranney, J., & Dunn, D.S. (2011). Psychological literacy and the psychologically literate

citizen: New frontiers for a global discipline. In J. Cranney and D. S. Dunn (Eds.), The psychologically literate citizen: Foundations and global perspectives (pp.3-12). New York: Oxford University Press.

(20)

Cranney, J., Provost, S., Katsikitis, M., Martin, F., White, F., & Cohen, L. (2008).Designing a diverse, future-oriented vision for undergraduate psychology in Australia. Retrieved 26 October, 2012, from http://www.olt.gov.au/resources.

Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp.1-19). Oxford: Elsevier.

Duff, A., Boyle, E., Dunleavy, K., & Ferguson, J. (2004). The relationship between

personality, approach to learning and academic performance. Personality and Individual Differences, 36, 1907-1920.

Duncan, T. E., & Duncan, S.C. (2004). An introduction to Latent Growth Curve Modeling. Behavior Therapy, 35, 333-363.

Facione, P.A., & Facione, N.C. (1993). Test Manual: The California critical thinking skills test, Form A and Form B. Millbrae, CA: The California Academic Press.

Hammer, S.J., & Green, W. (2011). Critical thinking in a first year management subject: the relationship between disciplinary learning, academic literacy and learning progression. Higher Education Research & Development, 30, 303-315. doi:

10.1080/07294360.2010.501075

Haw, J. (2011). Improving psychological critical thinking in Australian university students. Australian Journal of Psychology, 63, 150-153. doi: 10.1111/j.1742-9536.2011.00018.x Hossam, H. (2008). The fuzzy world of problem based learning. Medical Teacher, 30,

739-741.

Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Coventional criteria versus new alternatives, Structural Equation Modeling, 6, 1-55.

(21)

Johnson, D.W., Johnson, R.T., & Smith, K.A. (1998). Cooperative learning returns to college: What evidence is there that it works? Change, 30, 26–35. doi:

10.1080/00091389809602629

Moore, T. (2004). The critical thinking debate: how general are general thinking skills? Higher Education Research and Development, 23, 3-18. doi:

10.1080/0729436032000168469

O'Hare, L., & McGuinness, C. (2004). Skills and attributes developed by psychology undergraduates: Ratings by undergraduates, postgraduates, academic psychologists and professional practitioners. Psychology Learning and Teaching, 4, 35-42. doi:

10.2304/plat.2004.4.1.35

Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of Engineering Education, 93, 223-231. Retrieved fromhttp://www.jee.org/

Resnick, L.B., Levine, J.M., & Teasley, S.D. (Eds.) (1991). Perspectives on socially shared cognition. Washington, DC: American Psychological Association.

Rockwood, H.S. (1995). Cooperative and collaborative learning. The National Teaching and Learning Forum, 4, 8-9.

Roschelle, J., & Teasley, S. (1995). The construction of shared knowledge in collaborative problem solving. In O'Malley, C. E., (Ed.), Computer Supported Collaborative Learning (pp. 69-97). Heidelberg: Springer-Verlag.

Thomas, T. (2011). Developing First Year Students' Critical Thinking Skills. Asian Social Science, 7, 26-35. doi: 10.5539/ass.v7n4p26

Vallerand R.J., Pelletier, L.G., Blais, M.R., Briere, N.M., & Senecal, C., Vallieres, E.F. (1992). The Academic Motivation Scale: A measure of intrinsic, extrinsic, and

amotivation in education. Educational and Psychological Measurement, 52, 1003-1017. doi: 10.1177/0013164492052004025.

(22)

Williams, R.L., Oliver, R., Allin, J.L., Winn, B., & Booher, C.S. (2003). Psychological critical thinking as a course predictor and outcome variable. Teaching of Psychology, 30, 220-223. doi: 10.1207/s15328023top3003_04

Zimmerman, B.J., Kitsantas, A., & Campillo, M. (2005). Evaluación de la autoeficacia regulatoria: Una perspectiva social cognitiva. Evaluar, 5, 1-21.

(23)

Table 1.

Descriptive statistics of control variables and critical analysis and problem solving skills (T1-T4)

Variables M SD

Academic intrinsic motivation 4.73 1.08

Learning style – deep approach 31.04 5.72

Learning style – surface approach 25.72 5.74

Learning self-efficacy 6.65 1.24

Critical analysis & problem solving T1 7.43 1.08 Critical analysis & problem solving T2 7.66 1.20 Critical analysis & problem solving T3 7.71 1.35 Critical analysis & problem solving T4 8.00 1.27

(24)

Footnote 1

We also tested two other LGCM to model two forms of non-linear (i.e., polynomial) change – one testing quadratic change and another testing cubic change in students’ critical analysis and problem solving skills. We tested these non-linear forms of change twice – once without the individual difference variables and again with the individual difference variables. Across all model comparisons, the linear model without individual difference variables was of superior fit.

(25)

List of Figures

Figure 1. LGCM representing linear change in student critical analysis and problem-solving skills.

(26)
(27)

Appendix 1.

Items comprising the critical analysis problem-solving self-report measure administered in tutorials 2 to 4.

Items

1. To what extent were you able to develop solutions to learning activities in this week’s tutorial?

2. To what extent were you able to use knowledge from lecture notes and readings and apply it to learning tasks in this week’s tutorial?

3. To what extent were you able to identify opinions, bias and/or distortions in the information you hear or read as part of your studies in in this week’s tutorial?

4. To what extent were you able to identify opinions, bias and/or distortions in what other students said in this week’s tutorial?

5. To what extent were you able to gather and organise information so as to facilitate better or new understanding of this week’s case study?

6. To what extent were you able to offer an informed opinion supported by reason, evidence and/or examples when arguing a point with other students or staff in this week’s tutorial?

7. To what extent were you able to contribute an original interpretation to material you read in this week’s tutorial?

References

Related documents

These  methods  rely  on  sound  estimates  of  p,  q,  and  m.  Unfortunately  it  was  estab‐ lished  that  the  parameter  values  are  not 

But even if returning Mexican families face barriers to school enrollment, should we expect the same association in Mexico between duration of residence and school enrollment

This allows the sqlops method access to the nodes table where it builds a list of tuples containing each entry regarding the information for each node entered into the nodes table..

The facility to enter and store all data items required for management of the school, including information to monitor and support pupils, manage staff, manage school assets

Modify SWAP, as Necessary Establish Direction Establish Expectations Ensure Facility Execution Ensure Sys Execution &amp; Alignment EXECUTE &amp; MODIFY CURRENT YEAR PLAN

For the case of boundary-diffusion-controlled, simulation result is reported in Fig. Here, in contrast to the previous case, when the fast grain-boundary condition was taken

The movement to mobile and digital learning reflects the exploding popularity of mobile devices among consumers and the parallel growth in wireless network services to

With the predefined eight features in time and frequency domain, including Hurst exponent, skewness, kurtosis, largest Lyapunov exponent and frequency-band energy, the artifact