High Quality Engineering Education
Research (EER): Key Elements and
Persistent Misconceptions
Ruth A. Streveler Purdue University, West Lafayette, IN USAKeynote address AAEE Melbourne 3 December 2012
Overview
Where are these ideas coming from?
A bit about my experience
As a practitioner of EER
As a teacher of EER
Placing EER in a global context
Essentials and myths
As an engineering education researcher
My primary research centers on engineering students’
understanding difficult concepts in engineering science
What concepts are difficult to understand?
Why are those concepts difficult?
How does one measure conceptual understanding?
How does one create learning environments that
help students better understand these difficult concepts?
As a teacher of EER
With engineering faculty
The beginnings … 2004-‐2006
Rigorous Research in Engineering Education (RREE)
One-‐week summer workshop, year-‐long research
project
Funded by National Science Foundation (NSF),
2004-‐2006
About 150 engineering faculty participated
Goals
Develop faculty knowledge and skills for conducting
EER (especially in theory and research methodology)
Cultivate the development of a Community of
Phase 2 – 2008 to 2011
Create a new generation of professional development
opportunities to increase engineering education research capacity
Create a virtual community of engineering education
researchers
CLEERhub.org (CoLlaboratory for Engineering
Education Researchers)
HUB platform allows for community members to
1. Fundamentals of Engineering Education Research
2. Selecting Conceptual Frameworks
3. Understanding Qualitative Research
4. Fundamentals of Quantitative EER
5. Collaborating with Learning and Social Scientists
*Were recorded and posted on CLEERhub.org
These are listed as “Featured Resources” on the front page and on rotating banners.
Teaching EER PhD students
At Purdue since 2006 one year after the PhD program
began
Have developed and taught courses for beginning
Purdue Engineering Education PhD students
“Typical” PhD student
Has an engineering undergraduate and masters
degree
Summary of my lived experience with
EER – origin of this talk
Worked with over 500 engineering faculty from 3 (now
4!) continents
Worked with about 75 engineering education PhD
students at Purdue
Philosophically engaged in how the new paradigms are
• Groups, centers, departments
• Engineering education societies
• Forums for dissemination
What follows is a sample — it is NOT an exhaustive list!
An emerging global community
Purdue VT
Centers and Departments
Key elements and
misconceptions
Think-‐pair-‐share
Find a partner and introduce yourself
Individually, reflect on these questions:
What misgivings do you personally have about
engineering education research?
What misgivings do you perceive your colleagues
have about engineering education research?
Partners share their thoughts with each other
Some examples shared with the whole group
Reframe “key elements” and “misconceptions” as
polarities that continually interact
First polarity: the
relationship between
practice and research
Mea culpa
Level 0 Teacher
Teach as taught
Level 1 Effective Teacher
Teach using accepted teaching theories and practices
Level 2 Scholarly Teacher
Assesses performance and makes improvements
Level 3 Scholar of Teaching and Learning
Engages in educational experimentation, shares
results
Level 4 Engineering Education Researcher
Conducts educational research, publishes archival papers
Levels of Inquiry
Source: Streveler, R., Borrego, M. and Smith, K.A. 2007. Moving from the
“Scholarship of Teaching and Learning” to “Educational Research:” An Example from Engineering. Improve the Academy, Vol. 25, 139-‐149.
Practical issues regarding research and
practice
The formation of two “camps”
People on both sides feeling left out, resentful, defensive
Feelings that research has higher status than practice (teaching)
Pressure that educational research have “immediate”
impact on practice – standard not applied to technical research
Perception that research site educational research
MUST be the classroom.
The frustration that research results often do not inform
Second polarity – single or
multiple realities
Types of Educational Research
Quantitative
Qualitative
Source: Douglas, E.P. (2010).Understanding qualitative research workshop. Frontiers of Education Conference.
Types of Educational Research
Quantitative
Qualitative
Source: Douglas, E.P. (2010).Understanding qualitative research workshop. Frontiers of Education Conference.
Types of Educational Research
Post-‐Positivist
Interpretivist
Single reality.
Relationships among variables.
Methods and variables defined in advance.
Researcher is detached.
Context-‐free generalizations.
Multiple realities.
Description of situation.
Methods and themes emerge during study.
Researcher and participants are partners.
Context-‐bound descriptions.
Source: Douglas, E.P. (2010).Understanding qualitative research workshop. Frontiers of Education Conference.
1. Pose significant questions that can be
investigated empirically
2. Link research to relevant theory
3. Use methods that permit direct
investigation of the question
4. Provide coherent, explicit chain of
reasoning
5. Replicate and generalize across studies
6. Disclose research to encourage professional
scrutiny and critique
Source: Shavelson, R. J. & Towne, L. (2002). Scientific research in education.
Washington DC: National Academies Press.
Some results of the positivist legacy
Only acceptable research design is experimental Pre-‐test, intervention, post-‐test
Experimental and control groups
Collect data that can be quantified
Find statistical difference between groups
Necessitates a large N
Common criticism: “Two few data points!”
Results of EER are universal – will work for everyone!
Suggested reframing of
the polarities
Reframing polarity one
No Yes
Yes Pure basic research (Bohr) basic research Use-inspired (Pasteur)
No
Pure applied
research (Edison)
Source: Stokes, D. ( 1997). Pasteur’s quadrant: Basic science and technological innovation. Washington,
DC: Brookings Institution.
Use (Applied)
Understanding (Basic)
Practical Problem Research Problem Research Question Research Answer motivates informs leads to and helps Research Process
Used in the American
Society for Engineering
Education Report:
Creating a Culture for
Scholarly and
Systematic Innovation
in Engineering
Reframing polarity two (1)
The research QUESTION drives the design and method
What phenomenon do you want to understand?
Is this a phenomenon that is already well
understood with identified variables?
Lends itself to correlational or experimental
research
Is this a new area just being explored?
Lends itself to interpretivist approaches
Pose significant questions that can be
investigated empirically
Link research to relevant theory OR create theory
Use methods that permit direct investigation of the question
Provide coherent, explicit chain of reasoning
Replicate and generalize across studies OR transfer to applicable contexts
Disclose research to encourage professional
scrutiny and critique
Reframing polarity two (2)
Engineering thinking is used to modify and/or create
new social or learning science methodologies and theory.
Emerging examples
New theory: Comparison of data on engineering student misconceptions to existing theories of conceptual change.
Montfort, Herman et al.
New methods: New ways to represent qualitative data.
How this transition might
be experienced
Lee Rynearson+
“I think the primary hangup about quantitative vs qualitative methods for me was that, reading descriptions of qualitative methods, they didn’t seem generalizeable. I took it as
axiomatic at the time that getting generalizable results
was the goal of research, and I didn’t see how these
methods could be applied effectively to that end. Once I understood that generally, quantitative and qualitative methods are intended for different purposes, and that they can be complementary rather than in competition,
this objection gradually dissolved...”
Michele Yatchmeneff+ (1)
When asked to consider Borrego’s “conceptual hurdles”
article I said “(4) appreciation of qualitative or mixed-‐ methods approaches” would be a hurdle I would face.
I further said: “I always planned to ‘just’ develop a tool/
instrument/survey to try to obtain results to my dissertation questions. I already am planning to
consider mixed-‐method approaches for my dissertation. I think as an engineer I am opposed to these kinds of approaches because we tend to want to number
crunch. I can see in a majority of the articles I have read
so far in the program that qualitative and mixed-‐ methods are utilized a lot.”
Michele Yatchmeneff+ (2)
When asked if I find any research methods particularly
intriguing I say: “I am an engineer and thought the
only way to collect data was for quantitative analysis. I realize the value of qualitative analysis and have
transitioned to looking at now collecting qualitative
data. I am considering collecting ethnographic
information this summer and looking at getting a book about how to effectively take ethnographic field notes.” Again, obviously showing I am considering other
methods other than quantitative research.
Think-‐pair-‐share
Reconnect with your partner
Individual reflection:
Do you see evidence of a researcher-‐practitioner
divide?
Do you see evidence of a positivist-‐interpretivist
(single reality-‐multiple realities) tension?
Partners share their thoughts with each other
Thanks you!
To Dr. Llew Mann, for inviting me to present this keynote.
To you the audience for your attention, interest, and participation.
To past workshop participants and ENE PhD students have taught
me so much.
To the National Science Foundation for funding Rigorous Research in Engineering Education: Creating a Community of Practice (DUE 0341127) and Expanding and sustaining research capacity in
engineering and technology education: Building on successful programs for faculty and graduate students (DUE 0817461)
