Lecture 2

Lecture # 2

Lecture # 2

The Research Process

The Research Process

Research:

Research: A A way way of of ThinkingThinking

The Research Process: A Quick Glance

R R e e s s e e a a r r c c h h M M e e t t h h o o d d s s : : D D r r . . I I m m r r a a n n U U s s m m a a n n

1. Introduction

1. Introduction

1.

1.What is Research ??What is Research ??

i.

i. One of the ways to One of the ways to find answers to yourfind answers to your questions

questions

2.

2.It is implication of It is implication of a process, and the processa process, and the process::

i.

i. Is within a framework of a set Is within a framework of a set of philosophiesof philosophies ii.

ii.Uses procedures, methods, and techniques thatUses procedures, methods, and techniques that are tested for their

are tested for their validity and reliabilityvalidity and reliability iii.

iii.Is designed to be unbiased and objectiveIs designed to be unbiased and objective







R R e e s s e e a a r r c c h h M M e e t t h h o o d d s s : : D D r r . . I I m m r r a a n n U U s s m m a a n n

1. Introduction

1. Introduction

1.

1.What is Research ??What is Research ??

i.

i. One of the ways to One of the ways to find answers to yourfind answers to your questions

questions

2.

2.It is implication of It is implication of a process, and the processa process, and the process::

i.

i. Is within a framework of a set Is within a framework of a set of philosophiesof philosophies ii.

ii.Uses procedures, methods, and techniques thatUses procedures, methods, and techniques that are tested for their

are tested for their validity and reliabilityvalidity and reliability iii.

iii.Is designed to be unbiased and objectiveIs designed to be unbiased and objective







R R e e s s e e a a r r c c h h M M e e t t h h o o d d s s : : D D r r . . I I m m r r a a n n U U s s m m a a n n

2.i. Po

2.i. Po

ssible set of

ssible set of

Philosophies

Philosophies



  Your philosophical orientation may stem from  Your philosophical orientation may stem from

one of the several philosophical approaches one of the several philosophical approaches in research:

in research:

i.

i. PositivistPositivist ii.

ii. InterpretiveInterpretive iii.

iii. PhenomenolistPhenomenolist iv

iv.. Action orientedAction oriented v

v.. ParticipatoryParticipatory vi.

vi. QualitativeQualitative vii.

R R e e s s e e a a r r c c h h M M e e t t h h o o d d s s : : D D r r . . I I m m r r a a n n U U s s m m a a n n

2.ii Validity and Reliability

2.ii Validity and Reliability



 Deals with whether the procedures, methods,Deals with whether the procedures, methods,

and techniques adopted are tested for their and techniques adopted are tested for their validity and reliability

validity and reliability 1.

1.Validity:Validity:



 _{Ensures that in a research study correctEnsures that in a research study correct}

procedures have been applied to find procedures have been applied to find answers to a question

answers to a question

2.

2.Reliability:Reliability:



 _{Refers to the quality of a measurementRefers to the quality of a measurement}

procedure that provides repeatability and procedure that provides repeatability and accuracy

R e se a rc h M e th o d s: D r. Im ra n U sm a n

2.iii Unbiased and Objective

1.Unbiased:

 _{That you have taken each step in an unbiased}

manner and drawn each conclusion to the best of your ability and without introducing your own vested interests.

2.Objective:

 _{Should achieve a certain objective for which the}

R e se a rc h M e th o d s: D r. Im ra n U sm a n

3. An Overview of Research Methods

and Methodologies

Why Do I Need to Know About Different

Methods?



 As a graduate student...

 _{To be able to read and understand the empirical}

literature in your field; to become a critical consumer of information.



 As a graduate student preparing for a thesis or

dissertation…

 _{To be able to both design and implement your thesis}

or dissertation as well as future studies that interest you.

R e se a rc h M e th o d s: D r. Im ra n U sm a n  As a future practitioner…

 _{To be able to intelligently participate in research}

projects, evaluations, and studies undertaken by your institution.



 As an educated citizen ...

 _{To understand the difference between scientifically}

acquired knowledge and other kinds of information.

What’s the Difference Between

“Method” and “Methodology”?

Method:

 Techniques for

gathering evidence

 The various ways of 

proceeding in gathering

information

Methodology:

 The underlying theory

and analysis of how research does or should proceed, often influenced by discipline  

R e se a rc h M e th o d s: D r. Im ra n U sm a n

Epistemology, Methodology, and

Method

 a research method  is a technique for (or way

of proceeding in) gathering evidence“

 while "methodology  is a theory and analysis of 

how research does or should proceed”

 and "an epistemology  is a theory of 

R e se a rc h M e th o d s: D r. Im ra n U sm a n   

"It is the theory that decides what can be

observed."



 Albert Einstein

An Overview of Empirical Research

Methods

Descriptive (Qualitative)  Ethnography  Case Study  Survey/Sampling  _{Focus Groups}  Discourse/Text Analysis  Quantitative Description  Prediction/Classification   Experimental (Quantitative)  True Experiment  Quasi-Experiment  Meta-Analysis 

Ethnographies

+Observational field work done in the actual context being studied

+Focus on how individuals interrelate in their own environment (and the influence of this environment)

- Difficult to interpret/analyze -  Time consuming/expensive

Case Studies

+Focus is on individual or small group

+Able to conduct a comprehensive analysis from a comparison of cases

+Allows for identification of variables or phenomenon to be studied

-  Time consuming

- Depth rather than breadth

Survey Research

+An efficient means of gathering large amounts of data

+Can be anonymous and inexpensive - Feedback often incomplete

- Wording of instrument can bias feedback - Details often left out

Focus Groups

+Aid in understanding audience, group, users +Small group interaction more than individual

response

+Helps identify and fill gaps in current knowledge re: perceptions, attitudes, feelings, etc.

- Does not give statistics

- Marketing tools seen as “suspect” - Analysis subjective

Discourse/Text Analysis

+Examines actual discourse produced for a particular purpose (job, school)

+Helps in understanding of context, production, audience, and text

+Schedule for analysis not demanding - Labor intensive

Quantitative Descriptive Studies

+Isolates systematically the most important variables (often from case studies) and to quantify and interrelate them (often via survey or questionnaire)

+Possible to collect large amounts of data +Not as disruptive

+Biases not as likely

Prediction and Classification Studies

Goal is to predict behaviors:

 Prediction forecasts and interval variable

(Diagnostic/TAAS scores)

 Classification forecasts a nominal variable (Major

selection after taking 2311)

+ Important in industry, education to predict behaviors

- Need substantial population

- Restricted range of variables can cause misinterpretation

- Variables cannot be added together; must be weighted and looked at in context of other variables

Positive Aspects of 

Descriptive/Qualitative Research

 Naturalistic; allows for subjects to interact

with environment

 Can use statistical analysis

 Seeks to further develop theory (not to

influence action); Prescientific

 Coding schemes often arise from interplay

between data and researcher’s knowledge of  theory

Problems with Descriptive/Qualitative

Research

 Impossible to overlay structure  Impossible to impose control

 Subject pool often limited, not representative  Seen as more “subjective,” less rigorous

 Beneficial only in terms of initial investigation

Experimental Research: True

Experiment

+Random sampling, or selection, of subjects

(which are also stratified)

+Introduction of a treatment

+Use of a control group for comparing

subjects who don’t receive treatment

with those who do

- Adherence to scientific method (seen as

positive, too)

- Must have both internal and external

validity

Experimental Research:

Quasi-Experiment

+Similar to Experiment, except that the

subjects are not randomized. Intact

groups are often used (for example,

students in a classroom).

+ To draw more fully on the power of the

experimental method, a pretest may be

employed.

+Employ treatment, control, and scientific

method

- Act of control and treatment makes

situation artificial

Meta-Analysis

+ Takes the results of true and

quasi-experiments and identifies

interrelationships of conclusions

+Systematic

+Replicable

+Summarizes overall results

- C/C apples and oranges?

Positive Aspects of Experimental

Research

 Tests the validity of generalizations  Seen as rigorous

 Identifies a cause-and-effect relationship  Seen as more objective, less subjective  Can be predictive

Problems with Experimental Research

 Generalizations need to be qualified according

to limitation of research methods employed

 Controlled settings don’t mirror actual

conditions; unnatural

 Difficult to isolate a single variable

Assessing Methods

 Research Question(s) is/are key

 Methods must answer the research question(s)  Methodology guides application

 Epistemology guides analysis  All must include “rigor”

 Testing the Waters

 How do you come up with a good research

question?

 How do you determine if the method you plan

to use will answer your question?

 What epistemology should you use to analyze

Case Scenario

 Test your research savvy with the following

case. Assume that you are the Mayor of  Greenwood, a small town in Illinois, and

you’ve got to make a decision based on the information collected from the following

Crime Reduction Program, City of

Greenwood

103rd

 _{The chief of police wants to}

experiment with increasing the number of patrol officers (X) to reduce the crime rate (Y).

 _{The chief invites all}

twelve-precinct captains to participate in the experiment; only the 103rd volunteers.

 _{In October, patrol officers in the}

103rd are increased by 15%.

 _{Reported crime drops 5%}

between September & December. The chief now wants to implement the program citywide.

ou are e mayor. ou you

support this request based upon the

results of this study?



üCould severe weather in November and December have caused the crime rate to decline?

üIs crime seasonal, peaking in the summer and declining in the winter?

More Problems

üSince the captain of the 103rd volunteered for the program, could he have already implemented other programs that account for the decline in crime?

üSince the officers in the 103rd knew they were involved in a priority program, is it possible that they recorded reported crime differently?

More Problems

üWill the crime reduction impact last very long?

üCould random error in the measurement of the crime rate account for the difference?

üWas the crime rate in the entire city going down anyway?

ü

What Makes Research Good?

 Validity  Reliability  Replicability  Consistent application/analysis  “Trustworthiness”  Rigor

Validity in Research

 Refers to whether the research actually

measures what it says it’ll measure. Validity is the strength of our conclusions, inferences or propositions.



 _{Internal Validity: the difference in the dependent}

variable is actually a result of the independent variable



 External Validity: the results of the study are

generalizable to other groups and environments outside the experimental setting



 _{Conclusion Validity: we can identify a relationship}

between treatment and observed outcome



Reliability in Research

  The consistency of a measurement, or the

degree to which an instrument measures the same way each time it is used under the same condition with the same subjects. In short, it is the repeatability of your measurement. A

measure is considered reliable if a person's

score on the same test given twice is similar. It is important to remember that reliability is not measured, it is estimated. Measured by

Validity and Reliability

 The relationship between reliability and

validity is a fairly simple one to understand: a measurement can be reliable, but not valid. However, a measurement must first be

reliable before it can be valid. Thus reliability is a necessary, but not sufficient, condition of  validity. In other words, a measurement may consistently assess a phenomena (or

outcome), but unless that measurement tests what you want it to, it is not valid.

Rigor in Research

 Validity and Reliability in conducting research  Adequate presentation of findings:

consistency, trustworthiness

 Appropriate representation of study for a

particular field: disciplinary rigor

 Rhetorical Rigor: how you represent your

Key Considerations to Design Your

Research Approach

 What question do you want to answer?

 For what purposes is the research being done?

i.e., what do you want to be able to do or decide as a result of the research?

 Who are the audiences for the information

from the research, e.g., teachers, students, other researchers, members of a disciplinary community, corporate entities, etc.?

 From what sources should the information be

collected, e.g., students, teachers, targeted groups, certain documentation, etc.?

Key Considerations to Design Your

Research Approach

 What kinds of information are needed to make

the decisions you need to make and/or to enlighten your intended audiences, e.g., do you need information to really understand a process, the students who engage in a

process, strengths and weaknesses of a

curriculum or program, benefits to students or institution or agency, how aspect of a

program are problematic, etc.?

 

Key Considerations to Design Your

Research Approach

 How can that information be collected in a

reasonable fashion, e.g., questionnaires, interviews, examining documentation, observing staff and/or clients in the

program, conducting focus groups among staff and/or students, etc?

 How accurate will this information be?

 When is the information needed (so, by when

must it be collected)?

 What resources are available to collect the

information?

 How will this information be analyzed? 