Overview of Research Designs
Overview of Research Designs
Overview of Research Designs
The Marketing Research Process
The Marketing Research Process
The Marketing Research Process
The Marketing Research Process
Step
Step 1: 1: Defining Defining the the ProblemProblem
Step
Step 2: 2: Developing Developing an an Approach Approach to to the the ProblemProblem
Step
Step 3: 3: Formulating Formulating a a Research Research DesignDesign
Step
Step 4: 4: Doing Doing Field Field Work Work or or Collecting Collecting DataData
Step
Step 5: 5: Preparing Preparing and and Analyzing Analyzing DataData
Step
Research Design
Research Design
•• A master plan that specifies the
A master plan that specifies the
methods and procedures for collecting
methods and procedures for collecting
and analyzing needed information.
Define the Information Needed Define the Information Needed Design the Exploratory, Descriptive, Design the Exploratory, Descriptive, and/or Causal Phases of the Research and/or Causal Phases of the Research Specify the Measurement and Scaling Specify the Measurement and Scaling
Procedures Procedures
Construct a Questionnaire Construct a Questionnaire
Specify the Sampling Process and the Specify the Sampling Process and the
Sample Size Sample Size
Develop a Plan of Data Analysis Develop a Plan of Data Analysis
Tasks Involved In a Research Design Tasks Involved In a Research Design
Today’s Today’s
Topic Topic
A Classification of Market Research Designs A Classification of Market Research Designs
Research Research Design Design Exploratory Exploratory Research Research Conclusive Conclusive Research Research Secondary Secondary Data Data Experience Experience Surveys Surveys Pilot Pilot Studies Studies Case Case Studies Studies
See next slide See next slide
A Classification of Market Research Designs A Classification of Market Research Designs
Research Research Design Design Exploratory Exploratory Research Research Conclusive Conclusive Research Research Descriptive Descriptive Design Design Causal Causal Design Design Cross-sectional Cross-sectional Study Study Longitudinal Longitudinal Study Study Secondary Secondary Data Study
Data Study ObservationObservation
Experiment Experiment Survey
Types of Research Designs Types of Research Designs
Exploratory research
Exploratory research toto gain ideas and insightsgain ideas and insights
Newspaper facing decreasing sales to generate possible Newspaper facing decreasing sales to generate possible explanation.
explanation.
Descriptive research
Descriptive research to obtain summary measures to to obtain summary measures to addressaddress research questions (research objectives are clearly defined). research questions (research objectives are clearly defined).
Trends in lifestyle with respect to age, sex, etc.Trends in lifestyle with respect to age, sex, etc.
Causal research
Causal research for for cause-and-effectcause-and-effect connection betweenconnection between managerial decisions and market
managerial decisions and market outcome.outcome. –
– How people react to a How people react to a newspaper’s topic selection andnewspaper’s topic selection and space allocation.
Exploratory Research
Exploratory Research
•• Usually conducted during the initial stage of the
Usually conducted during the initial stage of the
research process
research process
•• Purposes
Purposes
–
– To narrow the scope of To narrow the scope of the research topic, andthe research topic, and –
– To transform ambiguous problems into well-definedTo transform ambiguous problems into well-defined ones
Exploratory Research Techniques
Exploratory Research Techniques
•• Secondary Data AnalysisSecondary Data Analysis
–
– Secondary data are data previously collected & assembled for Secondary data are data previously collected & assembled for some project other than the one at hand
some project other than the one at hand
•• Pilot StudiesPilot Studies
–
– A collective term for any small-scale exploratory researchA collective term for any small-scale exploratory research technique that uses sampling but does not apply rigorous technique that uses sampling but does not apply rigorous standards
standards –
– IncludesIncludes
•• Focus Group InterviewsFocus Group Interviews
–
– Unstructured, free-flowing interview with a small Unstructured, free-flowing interview with a small group of peoplegroup of people
•• Projective TechniquesProjective Techniques
–
– Indirect means of questioning that enables a respondent to project beliefsIndirect means of questioning that enables a respondent to project beliefs and feelings onto a third party or an inanimate object
and feelings onto a third party or an inanimate object –
Exploratory Research Techniques
Exploratory Research Techniques
•• Case Studies
Case Studies
–
– Intensively investigate one or a Intensively investigate one or a few situations similar few situations similar to the problem situation
to the problem situation
•• Experience Surveys
Experience Surveys
–
– Individuals who are knowledge about a particular Individuals who are knowledge about a particular research problem are questioned
Conclusive Research
Conclusive Research
•• Provide specific information that aids the decision maker inProvide specific information that aids the decision maker in evaluating alternative courses of action
evaluating alternative courses of action
•• Sound statistical methods & formal research methodologies areSound statistical methods & formal research methodologies are used to increase the reliability of the information
used to increase the reliability of the information •• Data sought tends to be specific & decisiveData sought tends to be specific & decisive
Types of Conclusive Research
Types of Conclusive Research
•• Descriptive ResearchDescriptive Research
–
– Describes attitudes, perceptions, characteristics, activities andDescribes attitudes, perceptions, characteristics, activities and situations.
situations. –
– Examines who, what, when, where, why, & how questionsExamines who, what, when, where, why, & how questions
•• Causal ResearchCausal Research
–
– Provides evidence that a cause-and-effect relationship existsProvides evidence that a cause-and-effect relationship exists or does not exist.
or does not exist. –
– Premise is that something (and independent variable) directlyPremise is that something (and independent variable) directly influences the behavior of something else (the dependent
influences the behavior of something else (the dependent variable).
Common Characteristic
Common Characteristics
s of
of
Descriptive Studies
Descriptive Studies
•• Build on previous information
Build on previous information
•• Show relationships between variables
Show relationships between variables
•• Representative samples required
Representative samples required
•• Structured research plans
Structured research plans
•• Require substantial resources
Require substantial resources
•• Conclusive findings
Conclusive findings
Major Types of Descriptive Studies
Major Types of Descriptive Studies
Descriptive Descriptive Studies Studies Consumer Perception Consumer Perception And Behavior Studies And Behavior Studies
•• ImageImage
•• Product UsageProduct Usage •• AdvertisingAdvertising •• PricingPricing Market Characteristic Market Characteristic Studies Studies ••DistributionDistribution ••CompetitiveCompetitive Analysis Analysis •• MarketMarket Potential Potential •• MarketMarket Share Share •• SalesSales Analysis Analysis Sales Studies Sales Studies
Sample Sample Surveye Surveye d at T d at T11 Sample Sample Surveyed Surveyed at T at T11 Same Same Same Same Sample Sample also also Surveyed Surveyed at T at T22 T T11 T T 2 2 Cross Cross Sectiona Sectiona l Design l Design Longitudina Longitudina l Design l Design Time Time
Cross Sectional vs.
Cross-Sectional vs. Longitudinal Designs
Cross-Sectional vs. Longitudinal Designs
C
Crroossss--SSeeccttiioonnaall LLoonnggiittuuddiinnaall
D
Deetteeccttiinng g cchhaannggee WWoorrssee BBeetttteer r
A
Ammoouunnt t of of daatta d a ccoolllleecctteedd WWoorrssee BBeetttteer r
A
Accccuurraaccyy WWoorrssee BBeetttteerr
R
Reepprreesseennttaattiivveenneessss BBeetttteerr WWoorrssee
R
Exploratory Exploratory Research Research ••Secondary DataSecondary Data
Analysis Analysis
•• Focus Groups Focus Groups
Conclusive Research Conclusive Research • • Descriptive/Causal Descriptive/Causal Conclusive Research Conclusive Research •• Descriptive/Causal Descriptive/Causal Exploratory Exploratory Research Research •
•Secondary DataSecondary Data
Analysis
Analysis
•
• Focus Groups Focus Groups
Conclusive Research Conclusive Research •• Descriptive/Causal Descriptive/Causal
Some Alternative Research Designs Some Alternative Research Designs
(a) (a) (b) (b) (c) (c)
Common Characteristics of Causal
Common Characteristics of Causal
Studies
Studies
•• Logical Time Sequence
Logical Time Sequence
–
– For causality to exist, the cause must For causality to exist, the cause must either precedeeither precede or occur simultaneously with the effect
or occur simultaneously with the effect
•• Concomitant Variation
Concomitant Variation
–
– Extent to which the cause and effect vary together Extent to which the cause and effect vary together asas hypothesized
hypothesized
How Descriptive & Causal Designs
How Descriptive & Causal Designs
Differ
Differ
•• Relationship between the variablesRelationship between the variables–
– Descriptive designs determine degree of associationDescriptive designs determine degree of association –
– Causal designs infer whether one or more variables influenceCausal designs infer whether one or more variables influence another variable
another variable
•• Degree of environmental controlDegree of environmental control
–
– Descriptive designs enjoy lesser degrees of controlDescriptive designs enjoy lesser degrees of control
•• Order of the variablesOrder of the variables
–
Comparison of Research Designs
Comparison of Research Designs
E
Exxpplloorraattoorryy DDeessccrriippttiivvee CCaauussaall
Purpose
Purpose IID D pprroobblleemmss, g, gaaiin n iinnssiigghhttss DDeessccrriibbe te thhiinnggss DDeetteerrmmiinne ce caauussee--aannd
d--effect relationships effect relationships Assumed Assumed background background knowledge knowledge M
Miinniimmaall CCoonnssiiddeerraabbllee CCoonnssiiddeerraabbllee
Degree of structure
Degree of structure VVeerry y lliittttllee HHiigghh HHiigghh Flexibility
Flexibility HHiigghh SSoommee LLiittttllee Sample
Sample NNoonn--rreepprreesseennttaattiivvee ReepR prreesseennttaattiivvee RReepprreesseennttaattiivvee Research
Research environment environment
R
Reellaaxxeedd FFoorrmmaall HiigH ghhlly y ccoonnttrroolllleedd
Cost
Cost LLooww MMeeddiiuumm HHiigghh Findings
Which is the “Best” Research Design &
Which is the “Best” Research Design &
Method?
Method?
•• “You cannot put the same shoe on every foot.”“You cannot put the same shoe on every foot.” –
– Publilius Syrus Publilius Syrus •• It depends on theIt depends on the
–
– problem of interest, problem of interest, –
– level of information needed,level of information needed, –
– resources,resources, –
What is Descriptive Research?
What is Descriptive Research?
•• Can involve collecting quantitative
Can involve collecting quantitative
information
information
•• Can describe categories of qualitative
Can describe categories of qualitative
information such as patterns of interaction
information such as patterns of interaction
when using technology in the classroom.
when using technology in the classroom.
•• Does not fit neatly into either category
Does not fit neatly into either category
What is
What is
Descriptive Research?
Descriptive Research?
•• Involves gathering data that describe events
Involves gathering data that describe events
and then organizes, tabulates, depicts, and
and then organizes, tabulates, depicts, and
describes the data.
describes the data.
•• Uses description as a tool to organize data
Uses description as a tool to organize data
into patterns that emerge during analysis.
into patterns that emerge during analysis.
•• Often uses visual aids such as graphs and
Often uses visual aids such as graphs and
charts to aid the reader
charts to aid the reader
Descriptive Research
Descriptive Research
takes a “what is” approach
takes a “what is” approach
•• What is the best way to provide access to computer What is the best way to provide access to computer equipment in schools?
equipment in schools?
•• Do teachers hold favorable attitudes toward usingDo teachers hold favorable attitudes toward using computers in schools?
computers in schools?
•• What have been the reactions of school administrators toWhat have been the reactions of school administrators to technological innovations in teaching?
Descriptive Research
Descriptive Research
•• Refers to the nature of the research question
Refers to the nature of the research question
•• The design of the research
The design of the research
•• The way that data will be analyzed for the
The way that data will be analyzed for the
topic that will be researched
topic that will be researched
The type of research question will determine
The type of research question will determine
whether descriptive research approach is
whether descriptive research approach is
appropriate to use.
Descriptive Research Advantages
Descriptive Research Advantages
•• Educational researchEducational research and experiences may and experiences may contain many
contain many
variables that cannot variables that cannot be realistically
be realistically controlled.
controlled.
•• Educational researchEducational research may require may require observations of life observations of life experiences experiences
•• Data collection may beData collection may be spread over a large
spread over a large
number of people over number of people over a large geographic
a large geographic area
Data Collection Methods
Data Collection Methods
S
Su
urrv
veey
yss
IIn
ntteerrv
viieew
wss
O
Descriptive Research
Descriptive Research
1. Surveys
1. Surveys
•• May be used to revealMay be used to reveal summary statistics by summary statistics by showing responses to showing responses to all possible all possible questionnaire items. questionnaire items. •• Often provide leads inOften provide leads in
identifying needed identifying needed changes
changes
•• May be used toMay be used to
explore relationships explore relationships between 2 or more between 2 or more variables. variables.
Descriptive Research
Descriptive Research
Critical Components
Critical Components
Sound
Sound
methodology
methodology
Well-designed
Well-designed
data collection
data collection
instruments
instruments
Descriptive Research
Descriptive Research
Survey Forms
Survey Forms
•• Written questionnairesWritten questionnaires •• Personal interviewsPersonal interviews
•• Telephone interviewsTelephone interviews
Factors to be considered Factors to be considered •• SamplingSampling
•• Type of populationType of population •• Question FormQuestion Form
•• Question ContentQuestion Content •• Response ratesResponse rates •• CostsCosts
•• Available facilitiesAvailable facilities
•• Length of data collectionLength of data collection
•• Computer assisted techniquesComputer assisted techniques for data collection
Descriptive Research
Descriptive Research
2.
2. Survey
Survey Form
Form -
- Interviews
Interviews
•• More time efficient
More time efficient
•• Allow the researcher to establish a rapport
Allow the researcher to establish a rapport
with the respondent
with the respondent
•• Allow the acquisition of more in-depth
Allow the acquisition of more in-depth
information
information
•• Allow for interviewer observation
Allow for interviewer observation
•• Allow the interviewer to obtain visual cues
Allow the interviewer to obtain visual cues
•• May be personal or telephone interviews
May be personal or telephone interviews
Descriptive Research
Descriptive Research
Survey Form – Personal
Survey Form – Personal
Interviews
Interviews
Disadvantages
Disadvantages
•• Require more staff time
Require more staff time
•• Require more travel time
Require more travel time
Descriptive Research
Descriptive Research
Survey Form – Telephone
Survey Form – Telephone
Interview
Interview
Advantages Advantages
Less expensiveLess expensive
Less time-consumingLess time-consuming
Disadvantages Disadvantages
Limited telephoneLimited telephone access
access
Lack of interviewer’sLack of interviewer’s ability to observe the ability to observe the respondent and obtain respondent and obtain visual cues
Descriptive Research
Descriptive Research
Survey Form – Mailed
Survey Form – Mailed
Questionnaires
Questionnaires
AdvantagesAdvantages
Ability to reach largeAbility to reach large
number of people across a number of people across a wide geographic area
wide geographic area
Ease and low cost of Ease and low cost of distribution
distribution
Minimal amount of staff Minimal amount of staff required
required
Allows respondents toAllows respondents to respond in their time respond in their time frame
frame
Disadvantages Disadvantages
Lower response rateLower response rate
Need to design a Need to design a
survey instrument with survey instrument with a simple format
Descriptive Research
Descriptive Research
Survey Form – Mailed
Survey Form – Mailed
Questionnaires
Questionnaires
•• A letter of transmittal should accompany mailedA letter of transmittal should accompany mailed questionnaires.
questionnaires.
•• Should state purpose and importance of rShould state purpose and importance of researchesearch •• Should state importance of respondingShould state importance of responding
•• Should give a time frame to respondShould give a time frame to respond
•• Should include a confidentiality statementShould include a confidentiality statement •• Should include an offer to share resultsShould include an offer to share results
Descriptive Research
Descriptive Research
Characteristics of a Good Survey
Characteristics of a Good Survey
•• Good questioning techniquesGood questioning techniques •• Use complete sentencesUse complete sentences
•• Offer a limited set of answersOffer a limited set of answers •• InterestingInteresting
•• Worded so that questions Worded so that questions mean the same to allmean the same to all •• Provide definitions for confusing termsProvide definitions for confusing terms
Descriptive Research
Descriptive Research
3. Observational Research
3. Observational Research
Methods
Methods
1.
1. Nat
Natura
urall
lly o
y occu
ccurr
rrin
ing b
g beha
ehavi
viors
ors obs
obser
erved
ved in
in
natural contexts
natural contexts
2.
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
•• Require direct observation of behavior
Require direct observation of behavior
•• Data gathered without intermediary
Data gathered without intermediary
instruments
instruments
•• Can yield a wealth of invaluable
Can yield a wealth of invaluable
information
information
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
•• Can be employed productively to support
Can be employed productively to support
many purposes in educational technology
many purposes in educational technology
•• Can be used to determine how people
Can be used to determine how people
interact with technology in various stages of
interact with technology in various stages of
design and implementation
Descriptive Research
Descriptive Research
Observation Research Methods
Observation Research Methods
•• How do learners interact with a sHow do learners interact with a specific program?pecific program? •• How do learners interact with a new hardwareHow do learners interact with a new hardware
system? system?
Observation makes it possible to explore the Observation makes it possible to explore the
implementation of a particular technological implementation of a particular technological innovation and assess the instr
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
•• 2 Forms of Observational Research
2 Forms of Observational Research
–
– StructuredStructured –
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
Structured Observations Structured Observations
Rigid and controlledRigid and controlled
PredeterminedPredetermined methods methods Unstructured Observations Unstructured Observations
Used to determineUsed to determine unselective, detailed, unselective, detailed,
continuous description of continuous description of behavior.
behavior.
Detects unintended effectsDetects unintended effects
More time consumingMore time consuming
because of time and labor because of time and labor
required to collect and required to collect and analyze sets of extensive analyze sets of extensive observations
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
Develop observation form
Develop observation form
May be paper and pencil or electronic
May be paper and pencil or electronic
May use a rating scale to evaluate behavior
May use a rating scale to evaluate behavior
Descriptive Research
Descriptive Research
Observational Research Methods
Observational Research Methods
Newer Mediated Observation Techniques Newer Mediated Observation Techniques
•• AudioAudio
•• VideotapeVideotape
•• Computers provide on-line monitoring (process of Computers provide on-line monitoring (process of capturing characteristics of the human-computer capturing characteristics of the human-computer interaction automatically)
interaction automatically)
–
– Keystroke recordsKeystroke records –
– Audit trailsAudit trails –
Descriptive Research
Descriptive Research
4. Portfolios
4. Portfolios
•• Provide a descriptive measure of student
Provide a descriptive measure of student
work based on actual performance
work based on actual performance
•• Consist of learner-created products that
Consist of learner-created products that
reflect the processes of learning and
reflect the processes of learning and
development over time
Impact and Future of
Impact and Future of
Descriptive Research
Descriptive Research
•• Although descriptive studies are important,
Although descriptive studies are important,
most educational studies involve questions
most educational studies involve questions
about cause.
about cause.
–
– What causes underachievement?What causes underachievement? –
– Will multimedia cause students to be moreWill multimedia cause students to be more
motivated or lead to high achievement levels? motivated or lead to high achievement levels?
Impact and Future of
Impact and Future of
Descriptive Research
Descriptive Research
•• Not always easy to isolate variable that will Not always easy to isolate variable that will
explain those causes, so descriptive research can explain those causes, so descriptive research can play an important role in providing
play an important role in providing informationinformation from another perspective.
from another perspective.
•• By gathering descriptions of “what is” andBy gathering descriptions of “what is” and comparing them to “what we would like” comparing them to “what we would like” educators can see the area that needs to be educators can see the area that needs to be addressed.
Impact and Future
Impact and Future
of Descriptive Research
of Descriptive Research
•• Descriptive Research methods have gained
Descriptive Research methods have gained
acceptance
acceptance
•• Number of descriptive studies published in
Number of descriptive studies published in
research journals has increased
research journals has increased
•• Descriptive research leads to prescriptions
Descriptive research leads to prescriptions
that instructional designers and educators
that instructional designers and educators
can heed as they consider future direction
can heed as they consider future direction
Questions Addressed by Causal
Questions Addressed by Causal
Research
Research
** Marketing dMarketing director oirector of local f local beer cobeer company,mpany,
“Will replacing TV commercial A with commercial B
“Will replacing TV commercial A with commercial B lead tolead to increase in consumer preference on our brand?”
increase in consumer preference on our brand?” *
* Chairman of Chairman of a charity oa charity organization. “Will rganization. “Will it be woit be worthwhile torthwhile to mail to previous donors an attractive and expensive brochure mail to previous donors an attractive and expensive brochure to solicit higher contributions this
to solicit higher contributions this year?”year?” *
* The sales manThe sales manager of ager of a local a local life insurance life insurance company.company. “Will training in the use
“Will training in the use of computers for client managementof computers for client management increase agents’ sales?”
increase agents’ sales?” *
* Marketing VP Marketing VP of fashion of fashion chain, “Can chain, “Can we improve we improve profitabilityprofitability of our fashion clothing line by increasing its
Causality: Cause-and-Effect
Causality: Cause-and-Effect
Change-in-X causes change-in-Y
Change-in-X causes change-in-Y
Evidence of Causality
Evidence of Causality
•• Concomitant variation:Concomitant variation: –
– If X changes, then Y also If X changes, then Y also changes.changes. –
– If X does not change, then Y does notIf X does not change, then Y does not
change.
change.
•• Time order: cause (X) occurs before effect (Y).Time order: cause (X) occurs before effect (Y). •• Elimination of alternative explanation:Elimination of alternative explanation:
–
– Could changes in Y be caused byCould changes in Y be caused by
changes in Z?
changes in Z?
Concepts and Terminology
Concepts and Terminology
Causality Causality
Manipulation (experimentation) Manipulation (experimentation)
Independent variable (X, manipulated, Independent variable (X, manipulated, computer skill)
computer skill)
Dependent variable (Y, reflect the impact of Dependent variable (Y, reflect the impact of independent variable, sales)
independent variable, sales)
Treatment Group: subject exposed to Treatment Group: subject exposed to
manipulation (salespeople with training) manipulation (salespeople with training) Control Group: part of sample that sees no Control Group: part of sample that sees no
change in independent variable (no computer change in independent variable (no computer training)
Validity of Experiment
Validity of Experiment
Internal validity:
Internal validity: elimination of other possible causeselimination of other possible causes..
–
– Sources: external factors (Asian crisis),Sources: external factors (Asian crisis),
maturation of subjects (agents more experienced). maturation of subjects (agents more experienced).
–
– Lab experiment > Field experiment.Lab experiment > Field experiment.
External validity:
External validity: generalizability of results.generalizability of results. –
– Artificial situation; inappropriate sample;Artificial situation; inappropriate sample;
inappropriate time. inappropriate time.
–
Types of Experiments
Types of Experiments
Laboratory experiment Laboratory experiment Field experiment Field experimentScientific investigation in which Scientific investigation in which an
an
investigator manipulates and investigator manipulates and controls
controls
one or more independent one or more independent variables and
variables and
observes the dependent observes the dependent variable for
variable for
variation concomitant to the variation concomitant to the manipulation
manipulation
of the independent variables of the independent variables
Research investigation in which Research investigation in which investigator creates a situation investigator creates a situation with exact conditions so as to with exact conditions so as to control some, and manipulate control some, and manipulate other, variables
other, variables
Research study in a realistic Research study in a realistic situation in
situation in
which one or more
which one or more independentindependent variables are manipulated by the variables are manipulated by the experimenter under as carefully experimenter under as carefully controlled conditions as the
controlled conditions as the situation will permit
Experiments are the
Experiments are the
STRONGEST Design Logic
STRONGEST Design Logic
•• The “best” ResearchThe “best” Research •• Design to determineDesign to determine
CAUSAL CAUSAL
RELATIONS RELATIONS
Use Experimental designs
Use Experimental designs
whenever possible
whenever possible
•• Types of Types of
EXPERIMENTAL EXPERIMENTAL DESIGNS:
DESIGNS: •• LabLab
•• Field ExperimentField Experiment •• Quasi-ExperimentalQuasi-Experimental