RESEARCH METHODS
FOR RESIDENT PHYSICIANS
MODULE ONE
RESEARCH PROTOCOL DEVELOPMENT
NOEL L. ESPALLARDO, MD
CLINICAL ASSISTANT PROFESSOR
DEPARTMENT OF FAMILY AND COMMUNITY MEDICINE UNIVERSITY OF THE PHILIPPINES COLLEGE OF MEDICINE
LECTURE I
RESEARCH IDEAS, RESEARCH QUESTION AND RESEARCH OBJECTIVES
Preparing for your research project
RESEARCH IDEAS
One conducts research because of pure interest, or it is a career or it is a requirement for promotion, training or even recognition. Whatever is the purpose of conducting a research one has to start with a good research idea.
Sources of Research Ideas
Experience
An expert researcher will always have his experience as the best source of new research idea. Unfortunately for those who are still in residency or are just trying to start a research project experience may not be enough. Other source of ideas should also be explored.
Another source of idea is the medical literature. They may be in an electronic data base like the MEDLINE, INTERNET or publications like medical journals in the library. These sources should be scanned very well in order to get a good and relevant research idea. It is therefore necessary for a serious researcher to have an adequate knowledge and skill in browsing through this medical literature.
Morbidity and mortality statistics in your area of practice can also be a good source of research idea. This will surely be relevant in your setting.
Be alert to new ideas
New ideas can also be taken from scientific gatherings, discussion with friends or colleagues or experts in your area of practice. Make sure that every time you attend a scientific convention or meeting, try to pick up new ideas or problem areas.
Every now and then you will also be assigned to report or discuss a clinical case in your group discussions. In most cases they are part of your training program. This exercise can also be a source of research ideas.
Oftentimes a research idea can pop out of the blue anywhere or anytime and then disappear. It is therefore advisable that when it occur write it down in any piece of paper or notebook that can be retrieved when needed.
Keep your imagination roaming.
A skeptical attitude to current practice will stimulate your mind to be creative and imaginative. Considering the increasing cost of treatment, there is always a need for alternative and equally effective but less expensive modalities.
Choose a good mentor.
If after going through the sources listed above you still don’t have a good research idea, you still have one last alternative. Ask your consultant!
Focusing Research Ideas
Many research ideas do not materialize because they are vague or too broad. Although a research idea may have one primary problem and a couple of secondary problems, it is advisable that for a beginning researcher focusing on just one research problem is enough. There is no formula on how to focus the research idea into a research question. It all depends on the interest of the researcher. The examples below can serve as a guide on how to focus the research idea.
A young researcher wanted to study better ways to improve the success of TB treatment.
There are so many ways that TB treatment can be improved. So the researcher focused in improving compliance because it is a grave problem among TB patients. The researcher further considered that decreasing the frequency of medication with slight dose modification or directly observing the intake of medication will improve compliance. In this scenario however, two factors are being considered i.e. decreasing frequency and observing therapy that may have a confounding effect on compliance. To avoid complications the researcher eventually chose to study the effect of directly observed therapy on treatment compliance.
The example above started from a general idea of improving success of TB treatment to a more focused research idea of investigating the effect of directly observed treatment to
compliance of TB treatment.
Another researcher wanted to investigate alternative ways of improving the control of hypertension other than or in addition to pharmacological treatment. Recent guidelines suggest that lifestyle modification and risk factor modification should always be considered as an additional intervention. However, encouraging the patients to adhere to lifestyle modification is often difficult. The researcher thought that if someone else probably a family member can encourage and remind the patient constantly, adherence will be improved. The researcher later decided to investigate the effectiveness of educating the patient and a family member in controlling the blood pressure and modifying the risk factors. The research idea is now more focused on the effect of educating the patient and the relative.
RESEARCH QUESTION
The research question is a long statement intended to focus the research project. It is usually written in the latter part of the introduction. In a protocol it should be stated in such a way that it can be answered by a yes or no. It usually embodies the research objectives.
Components of a Good Research Question
A good research question should contain at least all of the following:
1. Biologic or theoretical rationale.
2. Specific population to be studied.
3. Specific test intervention.
4. Specific primary outcome of interest.
5. Specific comparative intervention if a comparative study is planned.
6. Suggestion of a study design.
Below is an example of a research question on the effectiveness of a surrogate observer in the treatment of tuberculosis.
“Among patients consulting at the local health center and diagnosed to have PTB randomized to a clinical trial, will compliance to TB treatment be improved by encouraging a family member to be a surrogate observer to drug intake compared with usual clinic advise?”
Biologic rationale yes, family members can encourage drug intake thereby improving compliance
Specific population patients diagnosed to have PTB in a community health center Specific test intervention use of surrogate observer for drug intake
Specific primary outcome of
interest improving compliance to TB chemotherapy
Specific comparative intervention
usual clinic advise Suggestion of a study design randomized clinical trial
Another example is given below.
“Among the newly diagnosed hypertensive patients consulting in an outpatient clinic, will compliance to lifestyle modification and control of hypertnesion be improved by
educating the patient and family member about lifestyle modification than just educating the patient alone in a randomized clinical trial design?”
Criteria for a Good Research Question
The criteria for a god research question is best remembered by the mnemonics F-I-N-E-S- T (which stands for F-feasible, I-interesting, N-novel, E-ethical, S-significance and T-time bound).
Thus in undertaking a research project, the research question must at least fulfill these criteria.
Feasibility
Before you waste your time and effort in a particular research plan, first make sure that the undertaking is feasible. Feasibility can be affected by several factors, but the most prominent may be the cost of the project and the availability of study population. It is extremely difficult to conduct a study that needs P 1 million in funding if the available amount is only P 1 hundred thousand. Likewise it is difficult to study a disease that is very rare in a population. It will take a whole lifetime before you can adequately accumulate your cohort.
Interesting
This may depend on whose perspective is the research question being evaluated. It is helpful to view the interest first from the investigator, second from the patient and third from the health care provider.
Novel
A good research always contribute new information.
Ethical
A research project will definitely not pass the institutional review board if the study is not ethical. A study therefore must not place the participants in unnecessary risk or deny them of the necessary benefits. It should also not infringe on their privacy.
Significance
Just like interest, significance may be viewed from a different perspective. Significance may also be evaluated in the same manner as interest.
Time Bound
It is very important that when you undertake a study, it should be finished at least within your lifetime (or within your training, or within your deadline).
To make sure that your research question is a good one, apply the checklist as shown in Table 1.
Table 1 Checklist for a Good Research Question
Yes No Not Sure
Is it feasible?
Do I have enough subjects?
Do I have enough funding?
Is it interesting?
For the investigator?
For the patient?
For the health care provider?
Is it new?
Had the medical literature been searched?
Had the experts been consulted?
Are there any controversy that need answers?
Is it ethical?
Is there any risk for the subjects?
Will the study violate their privacy?
Is the study significant?
For the investigator?
For the patient?
For the health care provider?
Can the study be completed within a given period?
Deciding to Undertake the Research Project
After subjecting your research question to the checklist for a good research question it may be helpful to ask the following questions below before embarking on your study.
Elaborate significance
1. Is there a scientific rationale?
The answer to this question may be implied or you can design your theoretical framework based on your previous readings.
2. Is enough known about the problem?
To answer this question a thorough search of the medical literature should first be done.
An electronic database search is usually more efficient than a hand search of printed journals in the medical library.
3. Is the objective directed toward the improvement in health care?
4. Is it a priority problem?
If the answers to the questions above are all yes, then your study is significant. The next question to answer is “Is it feasible?” To answer this first make a general plan of your study.
Follow the outline suggested in Table 2. Then ask the three questions below.
1. Is the proposed methodology feasible?
2. Can I finish it in time?
3. Can I afford the cost of the study or is funding available?
If the answers to the questions above are all yes, congratulations you’re now ready to prepare your research protocol.
After formulating the research question and study plan a few problems might arise and this might lead you to discard the research project. However there are solutions I would like to suggest before discarding your research idea. First is consult Table 3 and adopt the suggested solutions for the problems. Then present your study plan to you friends, colleagues and mentors for comments and suggestions. This is the iterative process of forming a research proposal. Lastly rely on your creativity, judgment and tenacity.
No amount of problem will be left unsolved to a determined investigator.
Table 2 The Research Plan Title
Investigator/s
Research Question
Objectives
Study Design
Study Population Inclusion Criteria Exclusion Criteria Sample Size
Intervention Experimental Control
Outcomes
Procedures for Measurement of Outcomes
Statistical Analysis
Total
Budget/Schedule/Perso nnel
Table 3 Problems and Solution for a Difficult Research Question or Research Plan
Problems Solutions Vague or inappropriate
research plan Write the research question at an early stage.
Get specific in the study plan about:
how the subjects will be sampled how the outcomes will be measured Think of ways on how to make:
the subjects more representative of the population
the measurements more representative of the phenomena of interest
Not Feasible Too broad
Not enough subjects available
Methods inadequate or beyond the skills of the investigator
Too expensive
Specify a smaller set of outcomes or variables Narrow the question
Expand the inclusion criteria Eliminate exclusion criteria
Add other sources of subjects, multi-center Lengthen the time frame for entry into the study Use more efficient variables or design
Consult experts or review literature for alternative methods Learn the skills
Collaborate with colleagues who have the skills
Consider less costly study design and measurement methods Seek additional financial support
Decrease sample size?
Not interesting, novel or significant
Modify the research question Uncertain ethical
considerations Consult institutional review board Modify research question
RESEARCH GOALS AND OBJECTIVES
After formulating your research question, the next step is to formulate your research objectives from the research question. The aims, goals, or objectives are the building blocks of a research proposal. They provide a picture of what you plan to accomplish in your research project.
The characteristics of a good research objective is also best remembered by the
mnemonics S-M-A-R-T (which stands for S-specific, M-measurable, A-attainable, R-realistic and T-time bound). This criteria should be applied to both the general and specific objectives.
General Objective
Usually expressed in a broad health care issue or statement. The statement should include the disease it wants to study, the aspect of disease that the project intends to change and the setting where the change will occur.
For example, a researcher wanted to investigate the use of quality of life as an outcome for the treatment of hypertensive patients in family practice, the general objective may be stated as:
“The general objective of this study is to improve the care and treatment of hypertensive patients in family practice.”
Disease to be studied Hypertension Aspect of disease to be changed Care and treatment
Setting Family practice
This general objective can also apply to a study that test a new anti-hypertensive medication that may offer better BP control , improved survival and lower side effects.
Avoid incomplete statements like these:
“Improve the treatment of hypertensive patients.”
“Investigate the quality of life of hypertensive patients.”
Another researcher who is interested in trying to validate the modified Prime MD, a screening instrument to detect psychological problems in family practice may state the general objective as:
“The purpose of this study is to improve the screening program for psychological problems in family practice.”
Disease to be studied Hypertension Aspect of disease to be changed Care and treatment
Setting Family practice
Avoid these:
“To validate the modified Prime MD.”
“To use the Prime MD as a screening tool in primary practice.”
In the example of the study on directly observed therapy the general objective can be stated as:
“This study aims to improve the treatment of tuberculosis in the community setting.”
In the example of the effect of education on the patient and family members, the general objective can be stated as follows:
“The general objective of this study is to improve the treatment of hypertensive patients.”
Specific Objectives
The specific objective involve a specific clinical question that is usually embodied in the research question. Some author write their specific research objectives based on the methodologic strategy like describing the population, measuring the outcome and analyzing the outcome as shown below.
Specific objectives for the study on TB compliance:
1. To diagnose patients with PTB based on chest x-ray or sputum examination.
2. To educate the patient or the patients family to act as surrogate observer for drug intake.
3. To measure compliance of drug intake in the two groups.
4. To compare drug compliance between the two groups.
Although the above specific objectives can be summarized into like;
1. To educate TB patients and their relatives to act as surrogate obsrver for drug intake.
2. To compare compliance to drug intake between those with surrogate observer and no surrogate observer.
It can also be stated as a single specific objective like;
1. To determine the effect of surrogate observer on compliance to treatment of TB patients.
There is no definite rule on how to formulate the specific objectives but it should follow the SMART criteria. Special attention however should be given to the action verb used in the statement of objective. The action verb must be clear and specific. The choice of the verb may actually be the basis of the statement fulfilling the SMART criteria. For example the action verb
“to know” may be more appropriately stated as “to determine”.
LECTURE II
METHODOLOGY I: OVERVIEW OF STUDY DESIGNS
PRIMARY STUDY DESIGNS Non-comparative or Descriptive studies
The word comparative here is used as a method of distinguishing between two populations at the start of the study. These studies are designed simply to describe certain characteristics of a problem. Cause and effect relationship is not being answered by this design.
The importance of this design is that it can be a source to generate hypothesis that can serve as a topic for future research using more complicated designs. A typical example of this design is the cross sectional study design.
Cross sectional studies
The essential feature of this type of design is that the cause and the outcome are measured at the same point time. There is no attempt to establish a temporal relationship between the cause and effect. They are basically descriptive and try to establish prevalence.
Example of cross-sectional studies would be a study to describe the prevalence of different diseases in population, or sex distribution among hemophiliacs or prevalence of use of tobacco among cancer patients.
Data is gathered using a structured questionnaire or data extraction form.
General steps in doing a cross-sectional study:
1. Select a sample from the population
2. Collect data using a standardized data collection method 3. Analyze data
Comparative Studies
These study designs compare the presence or absence of a cause or an outcome
characteristics between two groups. It is a better design to establish cause and effect relationship.
Observational Studies
These studies are designed to attempt to define the relationship between the outcome and its causes (cause and effect relationship). The outcome can be a development of a disease or cure of a disease and the cause can be a risk factor such as genetic predisposition, an
environmental exposure or unhealthy behavior like smoking, sedentary lifestyle etc. In this designs the researcher do not manipulate the exposure of the subject to the cause but only observe for their presence or absence. For example the researcher do not decide on the
environmental exposure, or who should have sedentary lifestyle or smoke. Manipulating them may be difficult or even unethical especially when the outcome being observed is potentially harmful.
There are two types i.e. case-control studies and cohort studies. They are described below.
Case-control Studies
In a case-control study, inclusion of subjects starts with defining or selecting those who have the outcome or effect. These are considered as cases. Then this group is compared with subjects who don’t have the outcome or effect. These are considered as the controls. Both the cases and the control should be taken from within the same population. Then the two groups are investigated as to the presence or absence of hypothesized causes or risk factors for the outcome.
Case-control study can be prospective or retrospective depending on the manner of patient recruitment. If recruitment is being done as cases develop forward in time it is
prospective, but if the cases have already developed in the past and patient recruitment is being done by reviewing existing clinical records then it is retrospective.
General steps in doing a case-control study:
1. Identify cases in a certain population
2. Identify controls from the same population matched to the cases based on certain characteristics
3. Collect data from both cases and controls 4. Analyze data
Example in the literature (Shapiro et al. Oral contraceptive use in relation to myocardial infarction. Lancet 1979; 1:743-747.)
A group of researcher examined the relationship between oral contraceptive and myocardial infarction. They selected 234 women who develop myocardial infarction (cases) and 1,742 women who did not have myocardial infarction (control). Then they were all questioned about history of oral contraceptive use. The results are shown below.
MI No MI
OC use 29 135
Non OC user 205 1,607
Total 234 1,742
A history of OC use was reported in 12.4% of cases and 7.7% of controls. We might say that there is an association between OC use and MI because if there is none the proportion of MI cases should be the same for both groups. The validity of the association however is dependenet on the manner of population sampling or selection and determination of exposure.
Cohort Studies
A cohort is any group of individuals who share the same characteristics. In a cohort study, selection of subjects start with identifying individuals who have the same characteristics or presence or absence of a particular cause or exposure. They are then divided into two groups, those with the characteristics or causes and those without the characteristics. They are then observed forward in time and determine who among them develop the outcome or effect.
Cohort studies can also be prospective or retrospective depending on the manner of patient recruitment. If recruitment is being done forward in time it is prospective, but if the cohort already existed in the past and data gathering is being done by reviewing existing clinical records then it is retrospective.
General steps in doing a cohort study:
1. Identify a group or cohort who have the exposure or characteristics of interest in a certain population.
2. Identify a group or cohort who don’t have the exposure or characteristics of interest.
3. Observe these two groups forward in time for the outcome of interest (data collection).
4. Analyze the data Example in the literature
The Framingham study is the longest cohort study to date. In 1948 the investigators took a random sample of 5,209 men and women from the general population in Framingham,
Massachusetts. Baseline characteristics were determined and re-examined every two years. The result of one of the sub-study is shown below.
Alive at 20 years Dead at 20 years % Dead
Non smokers 819 132 16.1
Smokers 1,489 333 22
Total 2,317 465 20.1
Thg results show that 22.2% of cigarette smokers died compared to only 16.1% of non- smokers within 20 years of follow-up. If there is no association between smoking and early mortality the observed proportion should be the same in both groups. Thus smoking increases mortality in the general population. The validity of this conclusion also depend on the selection of the population and determination of exposure and outcome.
Experimental Studies
In this studies, there is manipulation in the exposure to the cause to establish its relation with the outcome. Manipulation may be ethical when the outcome being studied is potentially beneficial, for example testing a new drug that has advantage over the old one. It is therefore not ethical to do experiments in humans if the experimenter wants to see the effect of different radiation exposure to radiology technicians.
True experiments are randomized controlled trials. Non-randomized or uncontrolled trials are considered quasi-experimental design.
Quasi-experimental Studies
Quasi-experimental designs are non-randomized, or non-comparative studies that involve observation of the effect of a particular intervention. In a non-randomized comparative study, the investigator choose two groups and assign each group to the two type of intervention by convenience. The outcome is then compared between the two groups. In a non-comparative study the investigator administer an intervention to a group of patients and watch the effect before and after the study. The outcome is then described before and after the intervention. Some authors consider non-comparative studies as case series, others as before-and-after design, but for the purpose of the classification we have adopted, we will label them as quasi-experimental study.
Example (Villamangca, D. Effects of Patient Education on the Quality of Life of Patients with Bronchial Asthma, Dept Internal Medicine, Rizal Medical Center)
An investigator assembled a group fo asthmatic patients and enrolled them to an asthma education program. The program consists of a series of group lectures and interactive discussions patterned after the comprehensive asthma education program of the Philippine General Hospital.
At the same time he also assembled a group of asthmatic patients and educate them in the outaptient clinic. He measured the quality of life of both groups before and after the educational intervention. The results of AQLQ after the educational intervention are shown below.
Education Program Usual Education
AQLQ overall 4.68 3.85
Symptoms 4.44 3.58
Activity 4.94 4.11
Emotional 4.77 3.58
Environment 4.6 4.29 The results showed higher scores for the intervention group but the difference were not statistically significant. Further analysis showed that baseline AQLQ in the usual education group was really low from the start compared with the intervention group. This is the usual problem with a non-randomized design.
Randomized Controlled Trial
This is the strongest design of all study designs. If done properly the result will surely be of highest validity and reliability. In this type of design, individuals are randomly assigned (randomization) to two or more groups, one with the exposure, intervention or cause the other without the exposure, intervention or the cause. Randomization try to make the two groups similar for both known and unknown factors that may affect the outcome other than the exposure, intervention or cause being tested. Then they are observed forward in time and their outcome compared. The outcome can be the cure of a disease, relief of symptoms or
improvement in quality of life.
In most but not all cases blinding is done. Blinding is the process in which the subjects, investigator and other personnel in the study is not made aware of the type of intervention the subject is recieving. This will be discussed in more detail later.
General steps in doing a clinical trial:
1. Select a sample from a population 2. Measure baseline variables
3. Randomize into intervention groups 4. Apply intervention
5. Measure outcome in both groups 6. Analyze data
Example in the literature
The TRACE Study was designed to determine improvement of survival among patients with LV dysfunction after MI. Patients were randomized to receive either trandolapril 1-4 mg for two years or to placebo aside from their usual medications. Deaths and their causes were noted in both groups during the follow-up period of two years. At the end of the study the results are presented below.
Trandolapril Placebo Overall deaths 304 (34.6%) 369 (42.3%)
Cardiovascular deaths 226 (25.8%) 288 (33.0%) Non-cardiovascular deaths
78 (8.9%) 81 (9.3%)
The results of the study showed that the all-cause mortality, cardiovascular and non- cardiovascular mortality were all lower in the trandolapril group. Trandolarpil therefore improve survival among patients who have LV dysfunction and post-MI. The validity of this conclusion depends on the selection and randomization of patients, manner of intervention and
determination of outcome.
SECONDARY STUDY DESIGNS
Meta-analysis
A meta-analysis is a procedure that integrates and combine the results of two or more primary studies that are similar in the population enrolled the intervention used and the outcome measured. The pooled result is then subjected to a statistical analysis. A well conducted meta- analysis allow a more objective appraisal of the existing evidence about a problem than a
traditional review or overview. It may also be biased owing to the inclusion or exclusion of some irrelevant or relevant studies respectively.
General steps in doing meta-analysis
1.Formulate the objective of the meta-analysis
2. Formulate the collection of data, data to be included and excluded 3. Collect and pool the data
4. Analyze the data
Economic Analysis
Economic analysis can be defined as an analysis that uses analytic techniques of from primary studies to define the choices in resource allocation.
In this design the cost of a particular intervention is estimated. Estimation include direct and indirect costs. There are three types of economic analysis depending on the type of outcome. If the outcome being considered is effectiveness of treatment, it is called cost-effectiveness analysis.
If the outcome is savings in terms of monitary units it is called cost-benefit analysis. If the outcomes are equal and the cost is the only one being compared it is called cost minimization.
CHOOSING A STUDY DESIGN
The choice of a study design entirely depend on your research question. If you want to describe characteristics of an interesting population, you might want to choose a cross-sectional survey. If you want to establish a cause and effect relationship you might do an observational study or if you want to compare effectiveness of two interventions then you can do an
experimental study. The algorithm in Figure 1 might be very helpful in deciding what research design to use.
Are you going to compare two or more groups?
Yes No Are you studying causation,
prognosis, treatment or diagnostic tests or just compare clinical characteristics?
Are you describing a single case or groups of cases?
Single case Groups of cases
Case Report Case Series Cross-sectional
Studies
Causation, prognosis, Treatment
Diagnosis, compare clinical characteristics Are you going to recruit
subjects forward in time?
Cross-sectional Studies
Yes No Are you going to assign
treatment or exosure?
Are you going to recruit those with exposure or
those with outcome?
Exposure Outcome
Retrospective Cohort Retrospective Case- control
Yes No Will you randomize? Are you going to recruit
those with exposure or those with outcome?
Yes No Exposure Outcome
Randomized
Controlled Trial Quasi-experimental Prospective Cohort Prospective Case- control Figure 1 Algorithm for Choosing a Research Methodology
WORKSHOP I Objectives of the workshop
At the end of the workshop the participant should be able to:
1. Identify a problem in his/her area of practice.
2. Formulate a research question and research plan for his/her idea.
3. Decide which of his/her research idea to undertake as his/her research project.
1. Enumerate at least 3 research ideas you are interested in.
1.
2.
3.
2. Focus your research ideas.
1.
2.
3.
3. Prioritize your research ideas.
1.
2.
3.
4. Translate the top research idea into research question.
1.
5. Formulate the general and specific objectives of your research question.
1. General Specific
LECTURE III
METHODOLOGY II: STUDY POPULATION,
RANDOMIZATION, AND SAMPLE SIZE ESTIMATION
STUDY POPULATION
The larger group to which the study results are to be generalized is called the target population. Thus if one is studying a new drug for the treatment of hypertension, the target population is all the hypertensive patients in the world. It is however impossible to study all of them thus we only choose a representative group. This is called the study or sample population.
The study population must be defined in the early stage of the study. It should be appropriate enough to attain the objective of the study. For the study population to be clearly defined, we should be guided by the following questions:
1. What or who should be the study population?
2. When and where should the study population be recruited?
3. How should the study population be selected?
What and who should be the study population
The exact characteristics of the study population must be defined. As much as possible it should correspond to the characteristics of the target population. Defining the population is usually stated as an inclusion and exclusion criteria. Table 4 illustrates how to formulate the inclusion and exclusion criteria.
When and where the study population be recruited
A statement indicating when and where the study population will be recruited is oftentimes necessary. A study on the treatment of hypertension may be recruited as stated:
“Patients consulting for hypertension at the Family Medicine Clinic of the Philippine General Hospital between the period January 1 to December 31, 1998 will be recruited for the study”.
How should the population be selected
The method of selecting the study population is called sampling. It can be categorized as probability sampling or non-probability sampling. Probability sampling is made through a process called random sampling where every individual in the population has the same chance of being included in the study. Random sampling should be distinguished from randomization which will be discussed later. Non-probability sampling is made by non-random methods. This is often used in clinical research because of the difficulty of identifying the target population.
Probability Sampling
Random sampling is a process whereby each unit in the population has the same probability of being chosen – chance alone will decide which of the unit will be included. The steps in doing random sampling is illustrated in Table 5.
Table 4 Formulating Inclusion and Exclusion Criteria Inclusion Criteria
(should be specific)
Target population
Accessible population
Specifying the characteristics that define populations that are relevant to the research
question.
Demographic characteristics Clinical characteristics
Geographic
Temporal characteristics
A trial of calcium supplementation for preventing osteoporosis might specify that the subjects should be:
Females, age 45-50
In good health with no known life threatening illness, not previously diagnosed to have osteoporosis and no history of neurologic deficit or taking corticosteroid
Patients attending the medical clinic at the investigator hospital
Between January 1 and December 31, 1998
Exclusion criteria
(be parsimonious) Specify subsets of the population that will not be studied because of:
A high likelihood of being lost to follow-up
Inability to provide good data
Ethical barriers
The subject’s refusal to participate
The calcium supplementation trial might exclude subjects who are:
Alcoholic or plan to move out of the city or country
Disoriented or have a language barrier
Kidney stone formers
(contraindicates oral calcium) Unwilling to accept random allocation to placebo
Table 5 Steps in Doing Random Sampling
Step 1 Prepare the sampling frame, usually a list of the target or accessible population Step 2 Decide on the size of the sample
Step 3 Get a table of random numbers, arbitrarily select a starting point then get a series of random numbers equal to the sample size. The numbers in the series that correspond to the list are the subjects of the study. This process is called simple random sampling.
Another way is by systematic random sampling where the total number in the list is divided by the sample size to get the sampling interval. Randomly select a starting point in the list and include the subject in the list at every sampling interval. This is equal to simple random sampling as long as the list is not arranged in a particular order.
Special Types of Random Sampling
Stratified random sampling – the accessible population is first divided into non- overlapping strata such as range of age, sex, economic status etc. In each strata subjects are randomly selected either by simple or systematic random sampling.
Disproportional sampling – if different strata have different size, sampling can be done whose size is proportional to the size of the strata that may lead to different sizes between different strata.
Cluster sampling – sampling is done from similar units of the population like 5 clusters of provinces in the Philippines. This can also be done at several stages or multi-stage cluster sampling. For example we want to study the prevalence of TB in the Philippines and we need 1,500 barangays, we can start with a cluster of 30 provinces, a cluster of 5 towns in each province then a cluster of 10 barangays in each town selected.
Non-probability Sampling
The most common non-probability sampling is convenience sampling. In this method subjects are chosen based on their availability i. e. during the time of consultation, those who respond to the announcement or those who are near the study center etc. Purposive sampling is another common method wherein a researcher selects a certain subject because they fulfill some specific criteria. In snowball sampling, a subject who was already included are asked to identify others who also have the same requisite characteristics. Stratified non-probability sampling can also be done by quota sampling where volunteers are called to join the study and stop
recruitment in each strata once the proper size is achieved.
RANDOMIZATION
As previously mentioned the randomized controlled trial is the design considered as the
“gold standard” in clinical research. Randomization is an essential feature of such design. It is defined as the process in which each subject is given the same chance of being assigned to the different study groups. The purpose is to make the groups comparable with respect to known and unknown variables that might affect the outcome of the study.
Several methods of randomization is available, but in this module we will only discuss randomization with fixed allocation. This means that the randomization will not be alter as the study progress.
Simple Randomization
The simplest method of randomization is by a toss of a coin. If the coin turns up heads the subject is assigned to A, if the coin turns up tails the subject is assigned to B.
Another simple randomization procedure can be done using the table of random numbers. The steps in doing randomization using the table of random numbers are described in Table 6.
Table 6 Steps for Randomization Using the Table of Random Numbers
Step 1 Decide what group to assign odd numbers and even numbers and how to generate a series of random numbers in series or in columns
Step 2 Get a table of random numbers and point with a pencil where to start while being blindfolded (another way is to use a series of seed numbers)
Step 3 From the starting point get the sequence of random numbers equivalent to the expected sample size in the study
Step 4 Assign the groupings whether the number is even or odd
The use of computer generated randomization can also be done using the software called RALLOC.
Simple randomization is easy to implement but may result to unequal sample sizes in the study groups. This problem can be solved by blocked randomization.
Block Randomization
Blocked randomization is used to avoid imbalance in the number of subjects assigned to each group. If 20 subjects will be randomized to two groups using simple randomization might result to 12 subjects being assign to one group and 8 to the other. In blocked randomization each block, say 4, is designed to have equal number of A and B by enumerating the possible
combinations of 2 A’s and 2 B’s (AABB, ABAB, ABBA, BAAB, BABA, BBAA). The combinations are then selected at random until all 20 subjects are randomized (see Table 7).
Table 7 Block Randomization of 20 Subjects to Two Groups in Blocks of 4
Step 1 Number the following combinations of A and B in blocks of four as shown below.
1 2 3 4 5 6 A A A B B B A B B A A B B A B A B A B B A B A A
Step 2 Randomly select five sequence of numbers from 1 to 6, say 3, 5, 1, 6, 1.
Step 3 The randomization therefore is as follows
Subj Assign Subj Assign Subj Assign Subj Assign Subj Assign 1 A 5 B 9 A 13 B 17 A 2 B 6 A 10 A 14 B 18 A 3 B 7 B 11 B 15 A 19 B 4 A 8 A 12 B 16 A 20 B
Stratified Randomization
In stratified randomization, prognostic factors that may affect the outcome such as age, severity of illness etc. are identified. They are then divided into different strata such as less than 20 as the first strata, 20-50 as the second strata, and greater than 50 as the last strata. The subjects in each strata are then randomized (simple or blocked) to their group assignments.
SAMPLE SIZE ESTIMATION
The formula to be used for estimating the sample size may depend on the type of the study and the outcome to be measured. For the purpose of simplification, we will discuss computation primarily based on the type of outcome being measured and whether we are just describing a single population or comparing two populations. Only the formulas and the needed data to compute for the sample size will be presented.
Tables 8 and 9 are values of Za and Zb. Table 8 Values for (Za + Zb)2
Alpha = 0.05
Beta 1-tailed 2-tailed
0.5 2.71 3.84 0.2 6.18 7.85 0.1 8.56 10.51
Table 9 Values of Za at Different Confidence Levels
Confidence Level
0.80 0.90 0.95 0.99
Za 1.28 1.64 1.96 2.58
Note Value of Zb at 0.10 is 1.28
Sample size estimation to describe a single population, rate or proportion as the outcome measure:
N = Za2 PQ d2 Where:
N – sample size needed
Za – value of Z in normal distribution at desired alpha level
P – estimated rate or proportion of the population with the outcome Q – estimated rate or proportion of the population without the outcome or (1-P)
d – maximum tolerable error
Example
A community physician wanted to estimate the prevalence of TB among schoolchildren in a certain community. Previous study showed that the prevalence was 20%. He wanted to estimate the present prevalence with a desired precision of 5% and confidence level of 95%.
N = (1.96)2 (.2)(.8) (.05)2 N = 246
Sample size estimation to describe a single population with mean as the outcome measure:
N = Za2 S2 d2 Where:
N – sample size needed
Za – value of Z in normal distribution at desired alpha level S – standard deviation of the mean of variable being studied d – maximum tolerable error
Example:
An obstetrician wanted to determine the mean hemoglobin of pregnant patients consulting at the local health center during their first pre-natal visit. She wants an estimate of at least within 2 units of the true value of hemoglobin with a confidence of 90%. Previous survey showed the standard deviation of hemoglobin among pregnant patients during their first pre- natal visit was 10 grams %.
N = (1.645)2(10)2 (2)2 N = 68
Sample size estimation to compare two populations with rate or proportion as the outcome measure:
N = Za2 2PQ + Zb2 (P1Q1+P2Q2) (d)2
Where:
N – sample size needed per group
Za – value of Z in normal distribution at desired alpha level
P – estimated mean rate or proportion in the two population with the outcome or (P1/2+P2/2)
Q – estimated mean rate or proportion of the population without the outcome or (1-P)
P1 – estimated rate or proportion in the first population with the outcome Q1 – estimated rate or proportion in the first population without the outcome or (1-P1)
P2 – estimated rate or proportion in the second population with the outcome
Q2 – estimated rate or proportion in the second population without the outcome or (1-P2)
d – maximum tolerable error Example:
A researcher wants to determine the response rates of patients with CHF taking diuretics alone compared with trandolapril plus diuretic. The investigator want to detect a 20%
improvement in response from the 60% who respond favorably to diuretics alone based on previous studies. He wants to detect this at the level of significance of 0.05 (1-tailed) and a power of 90%.
Computation (exercise)
Sample size estimation to compare two populations with mean as the outcome measure:
N = 2(Za+Zb)2 S2 d2 Where:
N – sample size needed per group
Za – value of Z in normal distribution at desired alpha level Zb – Valuae of Z in normal distribution at the desired beta level S – standard deviation of the mean of variable being studied d – maximum tolerable error
Example:
An investigator wants to determine the effectiveness of a new anti-asthma agent (Asmalin inhaler) using improvement in PEFR as the outcome. Previous study showed that the variation in PEFR measurement in the ER was 12 L/min. He wants to detect a difference of at least 5 L/min between the new drug and the old standard at alpha of 0.05 and a power of 90%.
Computation (exercise)
WORKSHOP II Objectives of the workshop
At the end of this workshop the participant should have:
1. Written the contents of the methodology section (Subjects, Subject selection, Randomization, Intervention) of his protocol by completing the answers to all the questions below.
1. Based on your research question and objectives, describe in general your planned methodology.
Describe the population you plan to include in your study.
Inclusion criteria
A. Exclusion criteria
2. Describe how you plan to recruit your subjects.
3. Compute for your sample size.
4. If randomization will be done, describe how you will randomize your subjects.
5. Describe the exposure, treatment or diagnostic tests you plan to observe or compare.
Experimental
Control
LECTURE IV
METHODOLOGY III: DATA COLLECTION AND ANALYSIS
DATA COLLECTION
This section will discuss the process of collecting data. The lecture is divided into subsections that are important in collecting accurate and reliable data.
Data Collection Forms
Before data collection the researcher must review the objectives of the study and the design. Consultation with the statistician to design data collection forms that is adapted for the planned statistical analysis will always be helpful. Before designing data collection forms it might be wiser to use existing data collection forms that have withstood pre-testing and modifications.
Investigators should take special care in developing these forms so that data are complete and accurate. Before designing the form, the investigator should make an outline of all the data that are to be collected. Each page in the data collection form should contain an identifying mark for each subject. The data must be organized (modular) for entry into the computer software that is planned to be used. Modular structure means separating the data collection form into sections like demography, history, physical examination, laboratory testing, test drugs, other outcomes to be monitored (quality of life), adverse events etc. Having such ready made data collection forms improves efficiency and cost of reproduction.
Some rules in designing data collection forms:
1. collect data pertinent to the study objectives
2. be simple, easily understood, short as possible and yet comprehensive
3. use forms that can be completed with a check mark rather than requiring fill-in 4. implement easy to use automation system for data entry, editing and analysis I hope you’ll find helpful the sample data collection forms in Appendix I.
Blinding
Bias can occur at different stages of the research project. It can occur at the time of assignment to treatment groups. Bias in this case is reduced by randomization. It can also occur during data collection or observation. Bias in the second situation is reduced by blinding.
Blinding is the process of concealing the treatment or intervention to the patient or investigator.
As much as possible clinical trials should be double blind, meaning both the investigator and the patient is not aware of the treatment. But if it is impossible as in surgical trials, a single blind approach or other methods to reduce bias like concealing the intervention to the evaluator or the statistician can be done. Table 10 enumerates the types of blinding you can use.
Table 10 Types of Blinding
Open label Both the subject, the investigator and other members of the research team know the treatment assignment
Single blind Only the subject is not aware of the treatment assignment
Double blind Both the subject and the investigator is not aware of the treatment assignment
Triple blind This is just an extension or modification of the double blind design, where aside from the investigator and subject, other persons involved in the study like the monitoring team, data encoders and statisticians are also not aware of the treatment assignment
Making a Questionnaire
A questionnaire is a measurement instrument for assessing individual’s attitudes, beliefs, behavior or attributes. It is important that the researcher should establish first hand what he wants to measure. Measuring attitude is often difficult because they are very sensitive to variations in words. Attributes on the other hand is less sensitive to wording variation. Before constructing your questionnaire be clear about its purpose and the information you want to get.
Make sure that a questionnaire is the best possible method to get that information.
A questionnaire can be open-ended or close-ended. Close-ended questions limit the responses and the information to be collected to the available choices. The advantage is that coding and analyses are often easy. To avoid losing information it is best for close-ended questions to include all possible answers. If this is not possible include “others and specify” as one of the choices. Open-ended questions allow the researcher to obtain greater information especially on attitudes and opinions. But coding and analyses are often difficult. The most appropriate questionnaire will probably be a combination of both open and close-ended questions.
When constructing your questionnaire make sure that your grammar is correct and the wordings are simple. It should be targeted to the lo est educational level of potential
respondents. Sequence questions into logical order and group them into topics to make the respondents’ task pleasurable. Keep the questionnaire as short as possible. After constructing the questionnaire give it to your peers for comments. Pre-test it to a sample of possible respondents.
When changes are made pre-test the revision until you have decided on the trial questionnaire.
If you don’t intend to do these don’t gather your data by means of a questionnaire! Table 11 presents the listing of problems associated with use of questionnaires.
Table 11 Common Problems in Questionnaire Construction and Use Lack of attention by the
researcher
The construction of a good questionnaire requires a high level of experience and skill and is not to be undertaken lightly without extensive planning and consultation.
Ethics The respondents’ privacy and dignity should be respected.
Respondent characteristics Factors like age, sex, literacy and educational level may influence questionnaire results and their reliability.
Response error Response error may arise from failure of memory, motivation, communication and knowledge. In other words respondents may not answer correctly and accurately.
Response bias Social desirability bias is present when patients try not to offend other people or try to respond to behavior questions that are socially desirable when it may not be the real behavior of the respondent.
Return rate Low return rates often affect the reliability of the study.
Asking threatening or personal questions
Threatening questions may cause the respondents to be embarrassed and feel uncomfortable. This may also affect the accuracy of the responses.
Measurement of Study Variables
Measurement is defined as the process of assigning numerals to objects to represent quantities of their characteristics according to certain rules. We assign numerals to responses to a questionnaire like “0” for a no answer and “1” for a yes answer. We can also assign numerals to severity of symptom or illness like “0” for none, “1” for mild, “2” for moderate and “3” for severe. In some cases numeral assignment is very evident like for height and weight, age, blood pressure, temperature etc. This assignment must follow certain rules that must be set by the researcher. Thus a pediatric researcher might want to measure age in months but an adult medicine researcher might want it measured in years. Some might measure weight in pounds while others in kilograms. These assignment of numbers and rules must be explicitly set by the investigator.
Measurement is used in research to describe the quality or quantity of an existing variable i. e. the characteristics of what we want to observe. For example, if we want to analyze demographic characteristics in terms of age we measure it in number of years, if in terms of education we measure it in educational level (0 for none, 1 for elementary, 2 for high school, 3 for college etc.), if in terms of economic status, we measure it in terms of annual income.
Measurement can also help us make clinical decisions when we measure and compare
effectiveness of two alternative drugs or diagnostic tests. Finally we use measurement in research to help us draw some conclusion about the relation between two variables. This relation may be an association or difference.
Levels of Measurement
At the start of the study the investigator decides on what outcome to observe and how it will be measured or recorded.
There four different levels of measurement namely nominal, ordinal, interval and ratio.
Each level needs a different kind of analysis.
1. Nominal data – these are qualitative data which are mutually exclusive and
exhaustive and do not mean hierarchy. Numerals represent category or classification labels only. For example name, sex (male does not mean being better than female), address etc.
2. Ordinal data – reflects a rank order among the categories used to measure a variable, example is economic class (low, middle, upper), scale of severity (none, mild,
moderate, severe, very severe).
3. Interval measures – have more meaning than ordinal measures because the differences between two categories are known and definite. The intervals between numbers are equal but not related to true zero. They do not represent true quantity.
For example temperature, calendar years etc.
4. Ratio scale – Numbers represent units with equal intervals measured from true zero.
For example distance, weight, blood pressure etc.
Coding
Coding is the process of assigning numbers to answers or data collected for data entry and analysis. It must be differentiated from measurement where the purpose is to quantify characteristics. In pre-coding the data or response is coded before collection. This is applicable if the responses are known in advance and the questions are close-ended. It is more efficient because it makes data entry and analysis easier. However it may not be applicable to open-ended questions. In open-ended and complex questions, post-coding is usually done. In post-coding numbers are assign to answer categories after they have been collected.
Analysis
Statistical analysis should be planned before data collection. Since the type of data to be collected or measurement to be done is already planned prior to collection, statistical analysis can be planned in advance as well. The analysis must be simple as much as possible. Complex analysis might be impressive but interpretation becomes difficult. Consult a statistician (familiar with clinical research) for advice on the appropriate statistical analysis for your data. Make use of existing computer packages to analyze data.
General classification of statistical techniques commonly used in clinical research.
1. Correlation coefficients – a measure of the strength of relationship between two variables
2. Linear regression – given that a strong relationship exists between two variables, this technique allows the prediction of one variable from a given values of another variable
3. T-tests – compares the variance (spread of scores) of values obtained for two
variables – indicates whether the two samples or variables are significantly different 4. Analysis of variance – a more complex form of the t-test, where the variation in one
variable is broken down and attributed to other variables selected for the study
5. Factor analysis – a method of statistically grouping items which are answered by the respondents in similar ways
We will not go farther in discussing how to compute for these statistical techniques. Try to find the algorithm in Figures 2 to 9 helpful in deciding what statistical techniques you will use in analyzing your data.
Sample Population
Difference Association
Measure the size of the difference See Figure 3
Testing for statistical significance of
the difference
Measure of the degree of the
association See Figure 4
Testing for statistical significance of the association See Figure 5
Extent association
explains variations between groups
See Figure 6 Nominal Data
See Figure 7 Ordinal Data
See Figure 8 Continuous Data See Figure 9
Figure 2 Starting Point When Deciding What Statistical Techniques to Use
Measuring the size of the difference
Nominal or Ordinal Data
Descriptive Statistics Continuous Data
Descriptive Statistics Figure 3 Measuring the Size of the Difference
Measure degree of association Nominal Data
Odds Ratio or Relative Risk
Ordinal Data or when no linear relationship is
suspected
Spearman’s Rho or Kendall’s Tau
Continuous data when a linear relationship is suspected
Pearson’s Correlation Coefficient
Figure 4 Measurement of Degree of Association
Testing for statistical significance of association
Nominal Data Statistical Significance of Odds Ratio or Relative Risk
Ordinal data or when no linear relationship is
suspected
Statistical Significance of Spearman’s Rho or Kendall’s
Tau
Continuous data or when a linear relationship is
suspected
Statistical Significance of Pearson’s Correlation
Coefficient Figure 5 Testing for Significance of Association
Extent of association explains variation between groups
Nominal data
Attributable Risk Ordinal data or when no linear relationship is
suspected
Spearman’s Rho2 or Kendall’s Tau2
Continuous data or when linear is suspected Pearson’s Coefficient of
Determination (R2) Figure 6 Extent of Association Explains Variation Between Groups
Nominal Data
Small unmatched sample Fisher’s Exact Test
Small matched sample Sign Test
Large unmatched sample Chi-square with Yates Correction
Large matched sample Mac Nemar’s Test
Figure 7 Testing for Significance of Difference Between Two Nominal Data
Ordinal Data 1 comparison
(2 groups)
>1 comparison (>2 groups) Unmatched sample
Mann-Whitney U or Median Test
Matched sample Wilcoxon Matched
Pairs or Signed Ranks Tests
Unmatched sample Kruskal-Wallis one- way Analysis of
Variance
Matched sample Friedman two-way Analysis of Variance
Figure 8 Testing for Significance of Difference Between Two Ordinal Data
Continuous data 1 comparison
(2 groups)
>1 comparison (>2 groups) Unmatched sample
Student’s Unpaired T-test
Matched sample Student’s Paired T-
test
Unmatched sample F-test for Analysis of Variance followed by Pair-wise comparison
Matched sample F-test for Analysis of
Variance with Blocking or Analysis
of co-variance
Figure 9 Testing for Significance of Difference Between Two Continuous Data
The Consent Form
Informed consent is a very important factor for justifying your research. Thus it is very important that the consent form be designed according to ethical standards (amended Helsinki declaration). The form should be written in understandable language and contain the following:
1. A statement of the purpose of the study
2. Description of procedures both experimental and routine 3. Duration of the subject’s involvement in the study
4. Whom to contact in terms of adverse events or additional questions 5. Risks and discomfort associated with participation in the study
6. Alternative appropriate treatment available in place of experimental treatment 7. Benefits the subject may expect from participation in the study
8. A statement that participation is voluntary 9. A statement guaranteeing confidentiality
10. A statement regarding compensation due to adverse events
LECTURE V
WRITING THE RESEARCH PROPOSAL
The initial stages and often the most difficult part of the research process is the development of the research proposal. The proposal describes the purpose of the study, the importance of the research question, the methodology and justifies the feasibility of the project.
The proposal serves several purposes:
1. represents the synthesis of the researcher’s critical thinking and the scientific literature to ensure that the research question is refined enough to be studied 2. serves as an application for review by peers, administrative committees or funding
agencies
3. enhance communication among colleagues who may be co-investigators 4. serves as guide throughout the study to ensure that the researchers follow the
outlined rules of conducting the study
General Rules in Writing the Proposal
These are the basic grammar guidelines in writing a research proposal.
Word Choice
The words chosen should be simple, precise, necessary and familiar. Highly technical and scientific terms should be used less often and only when necessary i.e. no other familiar word with the same meaning. Avoid jargon or inventing new words by adding suffixes or prefixes to familiar words. Use few abbreviations as least as possible.
Wrong word choice Suggestions
Renal blood flow is drastically compromised if the aorta is obstructed.
greatly reduced or reduced by more than 50%
The change in current produced by M major protein was greater than 85%of the maximal response to isoproterenol.
increase
Infusion of serotonin was associated with an increase in microvascular pressure.
resulted to or led to After 4 hours of hemodialysis, we abruptly
ended the hemodialysis procedure.
of hemodialysis and abruptly can be removed i.e. After 4 hours, we ended the hemodialysis proceedure.
Heat stable materials will be utilized in the isolation and processing of samples.
used
We endorphinized the dogs. injected endorphins to
Sentence Structure
Use simple and direct sentences. This can be done if the core or the message is conveyed in a simple sentence structure i.e. subject, verb and predicate or completer. The topic should be in the subject and the action in the verb.
Another common mistake is the piling of nouns into noun cluster or putting too many ideas in one sentence. A sentence should only talk about one thing at a time. Aim for a mean sentence length of no more than 20 words per sentence.
Wrong sentence Suggestions The children with arteriovenous shunts had
their shunts opened and heparin injected.
The shunts of children arteriovenous shunts were opened and heparin was injected
The new drug caused a decrease in heart rate. The new drug decreased heart rate.
Noun clusters Suggestions
filament length variability variability of filament length peripheral chemoreceptor stimulation stimulation of peripheral stimulation Paragraph Structure
The paragraph should convey an organized idea and the continuity of these ideas must be clear. To do this a paragraph should have a definite structure. It should be started with a topic sentence followed by a series of logically arranged supporting sentences.
Sample Discussion (A) There are three different theories put
forward for the very slow relaxation of catch muscles of molluscs. (B) One theory holds that catch is due to some unusual property of myosin in these muscles that produce a slow rate of detachment. (C) In this theory
paramyosin would have no special role beyond that of providing the long scaffolding on which the myosin is positioned as well the mechanical strength for the large tension developed. (D) The second theory holds that tension is developed by actinomysin interaction but is maintained by paramyosin interactions. (E) Because the thick filaments are of limited length, interactions have to occur through fusion of thick filaments. (F) A third theory, to which I subscribe, pictures a structural change in paramyosin core affecting the rate of breaking of myosin-actin links at the filament surface.
A – topic sentence B – States first theory
C – Explain first theory
D – States second theory
E – Explain second theory F – States third theory
