Quantitative data are numbers that have been collected via surveys, logs, attendance records, or other methods. In quantitative analysis, data can be summarized by computing totals, percentages, and averages (means).
A variety of statistical approaches can be used to answer evaluation questions; some are more complex than others. Data from demographic, medical history, behavioral risk, and participant response surveys can be analyzed with simple descriptive statistical analyses (e.g., frequencies [modes], averages [means], and midpoints [medians]) and cross-tabulation procedures (i.e., comparison of a response or outcome by two or more subgroups). Simple statistical analysis of survey data can be conducted using spreadsheet software, as well as other statistical software packages. Exhibit 3.13 presents some software that may be used for quantitative data analysis.
The analysis plan should specify the quantitative analysis techniques that will be used and include details on how you will perform each of the following tasks:
• Entering the data into a database and checking for errors, including the data systems and software that will be used.
• Tabulating the data for each measure (e.g., providing counts/number, percentages, specifying the denominator).
• Stratifying your data by various demographic variables of interest (e.g., participants’ race, sex, age, income level, geographic location).
• Making comparisons (e.g., between testing venues, between geographic areas) and the relevant statistical tests that will be used.
• Looking at your analyzed data over time to see how your results change by tracking the measures. Results that are not changing in the desired direction can alert you to look more closely at the program and work with stakeholders to improve it.
Exhibit 3.13. Software Resources for Quantitative Data Analysis • • So • • •
Quantitative Data Entry and Analysis Software Resources
Software for Basic Quantitative Data Entry and Analysis
Microsoft Excel—http://www.office.microsoft.com/excel Microsoft Access—http://www.office.microsoft.com/access
ftware to Run Inferential Statistical Analysis
SPSS—http://www.spss.com SAS—http://www.sas.com Stata—http://www.stata.com
Exhibit 3.14 displays the data elements and calculations that are needed to answer the two example M&E questions.
Exhibit 3.14. Example Quantitative Calculations Needed •
• • •
Monitoring Question Data Elements Required Calculation Needed
Among all test results, how many were provided to persons who received HIV testing in the past quarter?
Numerator: Number of HIV tests returned to clients in the past quarter Denominator: Total number of HIV tests conducted in the past quarter
Tally Division What percentage of persons
testing HIV-positive who are referred to medical care attend their first medical appointment?
Numerator: Number of HIV tests returned to clients in the past quarter Denominator: Total number of HIV tests conducted in the past quarter
Tally Division
Qualitative Data Analysis
20Qualitative data are generally presented through words and text organized around concepts, themes, or patterns. During quantitative studies, data collection and analysis can be separate processes. In qualitative studies, however, data analysis begins during data collection and includes preliminary thoughts or conclusions during initial observations and interviews. Data analysis continues throughout the entire evaluation, including during coding and analysis. A common method used to analyze qualitative data is coding, which is the process used to distill raw field notes into usable findings. Coding consists of using a code word to label pieces of the text by selecting, focusing, and simplifying notes into summaries organized around themes or patterns, generally on the basis of the evaluation question(s).
During coding, the evaluator must keep a master list, also known as a codebook or code scheme, of all the codes that are developed and used and their definitions. The evaluator can modify the master list as the reading of notes progresses.
Increasingly, computers are being used to help with qualitative analysis. The data analysis plan should consider whether data will be analyzed manually or with analysis software and specify the qualitative analysis methods that will be used. Exhibit 3.15 presents some software packages that may be used to analyze qualitative data.
20 For more information on qualitative data analysis techniques, please refer to Patton, M. Q. (2008). Utilization-
Exhibit 3.15. Software Resources for Qualitative Data Analysis • • • • • ▪ ▪ ▪
Qualitative Data Entry and Analysis Software Resources
Software to Organize Themes and Codes
Microsoft Word—http://www.office.microsoft.com/word
Microsoft Excel—http://www.office.microsoft.com/excel (to manage key quotes)
Software to Conduct Qualitative Analysis
AnSWR—http://www.cdc.gov/hiv/software/answr.htm (free CDC software that helps to coordinate and conduct large-scale, team-based analysis projects that integrate qualitative and quantitative techniques) EZ-Text—http://www.cdc.gov/hiv/software/ez-text.htm (free CDC software that helps to create, manage, and analyze semistructured qualitative databases)
Commercially available qualitative data analysis packages
NVivo—http://www.qsrinternational.com/products_nvivo.aspx ATLAS.ti—http://www.atlasti.com/
Ethnograph—http://qualisresearch.com/
Exercise 11: Completing Your Data Analysis Planning
Template
Now that we have reviewed the types of analyses you may wish to conduct for your non- clinical HIV TLC program, you can complete your data analysis plan. For each prioritized M&E question and measure, the plan should list the analysis procedure, time, and person responsible for managing the data. Be sure to review the plan with relevant stakeholders.
Tools and Templates: Tool 12—Data Analysis Planning Template
Once you have determined your data collection methods, where you will gather information, who will collect data and when, answer the questions on this tool to help you develop the data analysis plan for the data.
Instructions for Completing Tool 12. Data Analysis Planning Template
What is the purpose of this tool? This table will help you document monitoring questions,
corresponding analysis activities, and individuals who will answer each.
Who should complete this tool? Program managers or coordinators may choose to
complete this table individually or as a group activity with program staff. If completing individually, make sure to gather input from HIV testing staff.
When should this tool be completed? Complete this tool in the beginning of your M&E
process and revisit it each time you review your M&E plan or if there are significant staff changes. This may be yearly or more frequently if necessary.
How should this tool be completed? For each M&E question and measure, decide how to
analyze the data (i.e., for quantitative data, consider generating frequencies, averages, proportions or percentages; for qualitative data, you may code the text by themes and identify patterns). Next, include for each an analysis procedure (quantitative or qualitative) and the timeline schedule (e.g., survey data frequencies will be completed by mm/dd/yyyy). Finally, in the last column, document the roles and responsibilities of all individuals involved. This will help you to estimate the workload of each individual and develop a reasonable timeline for the analysis of your data.
Example:
•
•
Evaluation Question: To what extent are test results being provided to clients who received HIV testing?
Monitoring
Question Measure Analysis Procedure Analysis Timeline Responsible Person(s)
Among all test results, how many were provided to persons who received HIV testing in the past quarter? Proportion of test results provided to persons who received HIV testing in the previous quarter. Perform quantitative analysis through a division of the following: Numerator: Number of HIV tests returned to clients in the past quarter Denominator: Total number of HIV tests conducted in the past quarter
Proportions will be calculated within 30 days of the end of the quarter.
Data analysis tasks will be coordinated by the lead data coordinator.
Tool 12. Data Analysis Planning Template
Evaluation Question:
Monitoring
Question Measure Analysis Procedure Analysis Timeline Responsible Person(s)
Evaluation Question:
Monitoring