A
N
A
PPROACH TO
C
LINICAL
D
ATA
M
ANAGEMENT FOR THE
D
OCTOR OF
N
URSING
P
RACTICE
C
URRICULUM
MARTHA SYLVIA, PHD, MBA, RN
⁎
ANDMARY TERHAAR, DNSC, RN
†
Strong data management skills are essential to doctor of nursing practice (DNP) education and necessary for DNP practice. Completion of the DNP scholarly project requires application of these skills to understand and address a complex practice, process, or systems problem; develop, implement, and monitor an innovative evidence-based intervention to address that problem; and evaluate the outcomes. The purposes of this paper were to describe the demand and context for clinical data management (CDM) within the DNP curriculum; provide an overview of CDM content; describe the process for content delivery; propose a set of course objectives; and describe initial successes and challenges. A two-pronged approach of consultation and a CDM course were developed. Students who participated in this approach were more likely to create and implement an evaluation plan; apply techniques for data cleansing and manipulation; apply concepts of sample size determination using power analysis; use exploratory data analysis techniques to understand population attributes and sampling bias; apply techniques to adjust for bias; apply statistical significance testing; and present project results in a meaningful way. On the basis of this evaluation, CDM has evolved from an elective to a required course integrated in a thread that crosses the entire curriculum. (Index words: Clinical data management; Doctor of nursing practice; Curriculum development; Data analysis) J Prof Nurs 30:56–62, 2014. © 2014 Elsevier Inc. All rights reserved.
S
INCE THE INCEPTION of the doctor of nursing practice (DNP) degree, a fundamental understanding has evolved within the profession that the DNP is designed for innovation rather than discovery. As a result, curricula focus not on research methodologies but on processes that will help the DNP to innovate in practice using evidence-based methodologies designed to improve health care quality, safety, efficiency, delivery, and accessibility (American Association of Colleges of Nurs-ing [AACN], 2006). However, because the distinction between the PhD and the DNP has traditionally focused on “academia/research” versus “practice/research appli-cation,”DNP programs may be downplaying the need forstrong data management skills and knowledge within the curriculum (Algase, 2010; AACN, 2006; Webber, 2008). The purposes of this paper were to describe the demand and context for clinical data management (CDM) within the DNP curriculum; provide an overview of CDM content; describe the process for content delivery; propose a set of course objectives; and describe our initial successes and challenges with implementation.
Background
It is generally accepted that the role of the practicing DNP includes translating evidence into practice on a systems, large-scale level, with the result being improvement in quality of health-related outcomes (AACN, 2006; Mun-dinger, Starck, Hathaway, Shaver, & Woods, 2009). The very use of the word improvement connotes understand-ing of how todefine,measure,analyze, anddemonstrate improvement. All require an understanding of data management techniques.
Within the core competencies outlined by AACN (2006) lie multiple content areas that require data management knowledge and skills such as quality improvement, program evaluation, cost and clinical
∗Assistant Professor, Johns Hopkins University School of Nursing, Baltimore, MD 21205–2110.
†Associate Professor, Johns Hopkins University School of Nursing, Baltimore, MD 21205–2110.
Address correspondence to Dr. Sylvia: Assistant Professor, Johns Hopkins University School of Nursing, Baltimore, MD 21205–2110. E-mail:[email protected]
8755-7223/13/$ - see front matter
Journal of Professional Nursing, Vol 30, No. 1 (January/February), 2014: pp 56–62 56
© 2014 Elsevier Inc. All rights reserved.
effectiveness evaluation, translational science, epidemi-ology, biostatistics, economics, financial management, risk management, population health management, pre-vention management, and designing, selecting, and using data collection systems. While AACN makes clear that DNP students are not required to demonstrate all competency requirements, one competency for all areas is strong data management knowledge and skills.
Typical DNP curricula offer three to six credits of statistics and/or research focused on analytic methods for advancing evidence-base practice (Mundinger et al., 2009; Wall, Novak, & Wilkerson, 2005). Content for these courses likely include critical appraisal of existing literature to identify and address a practice problem with emphasis on the skills necessary to evaluate rigor, design, and analysis of relevant research. However, the work of the DNP requires analytical and methodological skills and knowledge well beyond the ability to appraise existing literature. DNPs are faced with the challenges of answering data-driven clinical questions using pre-existing data sets within complex collection systems, which are often collected for purposes other than designing, implementing, and evaluating evidence-based changes in practice. In addition, DNP project evaluation commonly uses observational study techniques that require complex statistical methods to eliminate sam-pling biases that would be negated using controls in randomized clinical trials (Austin, 2011).
Table 1displays the similarities between data manage-ment processes in three of the DNP domains and compares them to data management processes in the non-DNP domain of knowledge-generating interventio-nal research, which falls within the domain of the doctor of philosophy. As a reference, standard textbooks in each of these domains were used to describe the several key data management functions performed in each, which include data collection, cleaning, manipulation, statistical analysis, and reporting (Rondel, Varley, & Webb, 2000). The first domain is the identification, stratification, and assessment of populations of interest. In this domain, complex algorithms using various integrated data sources are used to identify patients with certain conditions; predictive models may be developed to understand determinants of adverse outcomes such as hospitaliza-tions; and a risk level can be assigned to individual patients for stratification into appropriate intervention intensity (Gordis, 2009; Nash, Reifsnyder, Fabius, & Pracilio, 2011). The second domain is the ongoing management of programs and quality improvement activities where reports are developed to monitor key indicators such as program eligibility, enrollment, and engagement; patient lab and other clinical values; adverse events such as hospitalizations; quality measures; and costs (Bialek, Moran, & Duffy, 2009). In the third domain, outcomes evaluation, a comprehensive evaluation is performed to determine whether programs and interven-tions have met the original aims determined during the planning phase (Kleinpell, 2001; Nolan & Mock, 2000). Compared to the non-DNP research domain, the data
management processes under the DNP domains require more complexity with data cleaning and manipulation as well as different statistical techniques to adjust for the bias associated with the lack of control inherent in nonrando-mized control designs.
Because of the unique knowledge and skills necessary to manage data in these three domains, we created a two-pronged approach to provide each DNP student with individualized support. The first prong, consultation, was designed to develop the skills required to meet the goals and demands of individual DNP capstone projects and was provided 1:1 for each student. After a year of consulta-tions, the faculty recognized improvement in the quality of evaluations conducted as well as findings reported but identified the need to revise the curriculum. As a result, the second prong was developed. A CDM course focusing on strategies, procedures, and knowledge application to promote quality data management for evidence-based translation projects was added to the curriculum.
Frameworks for CDM
Data management processes are often covered in nursing texts that focus on research methods, outcomes measure-ment, evidence-based practice, and evaluation methods. Research texts typically focus on the design and methods of research studies and, as such, describe a process of data collection and analysis that acts upon data collected in a tightly controlled environment with minimal manage-ment needs (Polit & Hungler, 1999). Likewise, the outcomes measurement and evidence-based practice process touches on topics such as “analyzing the data” or“measuring the outcome,”but it does not go into the detail necessary to manage complex data sets that are often created for purposes other than research or outcomes measurement (Nolan & Mock, 2000).
Practicing DNPs need preparation to work within complex health care delivery systems to improve the quality, delivery, efficiency, and accessibility of health care services. A framework for data management within these complex organizations with disparate data collection systems requires a detailed process that includes designing feasible evaluative measures; acquiring data from multiple nonintegrated data collection systems; cleaning and ma-nipulating data; analyzing outcomes; and reporting results. Although the ideal framework for this process does not yet exist, there is increasing literature available that addresses data management for quality improvement purposes. For instance, a process of data quality control has been proposed to ensure the accuracy of data collection as well as analysis and reporting in which four phases are described: project design, data collection, data management, and data analysis (Needham et al., 2009). In addition, frameworks for CDM have been developed in the pharmaceutical industry, but these are more applicable to the rigor of randomized control clinical trials (Rondel et al., 2000). The CDM course content that we have developed incorporates concepts from the research, evaluation, outcomes measurement, and quality improvement methods literature.
Table 1. Competencies and Methodologies used in the Data Management Domains of the DNP Including a Comparison to a non-DNP Domain*
DNP domains Non-DNP domain
Competency/methodology category
1. Population identification/ stratification
2. Ongoing program
management 3. Outcomes evaluation
4. Knowledge- generating interventional research
Typical design/methods Descriptive Dashboards; business reports Quasi-experimental; observational Randomized control trial Purpose for using data Target high-risk groups for intervention Intervention development;
rapid cycle improvement
Process and outcomes measurement Test hypotheses Types of data Administrative; electronic medical record;
lab; radiology; survey; publicly available
Administrative; electronic medical record; workflow documents
Administrative; electronic medical record; lab; radiology; survey; publicly available
Data collected under rigorous protocol procedures Data collection/source Multiple existing databases; spreadsheets Multiple existing databases;
spreadsheets; paper
Multiple existing databases; spreadsheets Research database
Level of data cleaning Extensive Moderate Extensive Minimal
Level of data manipulation Extensive Moderate Extensive Minimal
Statistical techniques Weighting; risk adjustment; percentages; means
Trend/time series analysis; percentages; means
Parametric and nonparametric tests of means and proportions; adjustment for confounding/bias
Parametric tests of means and proportions
Consumers of results Executive leadership; administrators; program developers
Organizational stakeholders; administrators; program managers
Stakeholders internal and external to the organization including executive leadership; administrators; program developers; other similar organizations
DNPs; clinicians; executives; administration; other consumers of research
* References are noted in text.
SYLVIA
AND
Overview of CDM Content
Students in the DNP program are expected to carry out a scholarly project that addresses a complex practice, process, or systems problem. Our program focuses on translation of evidence into practice through the development and implementation of an evidence-based project using the Johns Hopkins Nursing Evidence-Based Practice Model (Newhouse, Dearholt, Poe, Pugh, & White, 2007). To determine if the implementation is effective, we require robust evaluation of outcomes. As a prerequisite for CDM, students must successfully com-plete course work in statistics; practice problem identi-fication and question generation; systematic literature review; critical appraisal and evaluation of the evidence; and translation of evidence into practice intervention. The CDM course currently focuses on evaluation of the DNP capstone project; however, it is important to note that the process of CDM described here can be applied to other processes requiring data management including but not limited to population health assessment, risk stratification, and management reporting.
Borrowing from processes in pharmaceutical clinical trials methods and nursing research methods, we have defined CDM as, “The process of planning, designing, collecting, cleansing, manipulating, analyzing, and reporting data generated in the assessment, development, delivery, and evaluation of health-related interventions, products, and services” (Polit & Hungler, 1999; Rondel et al., 2000).
The CDM process is laid out in seven phases: (a) planning, (b) data collection, (c) data cleansing, (d) data manipulation, (e) exploratory analysis, (f) outcomes
analysis, and (g) reporting and presentation. Some specific components of the process include identification of and linkages between project aims, outcomes, measures, variables, and data sources; measurement of statistical power; creation of data collection systems and processes; use of statistical software; management of sampling bias and confounding; identification and implementation of appropriate statistical testing; and meaningful presentation of results. The CDM course described here is offered completely on-line.
Planning Phase
Students begin the course by completing an evaluation plan for their scholarly project, which is framed within the project purpose and goals. The first component of the evaluation plan is a population or “denominator” des-cription(s), which will make up the rows of the eventual data set and is described in terms of inclusion and exclusion criteria, time frame for measurement, and comparison groups. The size of the denominator necessary for eventual statistically significant differences in outcome measures is determined using concepts of power analysis. Demograph-ic or descriptive variables are also defined for each denominator. Within this first evaluation plan component, the independent variable of interest is identified.
The second component of the evaluation plan breaks down each aim into outcomes measures, calculation of the measures, dependent variable(s) with descriptions, collection source, range of possible values, type of measurement, and statistical testing.Figure 1 shows an example of an evaluation plan. Evaluation planning is an iterative process that requires substantial commitment
and feedback from faculty. However, once the plan is complete, students use it as their compass throughout the remainder of the course, and the necessary diligence of faculty is relieved.
Data Management Phases
After completion of the evaluation plan, students progress through the phases of CDM. Table 2describes CDM module topics with related learning objectives.
CDM Support Tools
On-line Help Sessions
This course provides multiple support tools. One key to the success of the CDM course is on-line synchronous help sessions. Our students are located across the world, are often in different stages of project implementation and data management, and are working with their own data. Therefore, students need to be able to access help in a timely manner and on-demand as their project progression requires it.
The help sessions are offered weekly with an option of two different periods, led by the course instructors, and recorded for student reference. It is requested, but not required, that students submit questions via discussion board ahead of the help sessions. Students are encouraged to answer each other's questions whenever possible with interjection by the course instructors when needed. The synchronous sessions
also allow for demonstration of specific techniques in the statistical software, and students can submit their own data and request help with specific techniques with which they may be having trouble.
Although the modules provide overall content that can be applicable to any project, help sessions are invaluable in answering questions that may be unique to each student's project. In addition, since each student will not use all of the techniques presented throughout the course, help sessions allow them to view an application of a broader range of skills and knowledge.
Software and Learning Materials
IBM SPSS Statistics Base software is used as a technical tool for data management. Assignments that require evidence of success with technique (i.e., data structure, process documentation, data manipulation) are submit-ted in SPSS file format. SPSS was chosen because of its user-friendly, highly powered graphic user interface and its common use in nursing and the social sciences.
A textbook, on-line video tutorials, and a self-study guide are used to support SPSS coursework. Content for each phase of data management is provided through textbooks and relevant peer-reviewed literature. Where appropriate content does not exist, self-study workshops have been created. In addition, DNP graduates share their evaluation plans, presentations, papers, and publications with current students as exemplars.
Table 2. CDM Content Module Topics and Learning Objectives
Topic Learning objective(s)
Overview of data types and statistical tests •Define and describe CDM concepts and processes Overview of statistical software package •Create structure for data collection
•Analyze data
•Create meaningful display of results
Documentation of data management processes •Use syntax to document data management processes •Create reproducible process for data analysis
Design of data set and data dictionary •Create data set and dictionary through application of the evaluation plan Principles of data governance and fidelity to
data safety plan
•Assess organizational data governance structure •Assess fidelity to capstone data governance plan Maintenance of data integrity •Analyze evaluation data set for deviations
•Apply concepts of data cleansing to evaluation data set
Techniques for data manipulation •Recognize data manipulation needs to create final data set that will allow execution of evaluation plan
•Manipulate data using varied techniques
•Create and/or calculate variables defined in evaluation plan
Techniques for EDA •Analyze data using techniques of EDA, including descriptive statistics and denominator group comparisons
•Summarize and present results of EDA
Identification of confounding variables •Apply knowledge gained in EDA to recognize potential confounding factors on dependent variable
Creation of impactful graphs and tables •Translate results to graphic and table representation
Measurement of outcomes •Analyze effect of independent variable(s) on dependent variable(s) using comparative parametric and nonparametric statistics
Adjustment for confounding •Apply statistical techniques to adjust for confounding variables Techniques for quality written, oral, and visual
display of results
•Integrate results of analysis back into plan for evaluation •Summarize results
•Create impactful written, oral, and visual display
Proposed DNP Data Management
Objectives
Upon completion of the CDM coursework and gradua-tion from the DNP program, students are expected to be able to
• Create and execute a data management plan
• Create and maintain fidelity to a data governance plan
• Apply data manipulation techniques to address complex data collection and integration challenges
• Apply statistical techniques (i.e., power analysis, tests of means and proportions, adjusting for sampling bias) to increase strength and rigor of data analyses
• Recognize and convey the strengths and limitations of analyses
• Effectively present final results to multiple stake-holders and audiences.
CDM Successes and Challenges
CDM was offered as an elective in its first year, and more than 50% of eligible students registered. Our own observation of students of earlier cohorts that did not take CDM compared with those of later cohorts who did indicate that students who took the CDM course were more likely to create and implement a detailed evaluation plan, describe techniques for data cleansing and manipulation, understand and apply concepts of sample size determination using power analysis, use exploratory data analysis techniques to understand population attributes and sampling bias, appropriately apply techniques to adjust for bias, appropriately apply statistical significance testing, and present project results in a meaningful and impactful way. In addition, an in-house review of evaluation designs and statistical techniques revealed that a higher percentage of students in later cohorts moved beyond describing outcomes to statistically testing outcomes using comparison groups and adjusting for confounding/bias between intervention and com-parison groups. A formal, rigorous evaluation of the DNP program is planned for this year, which will help us to further understand the outcomes of CDM implementation.
On the basis of this informal evaluation, CDM evolved from an elective to a required course integrated in a thread of data management that incorporates relevant modules in Capstone courses as well as the Statistics course. In the final two semesters, CDM evolves into an independent course delivered on-line totaling four credits.
Although far outweighed by successes, we continue to address the challenges of integrating CDM into the DNP curriculum. We began the course as an elective, but soon realized the discrepancy in knowledge between those who did and did not complete the course. In the second year, the faculty made CDM a required part of the curriculum beginning with a thread within the statistics
and capstone courses in students' first year and two semesters of coursework in students' second and graduation year of the DNP program.
Perhaps one of the biggest challenges lies in providing the educational content necessary to meet the individual data management needs of each student's project while also providing that which is necessary for students to build a core set of competencies in data management. Providing a broad set of data management content that allows students to fill their CDM toolbox and creating forums for students to share their questions, challenges, and successes have been helpful strategies in addition to reducing the faculty-to-student ratio to 1:10 or less.
Conclusion
Advanced practice nurses of today and the future are challenged to lead a complex health care delivery system toward integration and coordination to improve the health of populations, communities, families, and in-dividuals. The ability to understand and apply data management skills and knowledge is a necessity for meeting these lofty aspirations. The addition of CDM to the DNP curriculum provides a basis for advanced practice nurses to lead in this arena.
References
Algase, D. L. (2010). Essentials of scholarship for the DNP: Are we clear yet?Research and Theory for Nursing Practice,24, 91–93.
American Association of Colleges of Nursing. (2006). The essentials of doctoral education for advanced nursing practice.
Austin, P. C. (2011). An introduction to propensity-score methods for reducing confounding in observational studies.
Multivariate Behavioral Research, 399.
Bialek, R., Moran, J. W., & Duffy, G. L. (2009).The public health quality improvement handbook. Milwaukee, WI: Quality Press.
Gordis, L. (2009).Epidemiology. (4th ed.). Philadelphia, PA: Saunders Elsevier.
Kleinpell, R. M. (2001). Outcome assessment in advanced practice nursing. (1st ed.). New York, NY: Springer Publishing. Mundinger, M. O., Starck, P., Hathaway, D., Shaver, J., & Woods, N. F. (2009). The ABCs of the doctor of nursing practice: Assessing resources, building a culture of clinical scholarship, curricular models.Journal of Professional Nursing: Official Journal of the American Association of Colleges of Nursing,
25, 69–74.
Nash, D. B., Reifsnyder, J., Fabius, R. J., & Pracilio, V. P. (2011). Population health: Creating a culture of wellness. (1st ed.). Sudbury, MA: Jones & Bartlett Learning.
Needham, D. M., Sinopoli, D. J., Dinglas, V. D., Berenholtz, S. M., Korupolu, R., Watson, S. R., et al. (2009). Improving data quality control in quality improve-ment projects. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care/ISQua, 21, 145–150.
Newhouse, R. P., Dearholt, S. L., Poe, S. S., Pugh, L. C., & White, K. M. (2007). Johns Hopkins nursing evidence-based practice model and guidelines. (1st ed.). Indianapolis: Sigma Theta Tau International.
Nolan, M., & Mock, V. (2000).Measuring patient outcomes. (1st ed.). Thousand Oaks, CA: Sage.
Polit, D. F., & Hungler, B. P. (Eds.). Nursing research principles and methods. (6th ed. p.–). Baltimore: Lippincott.
Rondel, R. K., Varley, S. A., & Webb, C. F. (2000).Clinical data management. (2nd ed.). New York, NY: Wiley & Sons.
Wall, B. M., Novak, J. C., & Wilkerson, S. A. (2005). Doctor of nursing practice program development: Reengineering health care.The Journal of Nursing Education,44, 396–403.
Webber, P. B. (2008). The doctor of nursing practice degree and research: Are we making an epistemological mistake?The Journal of Nursing Education,47, 466–472.
