Accreditation Movement
Informing the National Public Health Accreditation Movement: Lessons
From North Carolina’s Accredited Local Health Departments
Mary V. Davis, DrPH, MSPH, Margaret M. Cannon, MPH, David O. Stone, MS, Brittan W. Wood, MPH, Joy Reed, EdD, RN, and Edward L. Baker, MD, MPH, MSc
To advance understanding of public health accreditation, we analyzed data on the North Carolina Local Health Depart-ment Accreditation program. We surveyed accredited health departments on barriers to and supports of accreditation prep-aration, performance on accred-itation standards, and benefits and improvements after accred-itation.
All 48 accredited agencies responded to the survey. All agencies improved policies to prepare for accreditation and met most accreditation stan-dards. Forty-six percent received local funds for accreditation preparation. The most common barrier to accreditation prepara-tion was time and schedule lim-itations (79%). Fifty percent of agencies acted on suggestions for improvement, and 67% con-ducted quality improvement ac-tivities. Benefits of accreditation included improvements in local partnerships.
Agencies of all sizes con-ducted accreditation activities, were successfully accredited, and experienced benefits re-sulting from accreditation. (Am J Public Health.2011;101:
1543–1548. doi:10.2105/AJPH. 2011.300199)
A KEY STRATEGY FOR
IM-proving the functioning of local health departments (LHDs) is to institute a public health accredita-tion system that requires LHD capacity and performance to be measured against benchmarks or standards.1A voluntary national accreditation program for local public health agencies, the Public Health Accreditation Board (PHAB), was recently established.2
PHAB’s mission is to facilitate im-provement of state, local, and tribal public health departments while emphasizing that accreditation is not the ultimate goal but rather part of an overall strategy to improve the health of the nation.
Although the PHAB standards have been developed, the PHAB process beta test is complete, and state accreditation programs are under way, several public health accreditation issues are still being debated. These include the level of performance at which standards are written (i.e., capacity vs out-comes), accreditation’s costs rela-tive to its benefits, barriers to
preparing for accreditation, bene-fits of accreditation, and the ability of health departments of varying sizes and structures to meet ac-creditation standards.3---7Chief
among the debated issues is how to design an accreditation program that supports and fosters a quality improvement mindset in public health, rather than encouraging practitioners to see accreditation as an end in itself.2
Literature from the health care, education, social service, and public service fields suggests that public health accreditation can have pos-itive effects on service quality, op-erations, and service-related out-comes.8However, the literature on
public health accreditation’s impact on service capacity, delivery, and quality is in its nascent phase.9
The PHAB program is modeled on several state-based LHD ac-creditation systems, including the North Carolina Local Health De-partment Accreditation (NCLHDA) program. The NCLHDA history and program elements have been previously described.6,10Initiated
in 2004, with final state rules in place in 2006, this legislatively mandated program’s primary
objectives are to increase the ca-pacity, accountability, and consis-tency of the policies and practices of all North Carolina LHDs. Accredi-tation benchmarks are written pri-marily at the capacity-achievement level.10
As of July 2009, 48 of the 85 North Carolina LHDs had achieved accredited status under final state rules. (Two additional agencies received accreditation under pilot standards but had not achieved accreditation under the final rules.) Ten LHDs per state fiscal year will go through initial accreditation review, and all LHDs are required to have been re-viewed by 2014. Each LHD was allowed to choose the first fiscal year during which it would un-dergo initial accreditation. Ac-creditation status is valid for 4 years, after which LHDs must go through a reaccreditation process. Crosswalks between PHAB and North Carolina accreditation stan-dards yield 95% overlap on the content of the standards.
NCLHDA partners built evalu-ation into the program from its inception to ensure that the pro-gram performed as intended and
met the needs of stakeholders.11 Program evaluation reports reveal that 86% of participants (health directors, LHD staff, site visitors, and North Carolina Division of Public Health staff) are very satisfied with the accreditation administra-tor’s management of the program, and 92% of LHD directors report that they are satisfied with the accreditation program’s outputs relative to the time they and their staff spent to prepare for accredi-tation.12---14The NCLHDA pro-gram’s outputs are LHD achieve-ment of accreditation standards and LHD activities after accreditation to continue infrastructure and service improvements.
To date, no study has examined outputs of public health accredi-tation programs in light of LHD preparation activities, barriers to preparation, and benefits of par-ticipating in the program. To ad-vance understanding of public health accreditation, we examined the extent to which the 48 ac-credited North Carolina LHDs (1) conducted accreditation prepara-tion activities, experienced bar-riers to accreditation preparation, received support for accreditation preparation, and met the accredi-tation program standards; (2) conducted quality improvement activities after achievement of accredited status; and (3) experi-enced benefits as a result of being accredited.
METHODS
We reviewed data on LHD achievement of accreditation standards from program site-visit reports, and we collected data on accreditation preparation activities
from evaluation reports. Both types of reports covered the years 2006 through 2009. We also conducted an online survey of the 48 LHDs that achieved accredita-tion through the final rules pro-cess. We developed survey items on the basis of questions in the literature, review of evaluation reports, and consultation with program stakeholders. Survey items included 2 accreditation-preparation questions: we asked whether LHDs had received local funds to prepare for accreditation, and we asked whether LHDs had faced barriers to preparing for accreditation. The remaining sur-vey items assessed the extent to which LHDs had
1. updated policies and proce-dures after achieving accredita-tion;
2. acted upon site visitors’ sug-gestions for improvement; 3. conducted quality improvement
activities after the site visit; 4. experienced benefits after
being accredited; and 5. experienced barriers to
con-ducting improvements after achieving accreditation.
The survey included quantita-tive questions about specific ac-tivities and benefits, followed by
open-ended questions that asked respondents to explain how activ-ities were conducted or benefits were experienced.
We reviewed the survey items with the quality improvement and accreditation subcommittee of the North Carolina Association of Lo-cal Health Directors, and we soli-cited their feedback on the items. We e-mailed the survey invitation and link to the 48 local health directors, along with instructions for survey completion. Multiple e-mail and face-to-face reminders encouraged LHDs to respond. Three health directors from ac-credited agencies also contacted their colleagues to encourage them to respond.
Data analysis included summa-rizing program and evaluation data frequencies and percentages, compiling survey frequencies and percentages, and reviewing open-ended survey items to identify themes and explanations for closed-ended items.
RESULTS
The 48 accredited North Caro-lina LHDs served populations that ranged from 13 851 to 894 290. Most were single-county LHDs, but 2 were multicounty LHDs serving 3 and 7 counties each.
Table 1 shows the population sizes served by accredited and nonac-credited North Carolina LHDs. More than two thirds of the accredited LHDs served medium-sized popula-tions (50000---499999). The percentage of accredited LHDs that served medium-sized popula-tions was higher than was the percentage of nonaccredited LHDs serving those populations; conversely, the percentage of accredited LHDs serving small populations (< 50 000) was lower than was the percentage of non-accredited LHDs serving those populations. These differences were not statistically significant.
As a group, accredited LHDs met nearly all the accreditation standards: there were 60 total occurrences of unmet standards out of 7104 observations, for a rate of unmet standards of less than 1%. Fourteen agencies met all standards, 19 met all but 1 stan-dard, 8 missed 2 standards, 5 missed 3, and 2 missed 4. Of the 148 standards in the self-assess-ment, 25 standards were not met by at least 1 LHD, and 3 standards were not met by 5 or more LHDs (Table 2). As part of the program evaluation, LHDs reported their activities to prepare for accredita-tion. All 48 LHDs updated or created new policies or
TABLE 1—Local Health Departments, by State Accreditation Status and Size of Population Served: North Carolina, 2006–2009
Population Size Accredited LHDs (n = 48), No. (%) Not Accredited LHDs (n = 37), No. (%) All LHDs (n = 85), No. (%)
< 50 000 14 (29.2) 16 (43.2) 30 (35.3)
50 000–499 999 33 (68.7) 20 (54.1) 53 (62.3)
‡500 000 1 (2.1) 1 (2.7) 2 (2.4)
procedures during the preparation process, and most LHDs reported updating or creating multiple pol-icies or procedures to prepare for accreditation.
Accreditation and Funding
All 48 accredited LHDs com-pleted the accreditation follow-up survey. In response to a survey item asking whether agencies had received funding to prepare for accreditation, 22 LHDs (46%) reported receiving local funds to prepare for accreditation. These funds were used to cover salary costs for accreditation preparation, facilities improvements (e.g., sign-age, medical records storage and security), and office supplies. One LHD received extensive funding ($1 million) from its county com-missioners to cover the costs of filling new positions and renovat-ing the health department buildrenovat-ing to comply with standards and im-prove workspace organization.
One survey item asked whether LHDs had, as a result of being accredited, experienced increase or maintenance of funding levels from the county government or receipt of new funding. Agencies that said any of these things had happened were asked by an open-ended item to explain the funding benefit. Two agencies reported that funding from county commissioners had
increased as a result of accredita-tion, 4 agencies had maintained county funding, and 2 agencies had received new grant funding as a re-sult of being accredited. Increased funding went to support staff sal-aries and building improvements. The LHDs that had maintained funding used these resources to support administrative staffing needs and general services. The LHD that provided information on new grant funding indicated that it had received several small grants (sources not identified) to support a variety of activities, in-cluding online registration of appli-cations for septic systems, foreign-language interpreters, and HIV services.
Improvements and Benefits After Accreditation
Table 3 provides data on LHDs’ improvement activities after ac-creditation and relationship bene-fits they experienced after achiev-ing accreditation. Twenty-four LHDs (50%) reported acting on suggestions for improvement iden-tified by site visitors, including updating position descriptions, or-ganization charts, and strategic plans; improving client privacy in waiting and clinic areas; and in-creasing board of health diversity.
Thirty-two agencies (67%) reported conducting quality im-provement activities after achiev-ing accreditation. The majority of these agencies (56%) conducted 1
or 2 quality improvement projects, 25% conducted 3 or more pro-jects, and 19% conducted 5 or more projects. Processes used in-cluded the Institute for Healthcare Improvement’s model for im-provement, Lean, Six Sigma, and quality improvement tools such as Pareto charts. LHDs explained their projects as follows:
Used team improvement pro-gram to conduct H1N1 response, and it was extremely successful and flexible enough to meet the changing circumstances of this major outbreak.
Improved customer service by reducing wait time and total pa-tient visit time by evaluating clinic patient flow and identifying areas for improvement.
TABLE 2—State Accreditation Standards Not Met by 5 or More Accredited Local Health Departments: North Carolina, 2006–2009
Accreditation Standard Language
No. of Agencies Not Meeting Standard Activity 7.3: The local health department shall investigate and respond to environmental health complaints or referrals. 9 Activity 30.10: The local health department shall make efforts to prohibit the use of tobacco in all areas and grounds within fifty
(50) feet of the health department facility.
5
Activity 31.4: The local health department shall have current written position descriptions and qualifications for each staff position. 6
TABLE 3—Improvement Activities or Relationships That Benefited After State Accreditation for Local Health Departments: North Carolina, 2010
Improvement or Benefit, No. (%) No Improvement or Benefit, No. (%) Improvement activities
Updated polices 45 (94) 3 (6)
Acted on suggestions for QI 24 (50) 24 (50)
Conducted a QI project 32 (67) 16 (33)
Relationships that benefited
County commissioners 11 (23) 37 (77)
Community partners 15 (32) 33 (68)
Local hospital 11 (23) 37 (77)
Board of health 27 (56) 21 (44)
We created a ‘‘short service clinic’’ to streamline services such as TB skin testing. . .. This resulted in a significantly short-ened waiting period and im-proved customer satisfaction.
Accredited LHDs reported im-proved relationships with county commissioners (23%), community partners (32%), and local hospitals (23%) after achieving accredita-tion. LHDs provided the following accounts of these relationship improvements:
Accreditation brought forth good press, thus the commissioners recognized the value of the health department.
It has improved the general image of the department by substantiat-ing the level of quality and pro-fessionalism of the organization and its people. The accreditation process and our achievement are viewed by our partners as being grounded in best practice princi-ples and therefore validate the organization’s commitment to its vision of being ‘‘a model of best practice in public health.’’
Twenty-seven (56%) agencies reported improved relationships with their local board of health, which included general improve-ments in relationships, increased attendance at board of health meetings, and improved board member understanding of their role as it relates to the LHD’s function.
On an open-ended item, 22 agencies identified additional benefits of accreditation, which we coded into 13 themes. The most commonly reported benefits were pride as a result of achieving ac-creditation (7 LHDs), improved policies and processes (5 LHDs), team building and teamwork (4 LHDs), and improved staff appre-ciation of public health services
and functions (4 LHDs). One re-spondent commented that being accredited had helped agency staff feel that, when it came to LHD capacity, LHD size did not matter:
In essence, we realized that whether large or small, things equate out when it comes to benchmarks, organization and structure, and staffing (even if some of our staff do several jobs). Barriers to Accreditation and Implementing Improvements
The survey provided a list of barriers to accreditation and implementing improvements after accreditation, and LHDs were asked to check those that applied to them. The most common bar-riers to preparing for accreditation were time and schedule limita-tions (79%), resource limitalimita-tions (50%), and lack of perceived value or benefit of accreditation (42%; Table 4). The most common bar-riers to implementing improve-ments were the same as preparing for accreditation, but with fewer agencies identifying each of these as barriers––52%, 46%, and 15%, respectively. These survey items specifically assessed whether lack of county support or board of health support were barriers to preparation or conducting
improvements. Only 1 LHD reported that lack of support from the county was a barrier to ac-creditation preparation.
DISCUSSION
This review of NCLHDA gram data and survey results pro-vides new insights regarding what can be expected from public health accreditation programs. The agencies accredited through the NCLHDA program met nearly all accreditation standards, even though agencies only needed to meet 80% of standards to be accredited. This level of achieve-ment, however, resulted from considerable accreditation prepa-ration, a fact that may validate the original intent of the program: to improve the capacity, account-ability, and consistency of North Carolina LHDs’ policies and prac-tices by setting the accreditation bar at capacity level. As North Carolina LHDs go through reac-creditation, they will demonstrate quality improvement to meet ac-creditation standards, and LHD activities will be monitored ac-cordingly.
It is possible that the North Carolina LHDs that chose to
undergo accreditation in the early years of the program were already high performers, a pattern that has been observed in the public service industry accreditation lit-erature.8An examination of the
first 37 North Carolina agencies accredited found that these agencies tended to serve larger populations, have higher public health expenditures, and have more full-time---equivalent employees.15 These are factors identified in the public health systems and services research as being associated with higher performance.16---18Although the North Carolina---accredited agencies may be more likely to have factors present indicating the potential for high performance, all North Carolina-accredited agencies needed to conduct considerable work to prepare for accreditation. Thus, factors associated with high performance may not indicate readiness to apply for accreditation.
A critical reported barrier to preparing for accreditation is the perceived cost of employee time and other resources needed to be successful, which is exacerbated by the concern that local funding will not be available to assist agencies with the process.4Nearly half of the
North Carolina---accredited LHDs
TABLE 4—Barriers to Preparing for and Implementing Improvements After State Accreditation: Local Health Departments, North Carolina, 2010
Barrier Experienced During Preparations, No. (%) Experienced While Implementing Improvements, No. (%)
Resource limitations 24 (50) 22 (46)
Time and schedule limitations 38 (79) 25 (52)
Lack of county support 1 (2) 3 (6)
Lack of staff support 6 (13) 3 (6)
Lack of perceived value or benefit 20 (42) 7 (15)
reported receiving local funds to prepare for accreditation. Local governments may be more willing to support accreditation prepara-tion activities than was previously thought, especially when an LHD needs funds to make improvements to meet standards, such as facilities improvements. Nevertheless, half of the accredited agencies reported that resource limitations were a barrier to preparing for accredi-tation, but this item did not specify whether these were financial limi-tations. Moreover, 79% of LHDs reported time and schedule limita-tions as a barrier. Additional re-search may be needed to further explore the relative importance of these barriers.
Among North Carolina---accredited LHDs, 50% reported acting on suggestions for quality improvement identified by site visitors, and two thirds reported conducting quality improvement projects. Previous surveys of North Carolina agencies and of health departments nationwide found that agencies were con-ducting quality assurance activi-ties rather than quality improve-ment activities (North Carolina Institute of Public Health, unpub-lished data, 2010).19Quality
im-provement activities reported in the current study included the use of more ‘‘true’’ quality improvement techniques, including aim state-ments and use of methodologies specific to quality improvement. It is difficult to discern whether this shift to quality improvement work is attributable to participation in the accreditation program or to the plethora of public health efforts to advance quality improvement in LHDs.20Nevertheless, in NCLHDA
program evaluations and research projects, LHD directors and staff indicated that accreditation served as a platform for performance im-provement. Thus, there may be at least some relationship between participating in accreditation and conducting quality improvement activities.
Previous reviews of NCLHDA evaluation data revealed that agencies experienced considerable benefits as a result of preparing for accreditation, such as increased teamwork and improvements in partnerships with hospitals and community groups.4Our results show improvements in relation-ships with these partners as well as with county commissioners and boards of health. These benefits, which are related to improved community perception of LHDs, may help balance the perceived cost/benefit ratio of public health accreditation.4,8
In contrast to previously reported perceptions of the ability of small LHDs to meet accredita-tion standards,3---5North Carolina---accredited agencies of all sizes, in-cluding 14 that served populations of less than 50 000, were able to meet accreditation standards. According to 2008 profile data from the National Association of County and City Health Officials, among health departments that served populations of less than 50000 only 12% of respondents strongly agreed that their LHD would seek accreditation, and 35% agreed.21This is likely because of previous reports on the perceived costs and benefits of accreditation and the perceived ability of small agencies to meet accreditation benchmarks.3---5Results from the
review of NCLHDA program data and this survey may dispel the perception that LHDs that serve small populations will experience undue difficulties in achieving ac-creditation standards.
Because of the cross-sectional nature of this study, we cannot assert that participation in the accreditation program caused any of the specific outputs studied. Limited resources for this work and a small sample size precluded us from conducting more sophis-ticated analyses on these data. Future studies could examine dif-ferences in outcomes (particularly among LHDs that serve different sizes of populations), performance on accreditation activities, benefits experienced as a result of accred-itation, and factors associated with performance in the public health systems and services literature.
Although the NCLHDA pro-gram activities are written pri-marily at a capacity level, the North Carolina---accredited agencies profiled in this study conducted considerable im-provement activities before and after accreditation. These agencies reported experiencing a variety of benefits from being accredited, including improve-ments in partnerships with their county governments and boards of health. Our findings suggest that agencies of all sizes are able to meet accreditation standards and experience the benefits of accreditation.j
About the Authors
Mary V. Davis, David O. Stone, Brittan W. Wood, and Edward L. Baker are with the North Carolina Institute of Public Health, Gillings School of Global Public Health,
University of North Carolina at Chapel Hill. At the time of the study, Margaret M. Cannon was with the North Carolina In-stitute of Public Health, Gillings School of Global Public Health, University of North Carolina at Chapel Hill. Joy Reed is with the Division of Public Health, North Carolina Department of Health and Human Services, Raleigh.
Correspondence should be sent to Mary V. Davis, North Carolina Institute of Public Health, CB 8165, Chapel Hill, NC 27599 (e-mail: Mary_davis@unc.edu). Reprints can be ordered at http://www.ajph.org by clicking the ‘‘Reprints/Eprints’’ link.
This article was accepted February 14, 2011.
Contributors
M. V. Davis, M. M. Cannon, D. O. Stone, B. W. Wood, and E. L. Baker conceptual-ized the study and created data-collection instruments and procedures. M. V. Davis and M. M. Cannon implemented the survey. M. V. Davis conducted data anal-yses and prepared article drafts. M. M. Cannon, D. O. Stone, B. W. Wood, J. Reed, and E. L. Baker reviewed versions of the article and provided substantive comments on them.
Acknowledgments
This work was supported by funds from the North Carolina General Assembly.
The authors would like to thank Leah Devlin, DDS, for her contributions to the development of this work.
Human Participant Protection
The public health---nursing institutional review board of the University of North Carolina at Chapel Hill determined that this study protocol did not require in-stitutional review board approval.References
1. Institute of Medicine.The Future of the Public’s Health in the 21st Century. Wash-ington, DC: National Academies Press; 2003.
2. Bender K, Halverson PK. Quality improvement and accreditation: what might it look like?J Public Health Manag Pract. 2010;16(1):79---82.
3. Bender K, Benjamin G, Carden J, et al. Final recommendations for a voluntary national accreditation program for state and local health departments: steering committee report.J Public Health Manag Pract. 2007;13(4):342---348.
4. Davis MV, Cannon MM, Corso L, Lenaway D, Baker EL. Incentives to en-courage participation in the national public health accreditation model: a sys-tematic investigation.Am J Public Health. 2009;99(9):1705---1711.
5. Meit M, Harris K, Bushar J, Piya B, Molfino M.Rural Public Health Agency Accreditation: Final Report. Bethesda, MD: Walsh Center for Rural Health Analysis, NORC at the University of Chicago; 2008. 6. Davis MV, Reed J, Devlin LM, Michalak CL, Stevens R, Baker E. The NC accreditation learning collaborative: part-ners enhancing local health department accreditation.J Public Health Manag Pract. 2007;13(4):422---426.
7. Nolan P, Bialek R, Kushion ML, Lenaway D, Hamm MS. Financing and creating incentives for a voluntary na-tional accreditation system for public health.J Public Health Manag Pract. 2007; 13(4):378---382.
8. Mays G.Can Accreditation Work in Public Health? Lessons Learned From Other Industries. Princeton, NJ: Robert Wood Johnson Foundation; 2004.
9. Joly BM, Polyak G, Davis MV, et al. Linking accreditation and public health outcomes: a logic model approach.J Public Health Manag Pract. 2007;13(4): 349---356.
10. Beitsch LM, Mays G, Corso L, Chang C, Brewer R. States gathering momentum: promising strategies for accreditation and assessment activities in multistate learning collaborative applicant states. J Public Health Manag Pract. 2007;13(4): 364---373.
11. Tremain B, Davis M, Joly B, Edgar M, Kusion ML, Schmidt R. Evaluation as a critical factor of success in local public health accreditation programs.J Public Health Manag Pract. 2007;13(4): 404---409.
12. North Carolina Local Health De-partment Accreditation Program.July 2006---June 2007 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2007. 13. North Carolina Local Health De-partment Accreditation Program.July 2007---June 2008 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2008. 14. North Carolina Local Health De-partment Accreditation Program.July 2008---June 2009 Stakeholder Evaluation Report. Chapel Hill, NC: North Carolina Institute for Public Health; 2009.
15. Cilenti D.North Carolina Public Health Agency Accreditation and Per-formance: The Climb From Good to Ex-traordinary[dissertation]. Chapel Hill: Uni-versity of North Carolina; 2009. 16. Beitsch LM, Grigg M, Menachemi N, Brooks R. Roles of local public health agencies within the state public health system.J Public Health Manag Pract. 2006;12(3):232---241.
17. Erwin PC. The performance of local health departments: a review of the lit-erature.J Public Health Manag Pract. 2008;14(2):E9---E18.
18. Mays GP, McHugh MC, Shim K, et al. Institutional and economic determinants of public health system performance.Am J Public Health. 2006;96(3):523---531. 19. National Association of County and City Health Officials.2005 National Pro-file of Local Health Departments. Wash-ington, DC: National Association of County and City Health Officials; 2006. 20. Davis MV. Opportunities to advance quality improvement in public health. J Public Health Manag Pract. 2010;16(1): 8---10.
21. National Association of County and City Health Officials.2008 National Pro-file of Local Health Departments. Wash-ington, DC: National Association of County and City Health Officials; 2009.
