• No results found

10-1108_IJHG-05-2017-0020.pdf

N/A
N/A
Protected

Academic year: 2020

Share "10-1108_IJHG-05-2017-0020.pdf"

Copied!
16
0
0

Loading.... (view fulltext now)

Full text

(1)

Community pharmacists as

vaccine providers

Laura A. Rhodes

Division of Practice Advancement and Clinical Education, University of North Carolina Eshelman School of Pharmacy, Chapel Hill,

North Carolina, USA

Dennis M. Williams

Division of Pharmacotherapy and Experimental Therapeutics, University of North Carolina Eshelman School of Pharmacy, Chapel Hill,

North Carolina, USA

Macary W. Marciniak

Division of Practice Advancement and Clinical Education, University of North Carolina Eshelman School of Pharmacy, Chapel Hill,

North Carolina, USA, and

David Jay Weber

Division of Infectious Diseases, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA

Abstract

Purpose–The purpose of this paper is to describe the history of pharmacist involvement as vaccine providers in the USA and discuss examples of growing interests in other parts of the world.

Design/methodology/approach Literature searches were performed in PubMed as well as pharmacy-related journals.

Findings–Pharmacists have been involved with the storage and management of vaccines for more than a century. Based on the unmet needs in meeting national goals for vaccination rates among adults in the USA, efforts led to training and recognizing pharmacists as vaccine providers which is now within the scope of practice for a pharmacist in all US states and territories. Pharmacists complete a comprehensive training program in vaccine sciences, regulatory considerations, as well as demonstration of skills in administering vaccines. Over 300,000 pharmacists have been trained in vaccine delivery and this represents the majority of the pharmacist workforce in the USA. There are examples of the beneficial impact of pharmacist involvement as vaccine providers in community pharmacy settings.

Research limitations/implicationsThis review is based on a thorough review of the literature but was not conducted in a systematic fashion.

Originality/value–This review provides a historical perspective and evidence of the benefit of pharmacists as vaccine providers.

KeywordsHealth policy, Effectiveness, General practice, Health law or regulation, Infectious diseases, Vaccines or vaccination

Paper typeGeneral review

Background

Vaccines represent one of the most significant advances in healthcare during the twentieth century with progress continuing into the twenty-first century (Centers for Disease Control and Prevention (CDC), 1999, 2011). The discovery and development of vaccines began in the nineteenth century with smallpox vaccine, but did not gain momentum until the middle of the twentieth century with the introduction of several important vaccines, including the polio vaccine.

(2)

Prevention (CDC) and proof of current vaccination status is often a requirement for attendance in public schools (Centers for Disease Control and Prevention, 2017a; SchoolVaxView, 2017). Vaccine schedules are also available for adult patients that detail recommendations based upon age, medical conditions and risks (Centers for Disease Control and Prevention, 2017b). Adherence to adolescent and adult immunization schedules is less common, partly because there are fewer consequences related to access to community resources and programs. An exception is that attendance at colleges and universities often require proof of immunity against selected vaccine-preventable diseases. Another exception is the requirement for multiple immunizations by members of the US military or the requirement for those who work in healthcare settings to receive seasonal influenza vaccine. Therefore, vaccination status for adults lags behind the Healthy People 2020 goals. Annually, nearly 50,000 deaths in the USA are attributed to vaccine-preventable diseases (US Department of Health and Human Services, 2017).

There are numerous reasons for failure to meet national standards for adult vaccination coverage rates in the USA, including lack of time, access, awareness and resources. One solution to address this problem gained momentum nearly 20 years ago when the profession of pharmacy sought to include provision of vaccines in the scope of their practice. As this unmet need for US adults was identified, public health officials also recognized the potential of engaging pharmacists as an effective and accessible source for vaccines (Rimeret al., 2014; Schuchat, 2015; National Vaccine Advisory Committee, 2016). The administration of vaccines was also an opportunity for pharmacists to expand their scope of practice and develop a financial model that was sustainable.

This review summarizes the history and successes of pharmacists as vaccine providers, describes the training that pharmacists receive and the procedures that they practice under to administer vaccines, details the impact of engaging pharmacists as vaccine providers in the USA. Global involvement of pharmacists in vaccine delivery is also briefly summarized.

History

Pharmacists have played an indirect role with vaccines for nearly 125 years, with the first documented encounter of drugstores storing vaccines and preparing antitoxins for administration in 1,895 (Grabenstein, 1999). During the early twentieth century, pharmacies were a source of vaccine storage, preparation, and distribution. Later in the twentieth century, pharmacists became more proactive as educators and advocates for vaccines. In the 1980s, pharmacy businesses began contracting with traveling nurse organizations to conduct onsite influenza vaccination clinics (Hogueet al., 2006).

Activities began to advance quickly in the mid-1990s when the US Department of Health and Human Services (DHHS) sought to enlist pharmacists as a resource to improve the nation’s vaccination rates. The DHHS recognized the potential role of pharmacists in several areas including vaccine education and awareness, distribution, access, administration, and contributing to vaccine registries and systems.

Pharmacists first began administering immunizations in Washington State in response to a change in the state pharmacy practice act. As a result, 50 pharmacists received training in 1994, and began administering vaccines to adults (Hogueet al., 2006). In 1996, the American Pharmacists Association (APhA) responded to a request from the Mississippi Pharmacists Association and the Mississippi Department of Public Health to engage pharmacists in addressing disparities in vaccination rates for pneumococcal and influenza vaccine among adults. This need resulted in the development of the national training program for pharmacists in conjunction with the CDC (Hogueet al., 2006; Bach and Goad, 2015).

(3)

pharmacists in 52 states and territories in the USA are authorized to administer vaccines (American Pharmacist Association, 2016). However, the privileges for pharmacists varies among states based on the scope of practice defined in the state’s Pharmacy Practice Acts. An impetus for promoting pharmacists as immunization providers was the historically low rates of vaccinations among adults and disparities in selected populations, as well as the access to pharmacist services based on extended hours of operation. Initial efforts of pharmacists focused on adult vaccines, although a broader scope existed in some states. Pharmacists are trusted professionals with the ability to promote awareness and importance of vaccines. Factors supporting pharmacists as immunizers include access, convenience, and consumer confidence and trust. It is estimated that over 90 percent of Americans live within five miles of a community pharmacy (NACDS, 2010-2011). Approximately 275 million patient visits are made to a pharmacy each week (Centers for Disease Control and Prevention, 2014). In March 2014, the National Vaccine Advisory Committee (NVAC), which publishes standards for immunization practice for all providers, included pharmacists in their recommendations. The pharmacist’s role as vaccine providers has become a societal norm, as pharmacies are now recognized as stakeholders in the initiative to increase vaccination rates. However, there are still possible advantages in further expanding the scope of privileges of pharmacists to cover additional vaccines and eligible patient populations.

Training

Other clinicians report safety and training as concerns for patients receiving vaccines in a community pharmacy. The formal training and assessment that pharmacists are required to complete are often more stringent than training of other practitioners. State Pharmacy Practice Acts detail the required training for becoming an immunizing pharmacist, and the education and training for continuing maintenance and competence. The majority of pharmacists trained as immunizers have completed a nationally developed certificate training program developed by the APhA which is recognized by the CDC (Hogueet al., 2006). The program consists of a 12-hour home study, a live 8-hour session with practice applications, and a live skills assessment in which the participant demonstrates competence in vaccine administration technique. Other options are available for training but are similar in duration and scope to the APhA program. As of 2017, over 300,000 US pharmacists have completed this program, which represents the majority of the workforce (American Pharmacists Association, 2016).

Although not required by the Accreditation Council for Pharmacy Education, most student pharmacists across the USA have completed the certificate training program in immunization delivery as part of their required Doctor of Pharmacy curriculum. Qualified student pharmacists can often provide vaccine services under the supervision of a pharmacist with the credential, although this authority depends on the specific state of practice. Upon graduation, newly licensed pharmacists are eligible to provide vaccine services based on their credentialed training as a student and compliance with requirements set by State Pharmacy Practice Acts.

(4)

Another aspect of vaccine safety is related to errors made in selecting and administering vaccine products to patients. With the growth in vaccines available and increasing complexity of recommendations, this is an important issue. The Institute for Safe Medication Practices established the Vaccine Error Reporting Program (VERP) in 2012 with a goal to reduce errors with ongoing learning and experiences. From September 2012 through October 2013, a total of 433 errors were submitted to VERP and 90 percent resulted in a patient receiving a vaccine that they should not have. The most common setting for errors was outpatient medical clinics (35 percent of reports). Physician offices accounted for 29 percent of reports, and community pharmacies 2 percent (ISMP, 2013).

Similar to medical membership associations, leadership in pharmacy and the national pharmacy organizations support and promote up to date vaccinations for pharmacists and employees. Both the APhA and the American Society of Health System Pharmacists maintain policies advocating for annual influenza vaccine among pharmacists. As it relates to influenza vaccine, the goal for healthcare personnel according to the Healthy People 2020 initiative is 90 percent (US Department of Health and Human Services, 2017).

During the 2012-2013 season, the CDC began documenting influenza immunization rates for pharmacist-administered vaccine. The reported rate has been 85.7 and 95.3 percent, for 2013-2014 and 2014-2015, respectively. This compares favorably with the overall rates for healthcare personnel of 75.2 and 77.3 percent for the same time period (Blacket al., 2015).

Scope of practice

Currently, 52 states and territories in the USA recognize pharmacists as a resource for vaccinations. The pharmacist’s authority is governed by state pharmacy practice acts in a manner similar to medical practice acts. Therefore, the scope of immunization services varies among the states. This includes which vaccines can be administered, the age of eligible patients, and the strategy for vaccine provision (standing orders, protocols or individual prescription). An optimal approach involves pharmacists working collaboratively with other healthcare professionals to improve outcomes for the patient. This includes assessing and recommending appropriate vaccines, administering the vaccine, and reporting administered vaccines to the primary care provider and/or the immunization information system (IIS) (registry). In fact, the literature reports that approximately 75 percent of general internists, family medicine physicians, and pediatricians are willing to collaborate with community-based pharmacies as delivery sites for adolescent and adult immunizations (Kempeet al., 2012; Hurleyet al., 2011).

The initial engagement of pharmacists as vaccine providers was often restricted to individuals⩾18 years, and focused on influenza and pneumococcal vaccines. As of 2016, 28 states permit pharmacists to vaccinate patients of any age, 17 states permit vaccination of patients between 3 and 17 years, and seven states permit vaccines for individuals 18 years of age or greater. For example, a recent change in California law allows pharmacists to administer vaccines to individuals who are ⩾3 years of age; Texas law allows for⩾5 years of age; and in North Carolina, influenza vaccine can be administered to patients⩾14 years of age. Finally, the ability of student pharmacists to administer vaccines under the supervision of a pharmacist credentialed as an immunizing pharmacist varies depending on the state of practice (American Pharmacists Association, 2016).

(5)

(Immunization Action Coalition, 2017; New York State Department of Health, 2012). The components of a standing order or protocol are summarized in the list“Components of a protocol for pharmacist vaccination services”. The final strategy for pharmacist authority is through receipt of a prescription for the patient. This approach is the most restrictive because it does not address improved access to vaccines as it still requires a prescriber intervention.

Components of a protocol for pharmacist vaccination services are as follows:

(1) states pharmacist(s) covered by the protocol;

(2) states sponsoring physician;

(3) indicates the vaccines that can be administered;

(4) describes processes and procedures, including decision-making process by pharmacist to administer vaccine or refer patient;

(5) authorizes a plan for emergency management; and

(6) defines documentation and record-keeping procedures.

While barriers to pharmacist participation in vaccine services have lessened over time, challenges still exist to developing and improving these activities. Practice acts and the scope of services from a pharmacist are constantly changing based on unmet needs and collaborations between pharmacists and other medical providers. Additionally, coverage provided for pharmacist provision of vaccines by insurance companies remains a barrier. One factor promoting pharmacists as vaccine providers in the USA is that many adult vaccines are covered by Medicare Part D. Medicare Part D plans cover all commercially available vaccines as long as the vaccine is reasonable and necessary to prevent illness (e.g. herpes zoster or shingles vaccine) (Department of Health and Human Services, 2016). Under this program, the coverage for the vaccine is part of the prescription drug benefit and physician offices are often not able to bill for vaccine products under Medicare Part D. While this facilitates the delivery of pharmacy-administered vaccine services, full coverage of vaccine services by pharmacists is not provided consistently by insurance companies. Pharmacists are required to adjudicate the claim in order to determine if the service is covered or if the patient must pay out of pocket and file with the insurer directly. This may hinder the patient’s ability to receive select vaccines, depending upon the patient’s financial status and willingness to pay out of pocket at the time of vaccine administration.

Practice models and challenges

The effectiveness of pharmacists as immunizers is an important consideration in improving vaccination rates. Pharmacists are viewed as knowledgeable health professionals and trusted by the public. A randomized controlled trial in 1993 examined the effectiveness of pharmacists in encouraging patients to receive vaccine from their medical provider (Grabenstein et al., 1993). Participants who received advice from the pharmacist were 74 percent more likely to be vaccinated. A cost-effectiveness study suggested a savings to Medicare of $280,000 based on reduced hospitalization (Grabensteinet al., 1992). Subsequent studies over the next decade provided further evidence of the effectiveness of pharmacists in influencing patients to receive vaccines (Hogueet al., 2006).

(6)

is to incorporate the administration of vaccines into the normal dispensing activity of practice, although this is often combined with targeted vaccine clinic days that can be advertised and promoted. There are staffing considerations and workflow issues with both approaches.

There has been a steady increase in the availability of immunization services during the past two decades. In addition, the number of vaccine doses administered has increased with the most consistent data available for influenza vaccines. Between 2007 and 2013, the number of influenza vaccines administered in community pharmacy settings increased from 3.2 million to 20.9 million (McConeghy and Wing, 2016). In 2016, 25 percent of all influenza vaccine doses were administered by pharmacists (American Pharmacists Association, 2016).

Inserting vaccine services into the normal workflow in a community pharmacy setting requires planning and preparation, including staff education. Pharmacy personnel, including technicians and sales clerks, require training as to how to handle an immunization request and what information to provide and obtain from the patient. The patient should be informed about the expected wait time for a vaccine. In this scenario, the patient awaiting a vaccine is normally placed in the cue along with patients awaiting their prescriptions. When vaccine services are offered on an appointment basis, or during designated vaccine clinic days, the pharmacy must plan for additional staffing in order to provide these services (Watson and Bluml, 2013).

However, many community pharmacies are already busy with the traditional process of medication dispensing, and implementing any new service can be challenging. In a 2009 report, factors influencing the sustainability of immunization services in community pharmacists were evaluated (Westrick and Breland, 2009). Important factors that should be considered during program development and implementation are given in the list“Factors influencing sustainability of immunization service programs”. Additionally, the CDC provides practice standards for all healthcare professionals to help ensure adult patients are fully immunized. Steps include a vaccine status assessment, strongly recommending needed vaccines, administration of (or referral for) vaccines, and documentation of vaccine receipt (Centers for Disease Control and Prevention 2016a).

Factors influencing sustainability of immunization service programs are as follows:

(1) influential person in the organization serving as program champion;

(2) formal program evaluation for continuous improvement;

(3) willingness to modify program and services based on results;

(4) compatibility of service with organizational mission, resources, workflow, and pharmacist’s skills; and

(5) comprehensive integration into the organizational structure.

A recent study evaluated the utility of a community pharmacy-based vaccine screening program in determining patient eligibility for adult vaccines (Rhodes et al., 2017). The program was conducted in a community pharmacy operating in five separate locations. A vaccine screening tool and documentation form was utilized to collect information. If appropriate, a pharmacist recommended vaccines for the patient who either accepted the recommendation or provided a reason for declining. During this 30-day study, over 630 screenings were conducted primarily by technicians (95 percent) at the time of initial presentation for data entry at the pharmacy (82 percent). These results demonstrate how effective screening for vaccine needs can be incorporated into the workflow in a community pharmacy setting.

(7)

be efficient in that tens of thousands of vaccines have been administered at mobile clinics and health fairs.

In a report to the US DHHS in 2013, the APhA reported on survey results from nearly 7,000 pharmacists conducted earlier in the year (American Pharmacists Association, 2016). Over 75 percent of respondents were practicing in community pharmacy settings. Approximately 86 percent of the practice sites offered vaccine services, and the majority of practice sites (77 percent) offered immunizations to patients on a walk-in basis (no appointment required). Appointment-based services had declined but were still available in 36 percent of settings. The survey indicated an increase in referrals from physicians at 88 percent, while pharmacists referred an average of four patients per week back to the primary care provider to address an immunization issue, and another 14 for other medication-related issues. Over 90 percent of vaccines were administered to adult patients. This report supported a steady trend toward increasing collaborations with other vaccine providers to improve overall vaccination rates toward national goals.

Finally, a specialty area for vaccine services that some pharmacists have identified as an unmet need is related to travel vaccines. Individuals traveling abroad on business, pleasure or mission trips often require vaccinations in addition to other healthcare advice related to their travel. These services are often not available in their local communities. Participation in travel medicine, including vaccines, is not a standard part of State Pharmacy Practice Acts but can be accommodated through expanded authority or collaborative practice agreements in some states. Pharmacists can complete certification training in travel medicine provided by the CDC, and APhA offers a training program for interested individuals (American Pharmacists Association, 2016). The International Society of Travel Medicine (2017) serves as additional resource that may be utilized by pharmacists who wish to earn a certificate of knowledge in the practice of travel medicine.

In order to be effective in improving vaccination rates, pharmacists should develop and maintain skills in counseling patients, including the use of motivational interviewing (MI) strategies. The concept of MI was introduced approximately 30 years ago and involves patient-centered communication and information exchange to facilitate constructive thought among patients about their health (Emmons and Rollnick, 2001). MI is intended to result in behavioral change and requires the clinician to express empathy, develop discrepancy in thought, manage resistance, and support patient autonomy. Skill training in MI techniques is recommended and evidence supports it effectiveness (Possidenteet al., 2005). MI strategies were assessed in the community pharmacy setting in a recent pilot study that concluded that readiness to receive hepatitis B or pneumococcal vaccines was improved after receiving MI (Brackettet al., 2015). MI strategies hold promise in improving immunization rates and these skills should be encouraged for pharmacists.

Collaborations with public and private partners

Recognizing that adult vaccinations remain lower than national goals set forth by Healthy People 2020, multiple initiatives are being considered within pharmacy practice to close the gap. Healthcare organizations are realizing the unique role that community-based immunization providers may play in reducing vaccine-preventable disease (US Department of Health and Human Services, 2017; National Vaccine Advisory Committee (NVAC), 2012).

(8)

by the CDC. In this model, the medical necessity for HPV vaccine was assessed and the first dose would be administered by a medical provider. The patient would then be referred to a community pharmacy setting where the subsequent doses could be administered, and the pharmacist would provide documentation back to the medical provider (Centers for Disease Control and Prevention (CDC), 2012).

The DHHS drafted a National Adult Immunization Plan (NAIP) to address ongoing barriers and challenges to adult immunization. This guidance document outlines four goals and 16 objectives that promote coordinated planning and action across US governmental and non-governmental stakeholders to generate action through 2020. The four goals are to: strengthen the adult immunization infrastructure, improve access to adult vaccines, increase community demand for adult immunizations, and foster innovation in adult vaccine development and vaccination-related technologies. Community immunizers are recognized as part of the immunization neighborhood and are listed as non-governmental stakeholders in the initiative to vaccinate. Upon review of the NAIP, a clear opportunity exists for community-based immunizers to aid in the execution of each of the four goals and many of the 16 objectives (US Department of Health and Human Services, 2015).

The Pharmacy Quality Alliance (PQA), a non-profit alliance with greater than 200 member organizations, collaboratively promotes appropriate medication use and develops strategies for measuring and reporting performance information related to medications. Measures developed by the PQA are commonly used in the health insurance marketplace and by the US Centers for Medicare and Medicaid Services. The PQA currently has 20 measures in effect, and pharmacies are being assessed on their ability to perform each measure. Although the measures are broadly distributed among medication use areas, there are three potential immunization-related measures under consideration or in draft status by the PQA. These measures include: immunization status assessment within medication therapy management, Advisory Committee on Immunization Practices (ACIP) compliance following immunization status assessment within medication therapy management, and ACIP schedule completion in patients with diabetes. Pharmacists can expect to see these measures implemented in their pharmacies in the upcoming months (Pharmacy Quality Alliance, 2017). As such, pharmacists should begin to assess their current practices and envision how immunization status assessment can be incorporated into day-to-day responsibilities.

In 2016, PQA released a white paper containing recommendations for improving adult vaccination rates and reporting within community pharmacy practice. A key recommendation set forward by the PQA was to identify immunization screening opportunities through querying IIS, assessing immunization status by utilizing the pharmacy dispensing software, leveraging an appointment-based model, and providing immunization screening during medication therapy management services. The PQA also encouraged community pharmacies to be advocates for immunization documenting and reporting to state or jurisdictional IIS, electronic health records, and other pertinent healthcare providers.

(9)

In September 2015, Assistant Surgeon General of the US Public Health Service, Anne Schuchat, acknowledged the significant advances for pharmacists as vaccine providers, and the contributions to improving overall immunization rates (Schuchat, 2015). In September of 2016, the director of the National Center for Immunization and Respiratory Diseases of the CDC recognized the contributions and potential of pharmacists as vaccine providers in a letter sent to the profession (Messonnier, 2016). She acknowledged that 280,000 pharmacists had been trained as immunizers and that 25 percent of influenza doses as of November 2015 were administered in community pharmacy settings. She encouraged pharmacists to continue these efforts with specific recommendations regarding the current seasonal influenza season and acknowledged the importance of promoting other vaccines as well. The director endorsed the concept of collaboration and the“immunization neighborhood.”

Impact of pharmacists as vaccine providers

Pharmacist participation in immunization programs has had a steady impact. In 2011, an estimated 18 percent of all influenza vaccine doses were administered in community pharmacy settings; by 2017, 25 percent of all influenza vaccines are administered in pharmacy settings (Schwerzmannet al., 2017). Patients have identified convenience as a factor for receiving their vaccinations in a pharmacy setting (Anderson and Thornley, 2014). They can usually receive a vaccine without an appointment and without additional fees that might accompany a medical visit.

A statewide survey of chain pharmacies conducted in California in 2007 reported that vaccines were available in 500 pharmacies and from 1,000 pharmacist-immunizers (Pilisuket al., 2010). These services were available on an expanded basis including evening and weekend hours, and provided on a walk-in basis. Influenza and pneumococcal vaccines were the most often administered vaccines. There was evidence of expansion beginning toward other vaccines and the respondents expressed interest in resources to keep up to date, and in collaborations with other vaccine partners.

A 2015 systematic review included 36 studies that assessed the impact of pharmacists on vaccine administration rates. Pooled analysis of the six randomized, controlled trials demonstrated significant increases in vaccination rates when pharmacists participated as educators, facilitators, or vaccine providers with an overall risk ratio (RR) of 2.74 (95% CI 1.58, 4.74) (Isenoret al., 2016a). Another systematic review concluded that although evidence exists that supports the fact that providing vaccines in community pharmacies would increase overall vaccination rates, there are legal, organizational and attitudinal barriers that should be addressed (Bursonet al., 2016).

The capacity of pharmacists to improve rates of seasonal influenza vaccination during a pandemic was estimated in a model simulation study in which possible rates were evaluated with or without retail pharmacy involvement. The investigators reported that retail pharmacist participation increased vaccination capacity to 25 million doses weekly during a pandemic, and that the time to achieve 80 percent coverage nationally was reduced by seven weeks (Schwerzmann et al., 2017). The authors concluded that pharmacists should be integrated into pandemic vaccine response planning efforts. Another evaluation assessed the status of collaboration between public health and community pharmacies in pandemic planning in 2014 as reported to the CDC. According to the results, 88.7 percent of public health programs included pharmacies in pandemic vaccine distribution plans, 45.3 percent maintained processes to recruit pharmacists as vaccine providers, and 30.8 percent had formal, established relationships with pharmacies (Fitzgeraldet al., 2016). These authors concluded that further efforts were warranted to fully engage pharmacists in preparation for a pandemic event.

(10)

pharmacists should improve education and awareness about the benefits of HPV vaccine, and improve the rates of completion of the vaccine series where initial doses may be received in a medical office and subsequent doses in a community pharmacy (Rothholz and Tan, 2016). A national survey (n¼1,255) conducted in 2014 reported that 29 percent of parents were willing for their adolescent children to receive the HPV vaccine in a pharmacy. The survey assessed parents’perceptions of pharmacist vaccination skills with seven items; responses to these items (on a five-point Likert scale) were averaged to create a scale of perceived pharmacist vaccination skills. Willingness was higher among parents who thought pharmacists were skilled at vaccine administration (OR¼2.05, 95% CI 1.07, 2.52, where OR is the change in odds of parental willingness to get HPV vaccine at a pharmacy for every one-unit increase in scale score) (Caloet al., 2017). Also of interest, willingness of parents to have their children vaccinated by a pharmacists varied depending on the specific vaccine with influenza at 62 percent and tetanus-toxoid, diphtheria-toxoid, acellular pertussis vaccine (Tdap) at 41 percent.

A multimodal strategy for a pharmacist-lead intervention was evaluated in an independent community pharmacy setting in Tennessee during an eight week pilot study in 2015. An educational intervention designed to raise awareness about HPV vaccination was directed to a targeted population in the pharmacy, nearby physician offices, and the general public in the community. A subsequent survey (n¼21) in the pharmacy reported that ten patients had received one or more doses of HPV vaccine during the study period with the pharmacist recommendation being the most common education methods for awareness of the vaccine (n¼10), as well as the reason for being vaccinated (n¼6). Commonly cited reasons for not being vaccinated were costs and lack of insurance coverage (Hohmeieret al., 2016).

A six-month pilot study conducted in three independent community pharmacies in Tennessee significantly increased the number of zoster vaccines administered from 59 to 193 (immunization rate increase from 0.37 to 1.20 percent,po0.0001) through local marketing strategies and advocacy during prescription refills (Wanget al., 2013). As described earlier, Rhodes and colleagues demonstrated that a vaccine screening program was successfully incorporated into a community pharmacy practice and relied largely on pharmacy technicians for implementation (Rhodeset al., 2017).

A study at a large chain pharmacy was conducted over a 16 month period to improve pneumococcal vaccination rates among high risk patients (Taitelet al., 2011). When patients (n¼2,095,748) were receiving influenza vaccine at a pharmacy in the system, they were screened by the pharmacist to determine if they were at high risk for pneumococcal disease as well and were offered pneumococcal polysaccharide 23-valent vaccine (PPSV23) if they qualified (n¼1,343,751). Among eligible patients, 4.9 percent received the pneumococcal polysaccharide vaccine (PPSV23) which was significantly higher than the benchmark reported for traditional care (2.9 percent).

Another study at a chain pharmacy located on the campus of a women’s hospital was a two-year effort to improve vaccination with Tdap for individuals in close contact with newborns (Millset al., 2014). Information about the importance of receiving the vaccine was shared with delivering families and a pharmacist held a vaccination clinic for two hours each day. The average monthly rate of Tdap vaccine administered during the study period was more than ten times higher than the rate at comparable pharmacies in the area (83.9 vaccines/month vs 2.7 vaccines/month, po0.01). The investigators reported that collaboration among all parties involved was a key to successful implementation of the program.

(11)

43 percent of the US population). Of the 4.5 million doses of influenza vaccine administered by the chain during the 2009-2010 influenza season, nearly 38 percent were administered to patients in medically underserved areas. The authors highlighted the potential impact for over 130 million Americans, and the convenience and access for receiving vaccines in community pharmacy settings.

In a meta-analysis of pharmacist-administered vaccine programs, investigators included eight studies and found that the overall RR for receiving a vaccine with pharmacist participation compared to a non-pharmacist-based approach was 2.95 (po0.001), and it varied among the specific vaccine administered. The relative risk was lowest for influenza (2.23) and highest for zoster vaccine (4.78) (Baroy et al., 2016). These data support the position that pharmacist-based immunization programs containing an advocacy component can significantly improve vaccine rates among adults, which will likely be needed to achieve national goals and initiatives for adult vaccination.

Global activities of pharmacists in vaccine education, advocacy, and delivery Although this review has focused on the evolution and implementation of vaccine services in the USA, there are numerous examples of the state of pharmacy practice in other countries. For example, pharmacist involvement in England and Canada has been substantial. Nonetheless, there are numerous countries where pharmacist involvement as vaccine providers has not been considered.

Pharmacist involvement in administering vaccines varies globally. Authority is granted to various extents in some developed countries, including Canada, Ireland, New Zealand, Portugal, and the UK. New authority is also emerging in parts of Australia (Hattingh et al., 2016). However, the potential benefits of including pharmacists as vaccine providers is largely unrealized globally (Isenoret al., 2016b).

In May 2012, the Ministers of Health from 194 countries convened at the World Health Assembly and endorsed a Global Vaccine Action Plan (GVAP, 2012). The action plan serves as a roadmap to“improve health by extending the full benefits of immunization to all people, regardless of where they are born, who they are, or where they live”by 2020 and beyond. The GVAP employs six guiding principles: country ownership, shared responsibility and partnership, equity, integration, sustainability, and innovation. There exists six strategic objectives and measurement indicators as well as recommended action steps action steps to ensure the achievement of the goals. It is hoped that the profession of pharmacy can be engaged with the GVAP in similar ways that US pharmacists have collaborated with the NVAC. A partnership would help the GVAP meet worldwide objectives.

The possible benefit of enlisting community pharmacists to provide influenza vaccine was evaluated in Canada. The investigators cited the historically low rate of influenza vaccination among its citizens, as well as patient interest and pharmacist willingness to provide this service. During the period of 2007-2014, the reported rate for influenza vaccination was 28.8 percent which would be increased by 2.2 percent if policy changes provided for pharmacist to provide the vaccine. The authors suggested that this modest increase may be beneficial if it improved awareness of overall strategies to reduce the risk of influenza infection. They also noted that 80 percent of citizens receive non-vaccine services from a pharmacist annually and could receive vaccine reminders (Buchanet al., 2017).

(12)

included safety and reporting and sharing of vaccination status among healthcare providers. Concerns were more common in areas of the country where pharmacists were not yet acting as vaccine providers (MacDougallet al., 2016).

A study from Nova Scotia evaluated influenza vaccination coverage rates as a result of adding pharmacists as vaccine providers in 2013. Vaccination rates for all residents were reported as 41.8 and 39.9 percent during the first two years, compared to 35.8 percent in the year prior to pharmacist authority. For patients at least 65 years of age in the community, the rates were 71.6 and 73.3 percent compared to historical rates of 61.8 percent prior to pharmacist participation (Isenor, 2016b).

A comprehensive literature review from the UK in 2010 focused on vaccination services in community pharmacies (Francis and Hinchliffe, 2010). At the time of the report, vaccine services in community pharmacy settings were widely available and focused primarily on influenza and travel vaccines, although other vaccines were offered. The convenience and access to community pharmacists was viewed as a benefit and the potential to improve national immunization rates was acknowledged although research in this area was limited. The impact of a community pharmacist initiative to provide seasonal influenza vaccine in London was assessed during 2011 and 2015. The reporting for vaccines received in pharmacies (2013-2015) was separate from the reports received from general practitioners in London (2011-2015). Influenza vaccination rates ranged from 59.7 to 60.4 percent during the study period and did not reflect any differences. The investigators reported that the cost to the National Health Service was lower when patients received influenza vaccine in a community pharmacy but that separate reporting systems was problematic and should be addressed (Atkinset al., 2016).

In the second year of a newly initiated program in community pharmacies in Western Australia, over 15,000 influenza vaccines were administered at 76 community pharmacies between March and October of 2015. There were no reports of serious adverse reactions. Pharmacists expressed confidence in their abilities as vaccine providers and patients were satisfied with the access and convenience of the services. Many of the vaccines were administered in rural and remote areas of the country (Hattinghet al., 2016).

Conclusion

Pharmacists have been involved with vaccine storage and handling for more than a century. During the 1990s, interest in engaging pharmacists as vaccine providers occurred within and outside of the profession. The involvement of pharmacists in administering vaccines is now common in the USA and there are examples of an increasing role globally. The evidence regarding the impact of pharmacists in improving vaccination rates is inconsistent, but substantial needs and opportunities are available for collaboration with other health professionals to improve the health of the public through vaccination.

References

American Pharmacists Association (2016), “Pharmacist administered vaccines”, American Pharmacists Association, Washington, DC, available at: www.pharmacist.com/sites/default/ files/PharmacistIZAuthority.pdf (accessed April 19, 2017).

Anderson, C. and Thornley, T. (2014),“ ‘It’s easier in pharmacy’: why some patients prefer to pay for flu jabs rather than use the National Health Service,BMC Health Services Research, Vol. 14 No. 35, pp. 1-6.

(13)

Bach, A.T. and Goad, J.A. (2015), “The role of community pharmacy-based vaccination in the USA: current practice and future directions”, Integrated Pharmacy Research and Practice, Vol. 4, pp. 67-77.

Baroy, J., Chung, D., Frisch, R., Apgar, D. and Slack, M.K. (2016), “The impact of pharmacist immunization programs on adult immunization rates: a systematic review and meta-analysis”,

Journal of the American Pharmacists Association, Vol. 56 No. 4, pp. 418-426.

Black, C.L., Yue, X., Ball, S.W., Donahue, S.M., Izrael, D., de Perio, M.A., Laney, A.S., Williams, W.W., Lindley, M.C., Graitcer, S.B., Lu, P.J., Bridges, C.B., DiSogra, C., Sokolowski, J., Walker, D.K. and Greby, S.M. (2015),Influenza vaccination coverage among health care personnel United States, 2014-15 influenza season”,Morbidity and Mortality Weekly Report (MMWR), Vol. 64 No. 36, pp. 993-999.

Brackett, A., Butler, M. and Chapman, L. (2015),“Using motivational interviewing in the community pharmacy to increase adult immunization readiness: a pilot evaluation”,Journal of the American Pharmacists Association, Vol. 55 No. 2, pp. 182-186.

Buchan, S.A., Rosella, L.C., Finkelstein, M., Juurlink, D., Isenor, J., Marra, F., Patel, A., Russell, M.L., Quach, S., Waite, N. and Kwong, J.C., Public Health Agency of Canada/Canadian Institutes of Health Research Influenza Research Network (PCIRN) Program Delivery and Evaluation Group (2017),“Impact of pharmacist administration of influenza vaccines on uptake in Canada”,

Canadian Medical Association Journal ( Journal de l’Association Medicale Canadienne), Vol. 189 No. 4, pp. E146-E152.

Burson, R.C., Buttenheim, A.M., Armstrong, A. and Feemster, K.A. (2016),“Community pharmacies as sites of adult vaccination: a systematic review”,Human Vaccines & Immunotherapeutics, Vol. 12 No. 12, pp. 3146-3159.

Calo, W.A., Gilkey, M.B., Shah, P., Marciniak, M.W. and Brewer, N.T. (2017),“Parents’willingness to get human papillomavirus vaccination for their adolescent children at a pharmacy,Preventive Medicine, Vol. 99, pp. 251-256.

Centers for Disease Control and Prevention (CDC) (1999), Ten great public health achievementsUnited States, 1900-1999,Morbidity and Mortality Weekly Report, Vol. 48 No. 12, pp. 241-243.

Centers for Disease Control and Prevention (CDC) (2011), “Ten great public health achievements – United States, 2001-2010”,Morbidity and Mortality Weekly Report, Vol. 60 No. 19, pp. 619-623.

Centers for Disease Control and Prevention (CDC) (2012), “Human papillomavirus-associated cancers – United States, 2004-2008”, Morbidity and Mortality Weekly Report, Vol. 61, pp. 258-261.

Centers for Disease Control and Prevention (2014),“How pharmacists can improve our nation’s health”, Public Health Grand Rounds, Centers for Disease Control and Prevention, Atlanta, GA, available at: www.cdc.gov/cdcgrandrounds/archives/2014/oct2014.htm (accessed May 5, 2017). Centers for Disease Control and Prevention (2016a), Standards for adult immunization practice,

Centers for Disease Control, Atlanta, GA, available at: www.cdc.gov/vaccines/hcp/adults/for-practice/standards/index.html (accessed April 25, 2017).

Centers for Disease Control and Prevention (2016b),“Immunization Information Systems (IIS)”, Centers for Disease Control and Prevention, available at: www.cdc.gov/vaccines/programs/iis/index. html (accessed April 19, 2017).

Centers for Disease Control and Prevention (2017a), “Recommended immunization schedule for children and adolescents aged 18 years or younger, United States”, Centers for Disease Control and Prevention, available at: www.cdc.gov/vaccines/schedules/hcp/child-adolescent.html (accessed May 31, 2017).

(14)

Department of Health and Human Services (2016),“Vaccine and vaccine administration payments under Medicare Part D”, Department of Health and Human Services Centers for Medicare and Medicaid Services, available at: www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNProducts/Downloads/Vaccines-Part-D-Factsheet-ICN908764.pdf (accessed May 31, 2017).

Emmons, K.M. and Rollnick, S. (2001),Motivational interviewing in health care settings. Opportunities and limitations”,American Journal of Preventive Medicine, Vol. 20 No. 1, pp. 68-74.

Fitzgerald, T.J., Kang, Y., Bridges, C.B., Talbert, T., Vagi, S.J., Lamont, B. and Graitcer, S.B. (2016),

“Integrating pharmacies into public health program planning for pandemic influenza vaccine response”,Vaccine, Vol. 34 No. 46, pp. 5643-5648.

Francis, M. and Hinchliffe, A. (2010), “Vaccination services through community pharmacy: a literature review, Public Health Wales NHS Trust, London, pp. 1-23.

Global Vaccine Action Plan (GVAP) (2012), available at: www.who.int/immunization/global_vaccine_ action_plan/GVAP_doc_2011_2020/en/ (accessed April 4, 2017).

Grabenstein, J., Hartzema, H., Guess, H., Johnston, W. and Rittenhouse, B. (1993), Community pharmacists as immunization advocates: a clinical pharmacoepidemiologic experiment”,

International Journal of Pharmacy Practice, Vol. 2 No. 1, pp. 5-10.

Grabenstein, J.D. (1999), “Pharmacists and immunization: increasing involvement over a century”,

Pharmacy in History, Vol. 41 No. 4, pp. 137-152.

Grabenstein, J.D., Hartzema, A.G., Guess, H.A., Johnston, W.P. and Rittenhouse, B.E. (1992),

“Community pharmacists as immunization advocates. Cost-effectiveness of a cue to influenza vaccination”,Medical Care, Vol. 30 No. 6, pp. 503-513.

Hattingh, H.L., Sim, T.F., Parsons, R., Czarniak, P., Vickery, A. and Ayadurai, S. (2016),“Evaluation of the first pharmacist-administered vaccinations in Western Australia: a mixed-methods study”,

BMJ Open, Vol. 6 No. 9, pp. e011948-2016-011948.

Hogue, M.D., Grabenstein, J.D., Foster, S.L. and Rothholz, M.C. (2006),“Pharmacist involvement with immunizations: a decade of professional advancement”,Journal of the American Pharmacists Association, Vol. 46 No. 2, pp. 168-179.

Hohmeier, K.C., Randolph, D.D., Smith, C.T. and Hagemann, T.M. (2016),A multimodal approach to improving human papillomavirus vaccination in a community pharmacy setting”,SAGE Open Medicine, Vol. 4, pp. 1-5.

Hurley, L.P., Wortley, P., Allison, M.A., O’Leary, S., Daley, M.F., Babbel, C., Crane, L.A., Stokley, S., Beaty, B., Dickinson, L.M. and Kempe, A. (2011),“Seasonal influenza vaccination in adults: practice and attitudes about collaborative delivery with community vaccinators, Vaccine, Vol. 29 No. 47, pp. 8649-8655.

Immunization Action Coalition (2017), Adults: standing orders, Immunization Action Coalition, St Paul, MN, available at: www.immunize.org/handouts/adult-vaccination.asp# standingorders (accessed May 31, 2017).

International Society of Travel Medicine (2017),“Certificate of knowledge”, International Society of Travel Medicine, Dunwoody, GA, available at: www.istm.org/certificateofknowledge (accessed April 19, 2017).

Isenor, J.E., Edwards, N.T., Alia, T.A., Slayter, K.L., MacDougall, D.M., McNeil, S.A. and Bowles, S.K. (2016a),“Impact of pharmacists as immunizers on vaccination rates: a systematic review and meta-analysis”,Vaccine, Vol. 34 No. 47, pp. 5708-5723.

Isenor, J.E., Killen, J.L., Billard, B.A., McNeil, S.A., MacDougall, D., Halperin, B.A., Slayter, K.L. and Bowles, S.K. (2016b),“Impact of pharmacists as immunizers on influenza vaccination coverage in the community-setting in Nova Scotia, Canada: 2013-2015”,Journal of Pharmaceutical Policy and Practice, Vol. 9, pp. 1-16.

(15)

Kempe, A., Wortley, P., O’Leary, S., Crane, L.A., Daley, M.F., Stokley, S., Babbel, C., Dong, F., Beaty, B., Seewald, L., Suh, C. and Dickinson, L.M. (2012),“Pediatricians’attitudes about collaborations with other community vaccinators in the delivery of seasonal influenza vaccine”,Academic Pediatrics, Vol. 12 No. 1, pp. 26-35.

McConeghy, K.W. and Wing, C. (2016),“A national examination of pharmacy-based immunization statutes and their association with influenza vaccinations and preventive health”, Vaccine, Vol. 34 No. 30, pp. 3463-3468.

MacDougall, D., Halperin, B.A., Isenor, J., MacKinnon-Cameron, D., Li, L., McNeil, S.A., Langley, J.M. and Halperin, S.A. (2016),“Routine immunization of adults by pharmacists: attitudes and beliefs of the Canadian public and health care providers”,Human Vaccines & Immunotherapeutics, Vol. 12 No. 3, pp. 623-631.

Messonnier, N. (2016),“Dear pharmacist”, Centers for Disease Control and Prevention, Atlanta, GA, available at: http://media.pharmacist.com/documents/Dear_Pharmacist_letter_9-2016.pdf (accessed August 3, 2017).

Mills, B., Fensterheim, L., Taitel, M. and Cannon, A. (2014),Pharmacist-led Tdap vaccination of close contacts of neonates in a women’s hospital”,Vaccine, Vol. 32 No. 4, pp. 521-525.

Murphy, P.A., Frazee, S.G., Cantlin, J.P., Cohen, E., Rosan, J.R. and Harshburger, D.E. (2012),“Pharmacy provision of influenza vaccinations in medically underserved communities,Journal of the American Pharmacists Association, Vol. 52 No. 1, pp. 67-70.

NACDS (2010-2011), “Chain pharmacy industry profile illustrates pharmacy value”, Drug Topics, available at: http://drugtopics.modernmedicine.com/drug-topics/news/ modernmedicine/modern-medicine-news/nacds-2010-2011-chain-pharmacy-industry-profile? page=full (accessed February 17, 2016).

National Vaccine Advisory Committee (NVAC) (2012),“A pathway to leadership for adult immunization: recommendations of the National Vaccine Advisory Committee: approved by the National Vaccine Advisory Committee on June 14, 2011,Public Health Reports, Vol. 127 No. S1, pp. 1-42. National Vaccine Advisory Committee (2016),“Overcoming barriers to low HPV vaccine uptake in the

United States: recommendations from the National Vaccine Advisory Committee, approved by the National Vaccine Advisory Committee on June 9, 2015,Public Health Reports, Vol. 131 No. 1, pp. 17-25.

New York State Department of Health (2012), “Non-patient specific order for pharmacists administering influenza vaccine to adults”, New York State Department of Health Bureau of Immunization, New York, NY, available at: http://c.ymcdn.com/sites/www.pssny.org/resource/ resmgr/docs/standing_orders_administerin.pdf (accessed May 31, 2017).

North Carolina General Statutes (2015),“Immunizing pharmacists”, 95.85.15B, North Carolina General Statutes, Raleigh, NC, available at: www.ncleg.net/EnactedLegislation/Statutes/PDF/BySection/ Chapter_90/GS_90-85.15B.pdf (accessed April 20, 2017).

Pharmacy Quality Alliance (2017),“Pharmacy Quality Alliance”, available at: www.pqaalliance.org (accessed May 1, 2017).

Pilisuk, T., Goad, J. and Backer, H. (2010),“Vaccination delivery by chain pharmacies in California: results of a 2007 survey”,Journal of the American Pharmacists Association ( JAPhA), Vol. 50 No. 2, pp. 134-139.

Possidente, C.J., Bucci, K.K. and McClain, W.J. (2005),Motivational interviewing: a tool to improve medication adherence?”,American Journal of Health-System Pharmacy: Official Journal of the American Society of Health-System Pharmacists, Vol. 62 No. 12, pp. 1311-1314.

Rhodes, L.A., Branham, A.R., Dalton, E.E., Moose, J.S. and Marciniak, M.W. (2017),Implementation of a vaccine screening program at an independent community pharmacy,Journal of the American Pharmacists Association, Vol. 57 No. 2, pp. 222-228.

(16)

Rothholz, M. and Tan, L.L. (2016),“Promoting the immunization neighborhood: benefits and challenges of pharmacies as additional locations for HPV vaccination”, Human Vaccines & Immunotherapeutics, Vol. 12 No. 6, pp. 1646-1648.

SchoolVaxView (2017),“Centers for Disease Control and Prevention”, available at: www2a.cdc.gov/nip/ schoolsurv/schImmRqmt.asp (accessed May 31, 2017).

Schuchat, A. (2015),Dear pharmacists letter, American Pharmacists Association, available at: www. pharmacist.com/sites/default/files/files/CDC_Dear_Pharmacists_Letter_AS_09282015.pdf (accessed April 20, 2017).

Schwerzmann, J., Graitcer, S.B., Jester, B., Krahl, D., Jernigan, D., Bridges, C.B. and Miller, J. (2017),

“Evaluating the impact of pharmacies on pandemic influenza vaccine administration,Disaster Medicine and Public Health Preparedness, pp. 1-7, available at: www.cambridge.org/ core/services/aop-cambridge-core/content/view/E216D8FC9C93DC8948C529C5080E98CA/ S1935789317000015a.pdf/evaluating_the_impact_of_pharmacies_on_pandemic_influenza_ vaccine_administration.pdf

Taitel, M., Cohen, E., Duncan, I. and Pegus, C. (2011), “Pharmacists as providers: targeting pneumococcal vaccinations to high risk populations”,Vaccine, Vol. 29 No. 45, pp. 8073-8076. US Department of Health and Human Services (2015), National Adult Immunization Plan,

US Department of Health and Human Services, Washington, DC.

US Department of Health and Human Services (2017),“Immunization and infectious diseases”, Healthy People 2020, US Department of Health and Human Services, available at: www.healthypeople. gov/2020/topics-objectives/topic/immunization-and-infectious-diseases (accessed April 15, 2017). Wang, J., Ford, L.J., Wingate, L., Uroza, S.F., Jaber, N., Smith, C.T., Randolph, R., Lane, S. and Foster, S.L. (2013),Effect of pharmacist intervention on herpes zoster vaccination in community pharmacies,

Journal of the American Pharmacists Association, Vol. 53 No. 1, pp. 46-53.

Watson, L. and Bluml, B. (2013),Pharmacy’s Appointment Based Model, July ed., APhA Foundation, Washington, DC, available at: www.aphafoundation.org/sites/default/files/ckeditor/files/ ABMImplementationGuide-FINAL-20130923.pdf

Westrick, S.C. and Breland, M.L. (2009),“Sustainability of pharmacy-based innovations: the case of in-house immunization services”,Journal of the American Pharmacists Association, Vol. 49 No. 4, pp. 500-508.

Further reading

American Pharmacists Association (2013),“Annual pharmacy-based influenza and adult immunization survey”, American Pharmacists Association, available at: www.pharmacist.com/sites/default/ files/files/Annual%20Immunization%20Survey%20Report.pdf (accessed April 12, 2017). Black, C.L., Yue, X., Ball, S.W., Donahue, S.M., Izrael, D., de Perio, M.A., Laney, A.S., Williams, W.W.,

Lindley, M.C., Graitcer, S.B., Lu, P.J., DiSogra, C., Devlin, R., Walker, D.K. and Greby, S.M. (2016),

“Influenza vaccination coverage among health care personnel–United States, 2015-16 influenza season,Morbidity and Mortality Weekly Report, Vol. 65 No. 38, pp. 1026-1031.

National Vaccine Advisory Committee (2015), “US Department of Health and Human Services”, available at: www.hhs.gov/nvpo/nvac/ (accessed April 12, 2017).

Corresponding author

References

Related documents

To To Duration Duration Change(s) Change(s) Train Train Accommodations Accommodations Days running Days running From

 Does the test-suite provide good enough data on performance on a platform you don't have access to, but for which public performance tracking bots give you feedback. 

The Australian Government’s Guiding Principles for Civil-Military-Police Interaction in International Disaster and Conflict Management (the Guiding Principles) has been developed

From the point of view of the financial institutions the, obstacles for lending to `SMEs, both start-up and existing, include (a) inadequate collateral to secure loan; (b) lack of

Instead, it will focus on an- alyzing several outstanding pitfalls in the current CTC technologies and clinical research, followed by, as an important contribution of this article,

Initially, the catalytic epoxidation of olefins was carried out under a nitrogen atmosphere, using omv-stilbene as a model substrate, 10% catalyst, and CU 3 CN as the

Isto surtirá mais efeito do que atentar a ideais ou então remeter ao sofrimento dos outros, (...). Entretanto, apesar de toda repressão psicológica, Estalingrado e