Starting a 340B program is not a random event. The selection elements that predispose
which HTCs start 340B programs, when, and under what CFIR related inner and outer
conditions need to be systematically examined in order to clarify the consequences of facilitating
and hindering 340B adoption and implementation. Four Evaluation questions are posed to help
address AIM 1.
EVALUATION QUESTION AIM 1a: What has been the pattern of adoption of 340B
programs by HTCs in each region? The 340B program has a twenty year history. HTCs were
eligible from the start. The HTCs had a stable regional configuration throughout this time period
fostering the availability of archival data. These structural elements permit a retrospective look
of when HTCs began to adopt 340B. Plotting the timing of 340B start by region is a necessary
first step to understand the changing and unchanging determinants of adoption patterns. The
CFIR constructs and subcategories previously described underpin this question.
To answer this first evaluation question, the evaluation should examine the time series
(Campbell et al., 1963) of HTC 340B adoption. Evaluators should first plot the number and
proportion HTCs that have started 340B programs by year, overall and by region, and then
annotate the chart with key events that might have influenced individual HTCs to decide to
adopt the program. The sampling plan is to include the entire population of HTCs within each
region during the entire twenty year period 1992 – 2012; this reduces selection and history bias.
The data source is the HRSA Comprehensive Care Grant progress report. This report is
41
340B. The unit of analysis should be the individual HTCs within a region. For this bivariate
analysis, the independent variable X = year 340B started selling 340B priced factor;38 the
dependent variable Y = HTC identifier. The analysis is to display data as a regional time series
map. An example of the type of display that would be useful in plotting trends is presented in
Figure 5, for the Western States Regional Hemophilia Network. In this figure, O is placed in the
column for each year that an HTC has a HRSA comprehensive hemophilia care grant, making
the HTC an eligible 340B entity. For each year that an HTC operates a 340B program, an “X” is
also placed in the cell.
Figure 5: Hemophilia Treatment Centers Operating 340B Programs 1992–2012 Western States Regional Hemophilia Network
`
Preliminary results of the evaluation efforts in the Western States HTC Region reveal four distinct phases. The time series map indicates a six year dearth phase (1992 – 1998)
when no HTC operated a pharmacy using 340B prices. Next was a five year early adopter
period (1999 – 2003) wherein three HTCs began 340B programs, raising the proportion of HTCs in this region that were operating a 340B program from 0% - 20%. In the five year cascade
(2004 – 2008) that followed, seven HTCs began 340B programs, bringing the proportion of
HTCs operating a 340B program up to 71.4% by 2008. In nearly each of the cascade years,
38
Registration on the OPA website is a necessary precursor to selling outpatient drugs using 340B prices. It indicates that an entity is eligible to do and has successfully completed the registration process. But registration in itself is does not equate with the successful implementation of all the operational processes involved in drug sales.
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 A O O O O O O O XO XO XO XO XO XO XO XO XO XO XO XO XO XO A B O O O O O O O O O XO XO XO XO XO XO XO XO XO XO XO XO B C O O O O O O O O O O XO XO XO XO XO XO XO XO XO XO XO C D O O O O O O O O O O O O XO XO XO XO XO XO XO XO XO D E O O O O O O O O O O O O XO XO XO XO XO XO XO XO XO E F O O O O O O O O O O O O O XO XO XO XO XO XO XO XO F G O O O O O O O O O O O O O XO XO XO XO XO XO XO XO G H O O O O O O O O O O O O O O XO XO XO XO XO XO XO H I O O O O O O O O O O O O O O O XO XO XO XO XO XO I J O O O O O O O O O O O O O O O O XO XO XO XO XO J K O O O O O O O O O O O O O O O O XO XO XO XO XO K L O O O O O O O O O O O O O O O O O O XO XO XO L M O O O O O O O O O O O O O O O O O O O O O M N O O O O O O O O O O O O O O O O O O O O O N
42
340B became operation at one and sometimes two HTCs. One more HTC began a 340B
program during the last, or late adopter phase (2009 – 2012). By 2012, 340B was operational at
twelve (85.7%) of the fourteen WSRHN HTCs. Furthermore, the map suggests 340B
sustainability: once a 340B program started, it continued operating unabated. This time trend
map of regional 340B adoption trends is useful: the metric provides a clear and comprehensive
illustration and suggests that something(s) occurred to trigger the cascade period, warranting
qualitative investigation via the next metric.
HTC 340B adoption patterns likely vary by region, as an HTC 340B program is a
complex intervention. Some regions may exhibit clusters of 340B adoption followed by seminal
phenomena that appear to stimulate clusters of intermittent or steady adoption. A time series
map spanning 1992 – 2012 is thus a valuable first analytic step to obtain a regional 340B
adoption and spread gestalt.
To further address evaluation question 1a, the evaluator next should conduct multiple
regression analyses with the year of adoption the dependent variable and characteristics of the Centers as explanatory variables to assess whether certain characteristics are associated with
earlier or later adoption. The sample should include the entire population of HTCs within each
region during the entire twenty year period 1992 – 2012. The data sources are the CDC
Universal Data Collection Surveillance (UDC) and the new organizational survey. The measure
is the time to adopt 340B.
Because early or late adoption may be influenced by both unchanging Center
characteristics, such as type of institution that hosts an HTC (e.g. large academic medical
center or small community hospital), and by historic events, such as changes in Medicaid
payment policy, evaluators should use hazard models to model time to adoption with
43
variables to be included in the regression to reflect non-time varying differences in Centers and
the historic events that should be coded as time varying covariates are listed in Table 9.
Table 9: Multiple Regression Model: Time Varying and non-Time Varying Covariates potentially influencing HTC 340B Adoption – AIM 1a
CFIR Construct
Subcategory Time Varying Covariates Non-Time Varying Covariates
Intervention
Relative Advantage % yes – 340B can bring in resources
to HTC
Complexity
% yes – have now or institution will provide sufficient resources to operate 340B program: line of credit, pharmacy, insurance authorization, billing, contracting with manufacturers and insurance, marketing, accounting
Costs % yes – have sufficient $ for factor inventory % yes – have separate refrigerator
Outer Setting
Patient needs/
resources % yes – HTC clinical & admin staff and service sufficient to meet patient needs
External policy/incentives
% Medicaid, % Uninsured
% yes - Medicaid factor reimbursement
rate is adequate
Inner Setting
Structural % HTC Director Tenure = <5 years, 5 – 10 years, 10> years
% Host institution academic medical center, community hospital, freestanding clinic % HTC patient base <100, 100-200, 300>
Networks % yes – I consult with a knowledgeable person in region about 340B
Implementation Climate % yes – HTC too busy to make initiate improvements
Readiness % yes – Human resources for 340B implementation is sufficient
Individuals % yes – 340B is valuable to HTC
Processes
Engaging
% yes – I consult with a knowledgeable person outside the region about 340B % yes – attended >1 340B educational conference
EVALUATION QUESTION AIM 1b: What organizational, financial and other dynamics influenced which Hemophilia Treatment Centers were early adopters, adopters, late adopters, non-adopters? Implementation research increasingly reveals the influence of organizational determinants on innovation and health care quality (Fixsen et al., 2005; Flood,
1994). To understand the variation in timing and extent of implementation of 340B programs
44
as part of the evaluation. The multivariate regression modeling described above will provide
some insight into these issues but should be augmented with qualitative information that is more
nuanced in capturing variations across the HTCs. The plan for evaluation should incorporate
methods to capture financial information and other dynamics into the sampling plans, measures,
data collection instruments and analyses. Definitions of adopter phases should be determined at
the regional level, informed by local findings from the time series analyses.
Organizational influences on 340B adoption should be assessed through examining data
derived from a new organizational survey, new qualitative semi-structured interviews, and
archival records from a 340B educational series.
Organizational heterogeneity and changes over time: The evaluator should use a
pre/post design to assess the range of organizational structures and processes, and how their change over time influences 340B adoption. The first time period should be the two years prior
to HTC 340B adoption. The second time period should be current.
The sampling plan should include all HTCs that operate a 340B program. The inclusion
criterion for the ‘adoption’ time period is the universe of regional HTC and host institution
stakeholders directly engaged in HTC 340B planning and implementation. These comprise HTC
Directors, Nurse Coordinators, and host institution officials from administration, finance,
contracting, and pharmacy. As the 340B adoption could have occurred 20 years ago, sample
attrition is a risk. Therefore, to identify the subset of sample members who have historic
information about 340B adoption from the early 1990’s, the evaluator should solicit information
from the Regional Directors and Coordinators (Coyne, 1997; Patton, 2005). The inclusion
criteria for the ‘current’ time period should be HTC Directors and host institution leaders with
45
The evaluator should develop a new data source: an organizational survey that will
provide more detailed and specific data about each HTC than is currently available from
archived data. The evaluator should create this survey by identifying key domains and
constructs, reliable and valid items and scales, by conducting cognitive interviews, and then
pilot testing the instrument with representative respondents. The evaluator should consider
adapting items and scales with demonstrated psychometric properties from surveys used by the
Veterans Healthcare Administration (VA) to assess organizational structure and process
predictors of innovation implementation. The VA surveys contain thirty common organizational
structure and process characteristics (EM Yano, 2008). These characteristics represent size,
academic affiliation, clinical service availability, leadership and authority, consultant availability,
resource sufficiency, organizational climate, case mix, and staffing. See Appendix 2 for CFIR
constructs mapped to organizational survey measures, variables, data sources and availability.
The analysis should start with descriptive (univariate) statistics of the adoption and
current time periods. The unit of analysis should be the individual HTCs, and analysis initially
should be conducted region by region to allow patterns to be more easily observed. The
evaluator should construct means and standard deviations for interval variables (e.g. FTE of
HTC clinicians by discipline), percentages for nominal variables (e.g. institutional affiliation =
academic medical center, community hospital or freestanding clinic), or ordinal variables (e.g.
HTC Director Tenure = <5 years, 5 – 10 years, 10> years). Next, bivariate analyses should be
conducted to examine changes in organizational characteristics over the two time periods: chi-
square for categorical variables, and either t tests or Pearson’s r for interval or ratio variables.
Statistical tests should be conducted to determine statistically significant association between
specific organizational characteristics and time of 340B adoption. The proposed dependent
46
variable (X) should be organizational characteristics. Preliminary information suggests this
hypothesis: increased HTC Director tenure is associated with earlier 340B adoption.
After the analysis of bivariate associations of individual variables with time to adoption,
the evaluator should use appropriate multivariate methods to further analyze time to
adoption. Because some of the variables that influence adoption vary over time, the evaluator
should use hazard (time to event) models with time varying covariates. The variables that
should be included in the model, non-time varying and time-varying, were identified in Table 9.
Preliminary results of the evaluation effort for the organizational survey: As part of the field pilot, over ten key informants critiqued the first draft of the sampling plan, proposed topic
areas, and measures. The informants include HTC Directors, Nurses, and Pharmacists;
Regional Coordinators; National Hemophilia Foundation leadership; and HTC 340B experts --
their recommendations were incorporated.
Qualitative semi-structured interviews: Data on the HTCs should also be collected
through interviews of key parties, and semi-structured interviewing is recommended. Semi-
structured interviews foster understanding the how’s and why’s of phenomenon by listening to the actual actors involved, in their own words and observing non-verbals. (Coffey et al., 1996).
The sampling plan should be identical to that described to conduct this evaluation question’s
organizational survey. This is a new data source. The evaluator should create an interview
guide with questions and probes that are informed by CFIR constructs and subcategories
relevant to organizational, financing, and other dynamics influencing HTC 340B program
adoption. See Appendix 3 for a list of topic areas and illustrative questions mapped to the CFIR
constructs, and Appendix 4 for the Interview Guide. The qualitative analytic methods should
47
and non-verbals into analyzable data or ‘codes’ whose content can be linked conceptually into
patterns (Charmaz, 2006; Coffey et al., 1996; Saldaña, 2012).
Preliminary results of the evaluation effort for the semi-structured interview: The same key informants discussed above critiqued the first iteration of the semi structured interview
guide, and proposed sampling plan. Key informants judged the instruments’ contents
comprehensive, and the burden level to be reasonable. Stakeholders recommended adding the
commercial factor vendor sector into sampling plan, and refining selected measures, questions,
and probes. Their recommendations were incorporated.. Patterns identified in the time trends
chart also propelled modifications to the interview guide, reaffirming the benefits of the proposed
sequence of first chart making then conducting semi-structured interviews.
Preliminary results of the evaluation efforts: Data analysis from the preliminary design work to answer Evaluation Questions 1a and 1b were used to create this second run chart. It
suggests that CFIR outer setting/policy dynamics, specifically a policy change in the California
Medicaid factor reimbursement rate, and the execution of regional 340B Technical Assistance
48
FIGURE 6: Timeline - HTC 340B Adoption and Spread Field Pilot, Western States Regional HTC Network, 1992 - 2012.
Educational Series: The evaluator should use a time series design to examine the
influence of educational activities on 340B adoption. The evaluator should examine archival
documents from regional 340B Technical Assistance seminars conducted to boost the
knowledge and skills of HTC clinicians and host institutions about 340B legal, fiscal, and
business operations. No sampling method should be used; the entire retrospective data set is
available, based on region. The evaluator should use these data sources: seminar registration
forms and formal post event evaluations. The measurements should be the proportion of each
region’s HTCs that attended the seminars, numbers of individuals who attended by discipline,
and the total proportion of attenders reporting that the seminar provided valuable information.
The unit of analysis should be the individual HTC. Bi-variate analyses should be conducted and
arrayed as a table, with the independent variable X = HTC attendance and seminar value rating,
49
Table 10: 340B Technical Assistance Seminars, Western States Regional Hemophilia Network, 2003- 2008 %
HTCs Attending Persons MD Nurse Pharmacy Business Patient Leader Health State Dept. 39 % Rating Seminar as Valuable 2003 87% 31 11 4 4 11 0 2 100% 2004 60% 20 3 3 3 10 1 0 100% 2005 71% 24 4 4 4 10 2 0 100% 2006 79% 40 7 9 9 13 1 1 96% 2007 79% 37 6 6 7 17 1 0 100% 2008 86% 4840 8 8 7 24 0 1 93%
On average, 80% of the WSRHN HTCs sent representatives to each Seminar. The
majority of the attenders were HTC Medical Directors, Nurses, Pharmacists, and, increasingly,
business officials from finance, contracting, grants, purchasing, billing, and administration.
California State Health Department leaders from Medicaid and the Children and Adults with
Special Healthcare Needs Departments attended and presented as did from the patient support
organization leaders. The formal evaluations document the seminars’ value.
EVALUATION QUESTION AIM 1c: Once a 340B program is operational, what have been the patterns of patient enrollment? Stakeholders indicate that a 340B program needs to
enroll a sufficient volume of HTC patients who have insurance that reimburses factor
adequately. Adequate reimbursement for factor is needed to generate income that can then be
used to build capacity and thereby serve more patients and expand services. Examining
patterns of 340B patient enrollment contributes to the evaluation in various ways. It is the first
step in assessing which CFIR constructs might influence growth barriers and facilitators.
Answering this question also adds transparency to the HTC 340B landscape by revealing the
HTC 340B factor market share. Stakeholders seek this transparency to address complaints that
HTCs may be pressuring patients to enroll in 340B (The Lewin Group, 2012).
39
2003 = California Children’s Medical Services; 2006 and 2008 = Medicaid Pharmacy Policy Unit 40
50
The evaluator should use a time series design to examine the patterns of patient
enrollment in 340B. The sampling plan should include the entire population of HTCs within
each region. The evaluator should use two existing data sources which provide information on
340B enrollment, are national in scope, and can be aggregated at the individual HTC or regional
levels. The data sources are: 1) The Factor Replacement Product (FRP) report and Hemophilia
Data Set (HDS). Both data sets are currently available and retrievable depending on region.