of DCF staff by level, as well as the roles and responsibilities of community partners. It also examines staff and partner experiences with ACCESS Florida to identify any potential impacts the changes may have on issues such as client access, efficiency, worker satisfaction and case errors.
4. Examine the Knowledge, Experiences, and Satisfaction of Clients. The study also explores the ACCESS Florida model from the client’s perspective.
We examine how FSP participants, as well as potential clients, view the new technologies and the new application and case maintenance procedures under ACCESS Florida.
7
I: Introduction 5. Track FSP Outcomes over Time. Finally, the study examines how FSP
outcomes have changed since the implementation ACCESS Florida. Outcomes include FSP participation patterns (overall and for key subgroups), application rates, payment accuracy, and administrative costs.
Although the study reflects ACCESS Florida from July through September 2006, it is important to understand that DCF made changes to the technologies, policies, and procedures that comprise ACCESS Florida leading up to, during, and after this study was conducted. Thus, some findings regarding staff and client experiences may have been different if the study were conducted at an earlier or later date.
RESEARCH APPROACH
To meet the research objectives, this case study uses data from three primary sources:
(1) interviews with DCF and community partner staff, (2) focus groups of clients, and (3) administrative data on FSP outcomes. A systematic analysis of data collected through interviews helped to document which changes were made and identify the potential impacts of the changes. This information was then compared with administrative data for Florida (and data on Florida was compared with data for other states) to identify any evidence of those impacts.
Below, we describe data collection activities, the DCF districts that participated in collection (including the rationale for their selection), and summarize limitations of the research approach.5
Data Collection
Table I.1 shows the data sources that were used to address each research objective. Our approach to data collection was influenced by the need to triangulate findings – that is, to assess whether multiple sources of data support each finding – and by the desire to examine variations in findings across the state. In order to triangulate findings, the data collection approach needed to include some redundancy. Specific queries were researched by asking the same interview questions to multiple individuals, and by pooling information across multiple data sources. In order to examine how findings vary by location, data were collected throughout the state in different types of communities.
Staff Interviews. DCF staff at all levels, and key community partner staff, were interviewed to collect information relevant to all five research objectives. Specific types of staff included:
5 Appendix B provides additional details on the number and location of staff interviews and focus groups.
8
I: Introduction
• State DCF policy and technology staff
• DCF district administrators and policy staff
• DCF Customer Service Center directors, intake specialists, case processors, supervisors, and clerks
• DCF Customer Call Center directors, call agents and supervisors
• DCF Case Maintenance Unit directors, processors and supervisors
• Community partner organization directors and line staff
Table I.1. Research Objectives and Data Collection Activities
Research Objectives
Interviews were conducted in 7 of the DCF’s 14 administrative districts. In four districts, interviews were conducted primarily during in-person site visits (although phone interviews with other staff in three of these districts also were conducted). These visits also included tours of DCF Customer Service Centers, Customer Call Centers, Case Maintenance Units, and community partner locations. In three districts, interviews were conducted via telephone. All interviews were conducted using a structured protocol. The protocols were tailored to the different types of staff that were interviewed, and similar questions were asked of similar types of staff. Interviews were generally conducted in small groups (typically two or three staff members of the same or similar levels in a given Customer Service Center) and lasted approximately one hour.
To supplement the staff interviews, the research team reviewed DCF documentation on ACCESS Florida. This included planning documents created by DCF when designing ACCESS Florida, various DCF implementation guides and policy manuals, and technical documentation on software used as part of ACCESS Florida.
Client Focus Groups. To gauge client satisfaction with the FSP under ACCESS Florida, we convened 12 separate focus groups of FSP clients and potential clients. The focus groups were designed to elicit client comments on the accessibility of the FSP after
9
I: Introduction modernization, and on client experiences with the various components of ACCESS Florida (including the web-based application, the Customer Call Centers, and the community partners). Clients were asked to compare experiences with ACCESS Florida to prior experiences with DCF and to experiences with other government agencies, such as the Department of Motor Vehicles. Clients also were asked what aspects, if any, of ACCESS Florida would discourage them or others from participating in the FSP.
Focus groups were convened in three DCF districts. In each district, four separate focus groups were conducted, each with a slightly different definition of client:
• New Participants. New participants included individuals who recently (within three months of the focus group) applied for FSP benefits and were participating in the FSP for the first time. These groups provided insights about application procedures and whether program changes implemented under ACCESS Florida may increase access to individuals who might not have participated before.
• Participants with Prior Participation. Participants with prior participation included individuals who recently (within two months of the focus group) applied for FSP benefits and also had prior experience with the FSP (before ACCESS Florida). These groups provided insight into application procedures and whether clients felt the changes implemented under ACCESS increased or decreased program access.
• Recent Applicants. Recent applicants included individuals who recently (within two months of the focus group) applied for FSP benefits and may or may not have become program participants. These groups may have included FSP participants who also qualified for one of the preceding groups, as well as individuals who submitted an application but never participated, either because they were deemed ineligible or because they did not complete some of the application requirements. This group provided insight into application procedures and clients’ perceptions about how changes in these procedures affected program access.6
• Non-participants who are Likely Eligible. Non-participants included individuals who were probably eligible for food stamps but were not participating in the program and had not done so in the previous 12 months.
This group provided insight into whether there are aspects of ACCESS Florida that discourage participation among eligible individuals.
6 Ideally, to explore questions about problems with the modernization procedures, we would prefer to limit this group to just those individuals who did not complete the application requirements. However, it was not possible to identify those individuals in advance.
10
I: Introduction
The 12 focus groups included an average of 10 participants each.7 The focus groups were conducted using structured discussion guides that were tailored to each of the four groups of clients and designed to facilitate an exchange of ideas among group members.
Administrative Data. To measure FSP-related outcomes over time, we acquired administrative data from DCF and FNS. These data were used to measure changes in participation patterns, changes in application rates, changes in payment errors and changes in administrative costs. A key source of administrative data was quarterly extracts from DCF case records. These extracts included data on each household participating in the FSP every third month from July 2001 through July 2007. We used these data to tabulate the number of FSP participants over time, by district and by individual characteristics (such as elderly or disabled). In addition to case record extracts, DCF provided the number of community partners, data on the volume of usage at the call centers and monthly statistics on the number of applications received. FNS provided data on FSP administrative costs and payment error rates for Florida and for other states.
Selection of Study Sites
Data collected via staff interviews and focus groups were collected in seven of DCF’s 14 districts. We selected four districts for in-person staff interviews. In three of those four districts, we also conducted focus groups with clients. We selected three additional districts for staff interviews conducted via telephone. The DCF districts that participated in the data collection were selected intentionally, not by a random process. Our selections were based on the following objectives:
• Diversity of Districts. We wanted to observe ACCESS Florida operations in diverse settings. This meant that we wanted districts in different locations of the state and with different urban/rural compositions. We also wanted districts with varied population characteristics, in part to see whether ACCESS Florida changes affect some populations (for example, elderly people, individuals with limited English skills) differently than others.
7 Participants in all four groups received a $25 incentive payment. To recruit participants for the first three groups – all of which included individuals who applied for FSP benefits – we obtained client contact information from DCF. To recruit participants for the fourth group – for which, by definition, DCF has no contact information – we recruited individuals utilizing emergency food services in the area. Previous research has demonstrated that a large proportion of people who use emergency food centers are not enrolled in the FSP. While eligible non-participants who are using emergency food services may be systematically different from other eligible non-participants, they are different in a way that makes them particularly relevant for efforts to improve access since they demonstrate a need for food assistance. Approximately one week before each focus group, a field interviewer conducted a short screening interview among willing individuals using the emergency food center. Individuals were asked whether they were enrolled in the FSP, and if not, the interviewer explained the purpose of the focus group and asked if they would be willing to participate. The interviewer used a short screening tool to identify whether the individual was likely to be eligible for food stamps.
11
I: Introduction
• Variation in Access Issues. We wanted to select districts that had different program access environments. Specifically, we wanted districts with one or more counties that had no Customer Service Centers. We also wanted a district with a relatively high number of community partners and a district with a relatively low number of community partners.
• Not Visited Under Planning Phase. We did not select any sites that were part of the planning phase of the study. The research team made an initial project planning visit to Tallahassee (in DCF District 2) in December 2005 to observe the ACCESS Florida changes firsthand and to interview state officials.
This initial visit included trips to Customer Service Centers and community partners in Tallahassee and surrounding areas, and a visit to the DCF Customer Call Center in Jacksonville.
• Call Centers. We wanted to select districts that contained or were close to the Miami and SunCoast Customer Call Centers to ensure we could visit both of those call centers during our site visits.8 The Jacksonville call center was not included in this decision because, as noted above, the study team visited that call center as part of the initial planning for the study.
Figure I.2 shows the location of the 14 DCF districts. Below, we describe the districts we selected and the rationale for those selections.
Table I.2 compares the 14 DCF districts across key characteristics. We selected the following four districts for in-person staff interviews:
District 3. District 3 was selected for two reasons. First, compared with other districts, District 3 has the highest percentage of the population residing in rural areas (52.2 percent;
the next highest percentage is 38.8 percent in District 2). Second, District 3 has 4 of 11 counties with no Customer Service Center. Only District 2 (which was visited as part of the planning phase of the study) has more counties with no Customer Service Center. Like other districts with rural populations, District 3 has a relatively high number of community partners per 1,000 FSP households.
District 8. District 8 is similar to many districts in that it does not stand out in terms of