Under modernization, not only have the application processing procedures changed, but the amount of verification clients provide, the way that clients provide information, and the length of eligibility interviews also have changed. Historically, many of these procedures were implemented as a way to ensure payment accuracy. The fact that they have changed could mean that accuracy could be compromised for efficiency. In this section, we describe the ways ACCESS Florida might be expected to contribute to changes in payment errors, and then examine whether these expectations are supported or refuted by data and staff feedback.
We would expect that some changes under ACCESS Florida could increase payment errors, while others decrease payment errors. However, this study cannot determine whether the changes actually are having an impact on payment errors. Although administrative data shows an increase in payment errors after ACCESS Florida was implemented, confounding factors such as disaster benefits that followed severe hurricanes also occurred in the post-implementation period. With some exceptions, staff generally believed that ACCESS Florida’s new policies and procedures would, if anything, reduce payment errors.
Potential Effects of ACCESS Florida Changes
Payment errors can occur as a result of both client and DCF staff actions. Clients can provide misinformation, either intentionally or unintentionally, to DCF or DCF staff can make an error in collecting and recording information. In examining the ACCESS Florida changes, eight have implications for payment errors. Of these eight changes, if they have any effect at all, three could lead to both increased and decreased payment errors, and five could lead to increased payment errors (Table VI.7). There are no changes that we think could contribute to lower payment errors.
Table VI.7. Potential Effects of ACCESS Florida Changes on Payment Errors Factors That Could
Decrease Errors Factors That Could Increase and/or Decrease Errors Factors That Could Increase Errors Specialization of Functions Simplified Reporting Community Partners Interview Procedures Relaxed Verification Recertification Procedures Web Application/Streaming Agency Downsizing Organizational Restructuring
Specialization of Functions. The specialization of caseworker functions under ACCESS Florida could lead to both increased and decreased payment errors. One key implication of the specialization of functions is that no one staff member has ownership for a given case. While modernization includes a set of performance standards for each new position, and those performance standards are set in part to ensure payment accuracy, staff
113
VI: Potential Effects of Modernization
are well aware of what is measured for their position. It is possible that when staff encounter a problem that could lead to an error but it is technically “someone else’s responsibility,” they do little about it. In other words, the incentives under the specialized system are for staff to focus on a narrow range of issues, potentially ignoring other problems.
On the other hand, the specialization could reduce errors as staff become focused on and proficient in specific tasks. A key example of this would be the CMUs. Staff in the CMUs are dedicated to tracking data exchanges and alerts, many of which can affect payment accuracy. Under the caseworker model, each caseworker was responsible for tracking the data exchanges and alerts for their cases. According to many staff, this work was often made a lower priority to the other demands on the caseworker’s time. By creating a specific staff function dedicated to tracking data exchanges and alerts, DCF ensures the function is completed. Specialization could also mean that, with several staff checking the same case, there are more opportunities to identify and address case errors.
Community Partners. The use of community partners could be expected to lead to both increased and decreased payment errors. If staff at community partners are not well- trained on the FSP application requirements, they could give misinformation to clients applying for benefits, and this misinformation could lead to payment inaccuracies. On the other hand, community partners could provide special needs clients (such as the elderly, disabled, or English as a second language clients) with assistance that they might not get at Customer Service Centers. This assistance could lead these applications to be more accurate than they otherwise would be.
Agency Downsizing. The downsizing of DCF could lead to increased payment errors because of two possible effects on remaining staff. First, the remaining staff could feel less job secure and less allegiance to DCF. As a result, they may care less about case accuracy. Second, the remaining staff could be overburdened by the increased workload to the point where they are unable to give each case enough attention to catch all potential errors. (DCF also increased the staff allocated to front-end fraud prevention under ACCESS Florida, so interviewing staff have more fraud investigation resources at their disposal than they did previously.)
Policy and Procedural Changes
Interview Procedures. The interview procedures under modernization could lead to increased payment errors. For most applicants, interviews are extremely short and many are conducted over the phone. As a result, DCF staff have fewer opportunities to collect valuable information that can help the worker determine whether the client has provided complete and accurate information.
Verification Requirements. Like the new interview procedures, the new verification requirements could also lead to increased errors. Workers are able to accept client statements without verification on important information regarding some assets, income and expenses. If clients misreport this information – either intentionally or unintentionally – it could lead to payment errors that might have been caught under the old system.
114
VI: Potential Effects of Modernization
Simplified Reporting. Although not technically part of ACCESS Florida, simplified reporting can lead to both increased and decreased errors. Under simplified reporting, clients do not need to report most income changes unless their income exceeds 130 percent of poverty. Clients receive a notice explaining what that threshold is for their case. This new rule can reduce errors because many unreported changes in income which would have been an error before simplified reporting are not considered errors under simplified reporting. However, clients may not track their income well enough to know when it exceeds their threshold, or they may misunderstand the policy rules and think that all income changes do not need to be reported. To the extent that this occurs, simplified reporting could lead to increased errors.
Recertification. Recertification procedures under ACCESS Florida could lead to increased errors. New passive recertification procedures permit many clients to recertify for benefits without an interview. Without the extra information obtained through this process, workers may inaccurately determine an individual’s eligibility or benefit amount. While clients could misrepresent their situation to a worker whether interviewing over the phone or in person, the worker is less able to gather visual feedback during a telephone interview.
Technology
Web Application and Streaming. The web application could lead to increased errors because it contains more questions than the paper application and clients may be inclined to leave information incomplete because they do not know the answers or do not understand the questions. Client frustration with the application, unclear language on the application and less interaction with DCF staff to clarify these areas of confusion could all lead clients to leave fields blank when completing the application. If so, it is up to the intake worker and case processor to catch this incomplete information. The fact that application data are streamed into the FLORIDA system means that workers spend less time examining the case data. As a result, workers may be less likely to catch incomplete information and other errors.
Measures of Errors
Florida’s Quality Control (QC) error rate increased after ACCESS Florida was implemented, and while this may suggest that modernization increased errors, other factors may also be at play. 54 In the years preceding ACCESS Florida, annual trends in Florida’s
average monthly error rates followed a pattern similar to that of other large states (Figure VI.10). Error rates increased between 2001 and 2002, and then dropped substantially by FFY2004. After 2004, average monthly error rates in Florida increased from 5.5 percent to 8.4 percent, returning to 2003 levels, while error rates in other states remained at or below their 2004 levels.
54 The FNS QC audit of payment errors is estimated from a sample of cases each month. If the QC reviewer determines that the client is ineligible or should otherwise have a different benefit level, then the case is considered an error.
115
VI: Potential Effects of Modernization
Figure VI.10. Annual Trends in the Average Monthly Food Stamp Program QC Error Rate
Source: MPR tabulations of data from the USDA Food and Nutrition Service.
Note: Error rates reflect underpayments and overpayments to clients (including payments to ineligible individuals). Large states include: CA, IL, MI, NY, OH, PA, TX. Southeast Region includes: AL, GA, KY, MS, NC, SC, TN.
While the timing of this increase coincides with the implementation of modernization, the fact that Florida was affected by numerous hurricanes in 2004 and 2005 also may help explain the increase in error rates. The state ran large-scale disaster FSP benefit programs those years, disrupting regular operations. To free up staff time to respond to the crises, federal waivers were obtained to extend certification periods for non-disaster FSP clients whose recertifications were due during those crisis months. When these delayed recertifications ultimately came due several months later and were added to the recertifications scheduled in the same month, the recertification workload was substantial, yielding a rushed and potentially error-prone environment as staff attempted to accommodate a high volume of new clients just as DCF was adjusting to some new and still- unfamiliar administrative procedures.
The size of the increase in Florida’s error rate is within the normal range of typical fluctuations in error rates. Between FFY2001 and FFY2003, Florida’s monthly error rate averaged 9.3 percent (Table VI.8). During this period, error rates changed by an average of 2.8 percentage points from month to month. Such large monthly fluctuations are common in most states. Thus, while Florida’s average annual error rate increased by 2.9 percentage points between 2004 and 2006 (from 5.5 percent to 8.4 percent), this increase is about the same size as the typical monthly variation in error rates.
0.0 2.0 4.0 6.0 8.0 10.0 12.0
FY2001 FY2002 FY2003 FY2004 FY2005 FY2006
Error Rate
Florida Other large states Other Southeast states All other states 8.4
5.4 5.6 4.6
116
VI: Potential Effects of Modernization
Table VI.8. Variation in the Average Monthly Food Stamp Program QC Error Rate for Large States Before and After ACCESS Florida Implementation
FFY2001-FFY2003 FFY2004-FFY2006 Average Across FFY01-FFY03 Average Monthly Change (Standard Deviation) Average Across FFY04-FFY06 Average Monthly Change (Standard Deviation) Florida 9.3 2.8 7.0 2.8
Other Large States
California 12.4 5.4 5.8 2.2 Illinois 7.1 3.3 5.7 2.4 Michigan 11.8 3.4 7.1 2.2 New York 6.9 2.7 4.9 2.8 Ohio 6.9 2.8 7.6 2.6 Pennsylvania 8.5 2.5 3.7 1.5 Texas 4.0 1.7 5.1 2.3
Other Large States, Combined 8.2 4.3 5.7 2.6 Other Gulf Coast States
Alabama 8.6 3.4 4.7 3.0
Louisiana 5.7 2.4 5.6 3.2
Mississippi 3.9 1.7 3.4 2.4
Texas 4.0 1.7 5.1 2.3
Other Gulf Coast States, Combined 5.5 3.0 4.5 2.8
All Other States 7.2 4.0 5.5 3.2
Source: MPR tabulations of data from the USDA Food and Nutrition Service.
Note: Error rates reflect underpayments and overpayments to clients (including payments to ineligible individuals).
In short, there is not enough evidence to disentangle the effects of modernization from the effects of disasters and the typical fluctuations in error rates. If modernization increased program errors, we are not able to determine the magnitude of the impact. Moreover, we cannot determine whether any ACCESS Florida related increase in errors is systemic or a temporary effect reflecting DCF’s adjustment to the modernization model. Regardless, the change in Florida’s error rate after FFY2004 did not affect the state’s ranking among other large states. Among the eight largest states, Florida had the third highest error rate before modernization, and it continued to have the third highest error rate after modernization.
117
VI: Potential Effects of Modernization
Nor did Florida’s ranking among Gulf Coast states change; Florida’s error rate was higher than that of other Gulf coast states in both periods.
Staff Impressions
Some DCF staff believed that specialization can create errors. Many DCF staff thought that errors under modernization have increased in part due to a decline in accountability—too many individuals handle a single case, but no one person ultimately bears responsibility for it. Moreover, staff felt that the process itself, which focuses on efficiency, leads to more errors. For example, some staff believed that with the specialization of functions, intake specialists often do not have the time to ask some key probing questions that would yield better information for the processing specialists to determine eligibility. Other staff described that the problem extends beyond the separation of intake and processing to the call center as well. Several processing specialists complained that call agents sometimes take action on a case without documenting it. When other staff work on the case, they do not have the most updated information, potentially leading them to make inaccurate determinations.
While specialization may create errors, many staff felt CMUs help reduce error rates. DCF workers and CMU supervisors in several districts said that not only are CMU staff processing bill tracking and data exchanges and alerts faster, but some units specialize to the point that workers become experts in specific kinds of cases. They can focus on detailed policies and investigate pending cases and alerts more accurately. Consequently, staff believed that the CMUs serve as a counterbalance for other ACCESS Florida changes that may create errors. One district observed that the CMU notices and responds to case changes w hen the call center did not handle it properly. At the same time, some DCF staff cautioned that the CMU can get backlogged and, as a result, pay less attention to the low priority data exchanges and alerts. As such, the unit should not be viewed as a safety net to catch all errors.
DCF staff felt applications originating at community partners are not more likely to have errors. While is difficult to measure the error rates of applications from partners as compared with those from the Customer Service Centers, workers believed that applications submitted by clients using partners (who are trained on the web application) contain no more errors, on average, than those completed at other access points.
Many staff were concerned about the loss of information as a result of new interview procedures. In general, staff reported that the fact that most clients have only a short intake interview, combined with the fact that many clients have no in-person interview, might generate additional errors. Because the abbreviated interviews are tightly scripted, staff said that intake workers do not have the opportunity or the time to probe potential problems. Intake workers could make a note of potential problems that processors could then pursue through a follow-up call, but they could only do this if they obtained enough information to identify the potential problem. Moreover, staff expressed concern that conducting interviews over the phone make it easier for clients who want to provide misinformation intentionally. One worker gave the example of a client who claims no vehicular assets on the application. In an in-person interview, this worker would check to
118
VI: Potential Effects of Modernization
see if the client had a car key on their key chain, and if so, ask more probing questions. In a phone interview, the worker would never see the key chain or other indicators of potential error.
Staff said that the policy of relaxed verification is a positive change that saves time and is unlikely to trigger payment errors based on reported expenses. In general, staff had positive impressions of the relaxed verification policy, especially as it pertains to household expenses, such as utilities, because verifying those expenses under the caseworker model was cumbersome. Some staff felt that verification of expenses often does not affect eligibility determination, so that increased reliance on client statements means less time investigating most information that clients report with what they perceive to be minimal effect on eligibility determination. On the other hand, staff expressed some reservations about the use of relaxed verification for income-related questions, due to a concern about the error rate. Notably, staff may still request verification in cases where information appears questionable, and some staff reported doing so.
Concern about the adverse impact of passive recertification was common among interviewed staff. There was some staff concern that the practice of passive recertification may increase errors. They worried that the absence of direct contact between DCF and the client at the time of recertification also raises the possibility that a client might intentionally or unintentionally fail to report a household’s situation accurately, leading to errors.
Staff did not think web applications would lead to increased errors. While staff agreed that the web application could lead to an increase in incomplete applications, they did not believe that these applications would be more likely to result in a payment error. Rather, they said that they would need to do additional work to get the missing information from the applicant. However, some staff did feel that applicants who want to submit false information will have an easier time doing so with a computer. As one worker said, “it’s easier to lie to a computer than to a person.”
Summary
The impact of ACCESS Florida on payment errors is unclear. While Florida’s payment error rate increased after modernization, this could reflect the impact of hurricanes and typical error rate fluctuation as well as modernization. While most staff agreed that most ACCESS Florida changes are unlikely to increase payment errors, some had concerns over specific policy changes. Specifically, reduced interaction between workers and clients, due to both changed interviewing procedures and passive recertification practices, caused some staff concern about an increased potential for errors. The CMU was popular among staff, because of its focus on doing work that could reduce errors; work for which case managers had limited time under the previous system. An opinion that the web application and use of community partners would not affect payment errors was prevalent among interviewed staff. POTENTIAL EFFECTS ON PROCESSING EFFICIENCY AND ADMINISTRATIVE COSTS
A central motivation for the changes made under ACCESS Florida was to increase efficiency and reduce the administrative costs of the FSP and other programs. As a result,
119
VI: Potential Effects of Modernization
most of the changes have potential implications for efficiency and costs. In this section, we discuss how modernization could affect these factors, discuss trends in measures of efficiency and costs, and summarize how DCF staff feel the changes affected efficiency and costs. In examining the effects on efficiency, we consider a change to lead to an increase in efficiency if the change could result in fewer total DCF resources being used to process a typical case. In examining the effects on costs, we consider a change to decrease costs if it