Where: n = sample size (to be increased by the standard 10% sampling overload factor to accommodate unforeseen situations such as worker transfers and terminations)
p = maximum anticipated rate of occurrence of the activities being observed (anticipated distribution of effort -- historically this value has been approximately 36% for Medicaid)
SE = desired sample precision (.02)
t = confidence level factor (1.96 for 95%)
Solving the equation yields:
1.96
.02
2
.36 (1-.36)
n =
= 2,213
Page 3
Increases (or decreases) in staff subject to the RMTS. SAMPLE SELECTION AND DISSEMINATION
The RMTS will be drawn from a listing of all active staff subject to the RMTS during the month preceding the sample quarter (i.e. the RMTS for the quarter beginning July 1 will be drawn from a May listing of active employees). The RMTS software will generate the random moments for the quarter from the population of the total number of eligible employees and total working minutes in each quarter. The moments generated are identified with a number (the observation number) and include the following information:
_ Day: Standard workday during the year, official holidays excluded _ Time: Any minute during the working day
_ ID#: Personnel Identification Number (PIN) _ Name: Employee name
_ Location: Location of Employee
A master list of the above information will be generated, as well as sample listings for each location. The location lists will be distributed along with observation forms, to the RMTS Coordinator at each location. In addition, preprinted labels will be included for each location. Each month a confidential envelope containing the labels will be mailed to the appropriate RMTS Coordinator. The RMTS Coordinator then attaches each label to an Observation Form and at the appropriate moment completes the form.
The RMTS Coordinator will insure that the observation is reflective of the actual sampled moment selected. The SSR will complete the observation form based on the instructions contained in this document. The RMTS Coordinator will validate the response based on direct observation of the sampled worker’s activities, questions asked of the worker or by assessing the interaction of the worker with a client or other person. Except for client or program specific travel, meetings or training where the worker is expected to be away from his/her work station for more than 1 hour, all observations will be accomplished within 15 minutes of the sampled moment. The RMTS Coordinator will make a photocopy of the form, retain the copy, and mail only the original form back to ESA.
STATISTICAL RELIABILITY
Preparation and Submission of Cost Allocation Plans,” dated November, 1981: "the precision (sometimes referred to as 'allowable error') is + 2% at the 95% confidence level for each activity whose frequency occurs 5 or more percent of the time.” For each activity whose frequency occurs less than 5 percent of the time, the precision can be up to + 5% at a 95% confidence level."
Computation of the standard error of proportion (SEi) resulting from a sample is important. This allows sample size modification within the quarter under review if the preliminary sample results are not within acceptable precision limits. Additionally, computation of the estimates (Pi) of worker time devoted to each program is also important. These values can
be derived using the following formulas:
n
P
Pi
=
andn
Pi
(1-Pi)
SEi
%
=±t
x 100
Where: PI = Proportion of responses for the
i
th programP = Actual rate of occurrence observed
n = Sample size (total number of worker-minutes responses, i.e., interviews) for the time frame for the population
Page 5
t = Confidence level factor (1.96 for 95%)
To compute the estimate of the proportion of response for the ith program use:
n
P
Pi =
To compute the standard error for the estimate of the proportion of responses for the ith program use:
n
Pi (1-Pi)
SEi = ±t
To illustrate this procedure, assume that the Food Stamp program had 360 observations (P) last quarter from a total of 2,434 samples (n), then:
2,434
360
P
i=
= 14.8%
SE
i=
2,434
±1.96
=
1.4%
Therefore, it is possible to say at the 95% confidence level that if the entire universe or population was observed, the actual rate of occurrence for the Food Stamp program would fall between 13.4% and 16.2% (14.8% + 1.4%). Further, the sample precision (SE) of 1.4% is within the statistical reliability desired of + 2% at the 95% confidence level.
OSERVATION FORM AND INSTRUCTIONS
Appendix A of this document contains a copy of the Observation Form that should be used. It identifies all of the programs and activities that the SSR may be engaged in. Appendix B contains the instructions that should be followed by the RMTS Coordinator and the sampled worker in completing the Observation Form. Each RMTS Coordinator and worker should be trained in the completion of the Observation Form and to the importance of providing accurate and timely responses. Appendix C provides a matrix of the possible responses that may be identified.
SAMPLE RETRIEVAL AND PROCESSING
The completed Observation Forms should be returned to ESA on a weekly basis. When ESA receives the completed Observation Forms, the forms are then reviewed to make certain that each interview was appropriately completed. If any discrepancies exist, the ESA Analyst and RMTS Coordinators will review the forms and resolve the problem. Corrections made to the responses are noted on the forms. If an oversight occurs and an interview is not properly completed, that interview is omitted from the study with the reason for omission documented.
At the end of each fiscal quarter the percentages of total responses are computed for each program listed in Section I of the Observation Form based on the results of the RMTS. Responses common to more than one program (Section I, Item 109) will be distributed back to the programs identified in Section I exclusive of item 109 on the ratio of participants by program identified in Section I. These ratios will be applied to the total costs (salaries/wages and related fringe benefits of the Units subject to the RMTS to distribute their costs to the specific programs). In addition, these ratios will be used to distribute related cost centers (clerical support, supervision, District Office related administrative costs, etc.) to the specific programs identified in Section I.
Page 7
Management is committed to the success of the RMTS approach.
Specifically, the ESA RMTS Analyst will:
• Generate the sample data and send a listing of the dates and times of the staff to be sampled to the appropriate RMTS Coordinator.
• Monitor ESA-wide RMTS activities and assist in problem resolution; and provide technical assistance, reference and resource material, and training when required.
• Insure that the sample universe is updated quarterly to include new hires and exclude terminations and transfers.
• Works with the Office of the CFO to ensure the necessary information is received for the preparation of reports.
• Analyze, edit, and summarize the sample results and prepare information for dissemination, for management purposes, as deemed necessary.
• Review and update the Cost Allocation Plan for Random Moment Time Sampling (RMTS), as necessary.