Develop Test and Acceptance Plan

The agency will participate in testing the RMS as it is being developed, in the case of a customized system, or as it is being configured and tailored, in the case of a COTS system. Observing and/or participating in testing allows the agency to detect problems and concerns as early in the development process as possible rather than waiting for the final acceptance test. Seeing the system during testing provides the opportunity to make minor tweaks and gives insight as to how the system will actually operate.

For a small agency, there is no shortcut when transitioning from a manual filing system. The files will need to be clerically entered into the new RMS.

Vendors conduct different kinds of tests, including module testing, integration testing, and final system testing. Usually they conduct their testing at the vendor’s plant before the system is delivered to the agency. However, if the software operates on an agency’s desktop computers, verification of early versions of the software might be possible during development. Acceptance testing is conducted at the agency, which ensures that the system truly meets the agency’s needs in its own environment.

5.9.1 Integration Testing of Tailored RMS

Basic RMS software is almost always tailored by the vendor to meet each agency’s unique requirements. For example, the agency name is placed on screen banners, form fields are in preferred locations, reports are specifically formatted, etc. Agencies may want to observe integration testing and final system testing at the vendor’s plant. During the testing, agencies should verify the system design in these areas:

û Modules. If an agency is purchasing more than one module from the vendor (e.g., Case Management Module plus basic RMS), integration testing of these modules should be observed to ensure that they interact properly and their interactions are fully understood.

û System Interfaces. If interfaces to other systems are specified, agencies should observe how the interface software is being tested. Has the vendor set up files or other networked computers to simulate the other systems to which the RMS must interface? Do the interfaces conform to the ICDs? û User Interfaces. The vendor will tailor the user interface for each

agency. This custom interface should be observed and tested before RMS delivery in case adjustments to the system or procedures are necessary. For example, agencies can verify that officers are recording incident information in the format expected by the RMS.

û Hardware Integration. If agencies are acquiring the processor separately from the RMS software (or if an existing processing suite is being used), it is crucial to ensure that the new RMS operates effectively on the new hardware. In particular, agencies must test response time under peak conditions to ensure that the processing platform is adequate before live operation begins. This testing should use desktop and mobile data computers identical to the ones that the agency will use during live operation.

û Communications Integration. Integration testing should include testing

of communications links identical to those that the agency will use during production operations. For example, remote access connections to the agency’s local area network should be tested.

û Other Tailoring. All other tailoring should be verified before acceptance

74 75 û NIBRS Reporting. Agencies should ask for a specific demonstration

of the RMS’s NIBRS reporting capabilities during one of the early tests, and make sure that all aspects of NIBRS reporting are present and meet mandatory requirements.

5.9.2 Testing Customized RMS Modules

If the RMS is customized, the agency must be an active participant in all testing, including the customized module testing, to ensure thoroughness and proper functioning of the customized features. Additionally, agencies should require the vendor to conduct a walk-through of the customized software. Experienced software staff are needed to conduct the walk-through.

5.9.3 Acceptance Testing

Acceptance testing is conducted at the agency after the system has been installed and thoroughly checked out by the RMS vendor. This test ensures that the system meets all requirements and is acceptable for operation. During the test, the agency runs the system through its full set of functions and capabilities under peak load conditions. The test is extremely important. Acceptance of the system has strong contractual implications for the agency. Depending on how the contract is written, there may be little recourse if the agency finds problems after acceptance testing is successfully completed. Some agencies recommend withholding final payment until live production data is successfully submitted.

5.9.4 Test Plan

Agencies should develop a test plan with the RMS vendor, making sure the agency’s priorities are a part of the plan. This means that agencies should have a strategy in mind before sitting down with the vendor. The test plan defines all tests to be performed and the agency’s role in each one. For example, agency staff may act as observers on some tests and as active participants on others. Agency staff can be active participants by preparing test data or by using one of the desktop computers during the test. The plan also defines when each test is conducted and the specifics of the test. For example, the agency should determine if system interface testing includes a simulation of the system with which the RMS will interface. Specifics such as who will develop test scripts should be included.

5.9.5 Test Preparation

The agency should prepare carefully for the tests and for the participation of staff in testing. Preparation for the acceptance testing requires the most time. It is important for the agency to take as much time and invest as much effort as needed

to ensure that the system is operational and meets the agency’s requirements. Test scripts must describe the step-by- step procedure to be followed during the testing. Also, test data should cover the full range of transactions and conditions to be supported by the RMS. For

example, an agency could develop a stack of Incident Reports to be entered into the system by officers and/or clerks. These reports should have deliberate errors in them to verify that the RMS catches the errors and interacts properly with the person entering the information to correct the mistakes.

If agencies participate in integration testing at the vendor’s plant, some of the materials developed for those tests may be useful for acceptance testing as well. 5.10 Develop Training Plan

Training and retraining of an agency’s staff in the use, maintenance, and operations of the new NIBRS-compliant RMS are vital to the success of the program.

5.10.1 Training Staff

All agency personnel who will interact with the new RMS and be involved at any level with NIBRS reporting must be trained on the system and/or on NIBRS. This extends from patrol officers to the police Chief, who may receive periodic reports generated by the system. It may also include personnel from other agencies that have access to system files. For example, if an agency provides processing support to another agency, their personnel must be trained.

5.10.2 The Training Plan

The training plan describes the level and extent of necessary training. Training is broader than the RMS because it is based on the new business processes defined for the new RMS and NIBRS reporting (see section 5.10.2). It encompasses new policies and procedures as well as new forms. Not all personnel should receive the same training. For example, patrol officers will receive training on the new form and procedure for recording information for the NIBRS-compliant RMS, but personnel responsible for NIBRS reporting will receive in-depth training on report preparation and quality control.

For a small agency, the software will probably be the vendor’s standard product line with some minor tailoring. Smaller agencies may not need to visit the vendor’s plant as long as the vendor agrees to correct all tailoring mistakes before the RMS is accepted.

Agencies should plan on running an acceptance test; not only will this provide assurance of the product’s correctness, it will also give agency staff an excellent exposure to the RMS operation.

76 77 RMS vendors provide a certain amount of

basic training on the RMS itself. These sessions are usually viewed as “train- the-trainer” sessions. Agencies must supplement these sessions with training on the revised policies, procedures, and forms. Agencies also must identify who their trainers will be. Depending on the agency’s business processes and the selected RMS, training may be available from reputable third-party contractors. 5.11 Monitor Vendor Performance

The selected vendor is responsible for providing a system that will be the key to an agency’s successful operations for years to come. Agencies should make sure that the vendor is performing as promised—agencies should not simply assume that “everything is okay.” RMS vendors are like other software businesses; sometimes they over-promise or over-book and miss delivery dates, and sometimes their employees do sloppy and/or incomplete work, and some go bankrupt. Discussions with other agencies can help determine RMS vendors that have presented problems in the past.

Participating in testing is one of the most effective means for monitoring vendor activity. In addition, agencies should ensure that deliverables such as test plans, training plans, ICDs, database schema, etc., are specified. Agencies should make sure that the deliverables are received on time, that they are quality products, and that they are specific to the agency. The vendor’s financial condition should be fully investigated during source selection. The contracting office should have an ironclad contract to help ensure that the agency gets what it wants (see section 6). It is appropriate to include penalty clauses.

Finally, agencies should conduct periodic reviews with the vendor to talk over their progress and discuss any problems they may be having. If the vendors are having problems, agencies can then work with them to help solve the problem to everyone’s satisfaction. Both the agency and the vendor share the goal of successful NIBRS-RMS installation.

For a small agency, the project manager will want to supplement the RMS vendor training with materials on NIBRS as well as new policies, procedures, and forms. There should be at least three people who are fully aware of how to use the system to help cover in cases where one or two key personnel may not be available.

For a small agency, the state UCR program may provide a list of vendors that have been proven to be NIBRS- compliant and meet the state’s reporting needs. Make sure those vendors are invited to bid on the RMS. Track vendor progress regularly.