This final round of review is intended to ensure that all previous requested changes are incorporated properly and that no errors were introduced in the process. Also in this round,

SBAC RFP #7 Item Authoring and Item Pool Application

2-23

further define the format and content specifications for each section of the guide. The User Guide will go through our internal editorial review process prior to being submitted to SBAC.

Consistent with established review procedures, we will build three rounds of SBAC review into the schedule for this task, assuming that SBAC will want to review the draft document.

Round 1: This round of review is intended to finalize the basic structure of the document and verify that the overall content of the document is correct and consistent with SBAC

requirements.

Round 2: This round of review is intended to ensure that all edits from Round 1 are

incorporated properly, and to confirm that all information and instructions provided are accurate.

Round 3: This final round of review is intended to ensure that all previous requested changes are incorporated properly and that no errors were introduced in the process. Also in this round, final cross references will be made throughout the document, including table of contents, header titles, general information, and page references.

To ensure accuracy in the process, all SBAC edits for each round of review should be

consolidated and provided to Pacific Metrics in a single electronic file. The User Guide will be posted to the SBAC website for easy access to all users.

Training Guides

In addition to the User Guide, Pacific Metrics recommends the development of targeted Training Guides to address specific user roles or tasks in the application. These Training Guides move a step beyond the User Guide in that they provide both the physical processes the application offers, as well as best practices that ensure the most efficient and effective methods are used to accomplish tasks. Examples of targeted Training Guides Pacific Metrics has created are listed below:

• Item Creation and Submission

• Item Review and Submission

• Form Creation

• Reports and Batch Data Upload

• Art Creation and Submission

SBAC RFP #7 Item Authoring and Item Pool Application

2-24

Figure 2-3: Sample from Training Guide

Self-Learning Tutorial Software

Pacific Metrics is experienced in creating self-learning tutorials and using Camtasia® Studio software. Pacific Metrics will design a tutorial module for each of the different users – for example, item writer, artist, administrator, etc. – accessible through their menu page. The tutorial will provide a step-by-step walk-through of each person’s responsibilities and how that person can address these responsibilities through the system. A voiceover explanation of each process step will also be included.

A detailed plan will be provided to SBAC for review and approval prior to script creation and production approval. The plan will include a timeline and a description of activities that will be conducted at each stage of the process. As part of this plan, four reviews will be included; an initial review of the script, finalization of the script, and two online reviews. The script will go through a copy edit prior to the first submission. The name and resume of the personnel responsible for developing the training modules will be included in the plan.

SBAC RFP #7 Item Authoring and Item Pool Application

2-25

3. O NGOING A PPLICATION H OSTING AND M AINTENANCE 3.A Application Maintenance and Support

As defined in the technical work plan, we will plan an initial software release of the Item Authoring and Pool application in August of 2012. At this point, the application will be

considered in production and will require ongoing support. Support of the application will include the following activities, and will be provided for the remainder of the contract period, August, 2012 – September, 2014:

1. Help Desk and production application support

2. Production fixes for issues (“bugs”) identified with completed features 3. Release of application enhancements after the initial software release

4. Monitoring and system maintenance of the application within the hosting environment 5. Document updates related to production fixes and additional features.

These areas are described in greater detail below.

Help Desk and Production Application Support

Pacific Metrics recognizes the need to support our existing technology products in a production environment and employs a well-defined process for escalating and managing technical issues identified on its production systems. The process defines three levels of escalation for technical issues such that the issues are resolved in a timely fashion by the appropriate staff. Pacific Metrics will provide all three levels of Help Desk support.

The first level typically consists of the customer service and help desk function that responds to general issues. We anticipate that many of the SBAC Level 1 questions will relate to

user-questions for SBAC’s IAIP. We will ensure there is trained Level 1 staff that is able to answer these questions.

Technical issues that cannot be resolved by the Level 1 Application help desk are escalated to Level 2, which consists of experienced product support engineers at Pacific Metrics. These engineers understand all functional and non-functional aspects of the system and can resolve more complex issues related to use of the system. Technical issues that cannot be resolved by Level 2 are escalated to Level 3, which consist of software engineers responsible for system development and maintenance. Level 3 issues are typically ones that require a system modification to resolve. Response times and resolution times are defined for each level of escalation, and these times depend on both the priority and severity of the issue being

managed. As defined in greater detail below, Pacific Metrics’ production support process allows for the quick deployment of system fixes outside of a regularly scheduled product release for those technical issues that require one.

Production Fixes

A complex online application normally requires production fixes for issues identified with the application. The purpose of a production fix is to correct issues identified with existing features of the application, to ensure the application functions as designed and at an expected level of performance. A production fix is not meant to add additional features to the application, which would rather be managed through a full release of the application software. Pacific Metrics typically releases production fixes in a patch release of the application; a software release that is incremental in nature and is limited to production fixes. Pacific Metrics will support production

SBAC RFP #7 Item Authoring and Item Pool Application

2-26

fixes through a patch release process for the Item Authoring and Pool Application after the first production release in August of 2012. Fixes will be limited to application issues as defined above.

The patch release process will prioritize application issues based on both application severity and customer priority. These attributes will define the timeframe surrounding when an issue is resolved and delivered in a patch release. Pacific metrics will work with SBAC to define the periodicity surrounding patch releases, associated with different priority/severity levels. For instance, an issue that impacts most users in a significant fashion, like an application outage, would likely require a patch release as soon as possible.

Release of Application Enhancements

Pacific Metrics will us a formal software release process for releasing full versions of the Item Authoring and Pool Application to the production environment. The full software release process will be employed for both the initial software release in August 2012 and in any subsequent releases that are required. A full software release contains significant features and functionality enhancements over a prior version of the software application. The release process is strongly tied to the application development process, and hence includes all components of that process such as analysis, design, implementation, validation, user acceptance testing, etc. The work plan outlines the anticipated releases that will be needed for the customized SBAC IAIP application. Pacific metrics will work with SBAC to ensure that the full software releases as planned integrate into larger activities related to the Item Authoring and Pool Application.

The type of enhancements we expect to be released after the initial release in August of 2012 are enhancements planned in original scope of work but not implemented in the initial release. If SBAC identifies additional application enhancements that are required as part of this effort but not covered by the original scope of work, the defined change control process will be used to scope, cost and manage these enhancements.

Monitoring and Application Maintenance within the Hosting Environment

Pacific Metrics will provide regular monitoring and application maintenance as part of hosting the Item Authoring and Pool Application. We describe each function in greater detail below.

Application and System Monitoring: All applications and hardware components within Pacific Metrics’ hosting environment are monitored using SNMP manager probes and all software systems are monitored by additional health checking capabilities that are specific to the application. All alert types have associated distribution lists and notification occurs across multiple channels including but not limited to: 1) email, 2) SMS, 3) monitoring management consoles. The monitoring system has components that are located both within and outside of the production environment, providing a comprehensive capability to detect system and

application issues. The system also contains capabilities to generate and escalate system alerts based on predefined rules and monitoring triggers.

Pacific Metrics has production staff that is responsible for observing and managing the data and alerts produced by the monitoring system. This staff in responsible for initiating required

maintenance and/or escalation procedures related to the reported data and alerts. Escalation is the responsibility of IT staff who generally contact customer service representatives that act as liaisons to the customers that are potentially affected. Customers are kept apprised of the situation until it has been resolved. Subsequent to resolution, the customer is provided with a detailed post mortem of what caused the problem and how it was resolved.

SBAC RFP #7 Item Authoring and Item Pool Application

2-27

Application Maintenance: Pacific Metrics production staff is responsible for all regular maintenance related to its production applications. This includes but is not limited to the following activities:

1. Regular maintenance and servicing of all production hardware components including servers, switches, firewalls, load balancers, etc.

2. Regular management and patching of production systems such as servers, operating systems, virtualization environments, network topology, monitoring and reporting systems.

3. Provisioning and installation of additional IT infrastructure to support increased application capacity.

4. Remediation and correction of production system failures.

5. Security management and monitoring.

As described in the next section, this proposal assumes Pacific Metrics will be hosting the Item Authoring and Pool Application.