Information Technology
QA Test Plan for MCESA REIL
Track A Proof of Concept Project
Author: Miruta Garg
Position Title: Quality Assurance
Department/Agency: Information Technology/Arizona Department of Education
Date Created: June 14, 2012
Document History
Author Tracked Changes Version
Miruta Garg Initial Draft 0.1
David Barcai Edits 0.2
Gary Kerekes Edits 0.3
Miruta Garg Edits 0.4
Miruta Garg Edits Section 1.4.1 and Section 1.4.2 0.5
Marina Stover
Editorial Review
Section 2.5.6: Expanded process of MCESA defect reporting.
Section 4: Added Al Dullum as a required approver
0.6
Miruta Garg Edits – Index (Updated the page numbers), Added
Section 3.1, Section 3.2, Section 3.3 and Section 3.4 0.7 Al Dullum
Edits 0.8
David Barcai
Versioned 1.0
Miruta Garg Included changes suggested by Al Section 3.1 and Section 3.4
1.1 Marina Stover Editorial Review
Section 3.5: Added per Miruta Garg 1.2
Marina Stover Removed Draft references.
Distributed to MCESA for review and signature. 1.3 Marina Stover Section 1.2: added “or unshadowed” per Al Dullum. 1.4
T
ABLE OFC
ONTENTS1.
Overview ... 2
1.1. Purpose ... 2 1.2. Project Overview ... 2 1.3. Target Locations ... 2 1.4. Scope ... 2 1.4.2. In Scope (ADE) ... 2 1.4.4. In Scope (MCESA) ... 3 1.5. Browsers ... 3 1.6. Outstanding Issues ... 31.7. Risks and Contingencies ... 3
2.
Development Process and Testing Types ... 4
2.1. Test Approach ... 4 2.2. Test Automation ... 4 2.3. Test Data ... 4 2.4. Requirements Validation ... 4 2.5. Control Procedures ... 4 2.5.1. Build Deployments ... 4 2.5.2. Test Execution ... 4
2.5.3. Defect Management Process ... 5
2.5.4. Defect Remediation ... 5
2.5.5. Defect Tools ... 5
2.5.6. Defect Submission Process ... 5
2.5.7. Required Defect Documentation ... 7
2.5.8. Problem Resolution and Escalation ... 7
2.6. Test Tools ... 7
3.
Testing Types ... 8
1. Overview
1.1.
Purpose
The purpose of this Test Plan is to ensure that all required test planning, test
preparation, and test execution activities are addressed by ADE QA and MCESA. The Test Plan describes the test approach, testing activities, assigned responsibilities, schedule of testing tasks, and the resources required to determine the production-readiness of the software product deliverable(s).
The Test Plan applies to all levels and types of testing planned throughout the system development life cycle, and should be used as a communication vehicle for the entire project team, including Project Management, Development teams, and business stakeholders, to gain an overview of how the system will be tested.
1.2.
Project Overview
The purpose of the Track A Proof of Concept project is to provide a working model of the data integration, Roster validation and display of Teacher Observation and Student Assessment (AIMS) data for a specific grade level for two subjects to ensure the concept, design and tools identified will meet the project needs before the next school year begins. The proof of concept will be conducted either via a WebEx by MCESA personnel to the U.S. DOE or access may be provided to U.S. DOE personnel while shadowed or unshadowed by MCESA personnel.
1.3.
Target Locations
This section describes the areas affected by the Track A Proof of Concept Test Plan: 1.3.1. The host website which contains the link to BFK- (www.mcesa.schoolwires.net) 1.3.2 The ODCT tool R1V1
1.3.3. The roster link to the BFK application from the REIL Performance Scorecard 1.3.4. The BFK-Link application
1.4.
Scope
1.4.1. This section describes what is in scope for the Track A POC Test Plan.
1.4.2. In Scope (ADE)
1.4.3. This section describes what is in scope for ADE QA testing:
1.4.1.2. Full regression testing of the ODCT
1.4.1.3. Verify that the roster link on the REIL Performance Scorecard screen takes the user to the BFK-Link application in another window.
1.4.1.4. Verify that the BASIS-driven scores in the REIL Performance Scorecard are updated.
1.4.1.5. Verify that the View Observations link on the REIL Performance
Scorecard screen takes the Evaluatee user to a view of the Observation Cycle Report, and takes the Evaluator user to the ODCT Teacher Selection screen.
1.4.4. In Scope (MCESA)
This section describes what is in scope for MCESA testing: 1.4.4.1. Full regression testing of the ODCT
1.4.4.2. Verifying the scores displayed in the ODCT 1.4.4.3. Testing the User Roles in the BFK-Link application
1.4.4.4. Testing the Organization, Teacher, Student, and Principal relationship in the BFK-Link application
1.4.4.5. Testing the functionality of BFK-Link application
1.5.
Browsers
1.5.1. The supported browsers for Track A Proof of Concept testing are the most current MS Internet Explorer and Safari browsers.
1.6.
Outstanding Issues
Currently Not Applicable for this project.
1.7.
Risks and Contingencies
1.7.1. Risk: Roster files are not uploaded to the BFK-Link application in the format required 1.7.2. Risk: Internal access testing limits
1.7.3. Risk: Viability and validation of external access
2. Development Process and Testing Types
2.1.
Test Approach
The types of tests which will be required are: 1.1.1. Regression Testing of the ODCT 1.1.2. Smoke Testing of the Test environment
1.1.3. Functional Testing of the Login feature at BFK-Link 1.1.4. Functional testing of the BFK-Link
2.2.
Test Automation
No Automation testing will be done for the Track A Proof of Concept project
2.3.
Test Data
2.3.1. The REIL Performance Scorecard data source is ‘mcesapoc’ database located on testdata40 server.
2.3.2. The BFK-Link data source is located on the trainingdata40 server: ‘link’ and ‘link_datastage’ databases (current installation).
2.4.
Requirements Validation
Test cases will be written and executed through Microsoft Test Manager. Test results will reside in Micorsoft Test Manager. If a defect is found during testing, then it will be reported in Team Foundation Server (TFS). All test cases will be mapped to the requirements stated in the Track A Project POC Project Definition Document.
2.5.
Control Procedures
Test cases and user stories will be the control procedures that will be tracked and documented.
2.5.1. Build Deployments
The Track A Proof of Concept project will be deployed to the Test environment. The project will not go into the Production environment.
2.5.2. Test Execution
Once the data is moved to the Test environment and is ready to be tested, ADE QA and MCESA will start testing the required items specified in this document.
2.5.3. Defect Management Process
The defect tracking and resolution process will be done in TFS. The defects reported by ADE QA and MCESA will be addressed depending upon the priority and severity of the issue.
2.5.4. Defect Remediation
The process by which defects are found, logged, fixed, and then merged back into circulation is a vital aspect of the Application Lifecycle. The following sections define the Defect Remediation Process that is used at ADE.
2.5.5. Defect Tools
Defect tools are the specific tools that are used either in the finding of a defect, the logging of the defect, or the remediation of a defect.
Visual Studio Test Manager 2010
Visual Studio Test Manager 2010 is a repository backed by TFS for all test assets including test cases.
Visual Studio Test Manager 2010 provides the following support:
Provides visibility into requirements coverage
Provides visibility into the progress of the testing cycle
Provides visibility to make effective decision on go-live based on known risks
Measures progress and effectiveness of quality activities
Collaborates in the software quality lifecycle with a single global platform
Manages manual and automated testing assets centrally
Facilitates standardized testing and quality processes
2.5.6. Defect Submission Process
During the development of a feature or sprint, defects will be identified. Some of the defects may be code related, others may be business case related. The way in which these defects are acknowledged and proposed for action by the team is as follows.
Defect Identified
Defects can be identified as a part of development, or as a part of either informal or formal QA testing. Once the defect has been identified for the current development effort, the process to get that information to the team for remediation differs between ADE QA and MCESA:
ADE QA Defect Submission Process:
1. ADE QA will log defect directly into TFS as a “bug” task type under the current project and assign to the Technical Lead
MCESA Defect Submission Process:
1. Fill out a Support Information Request Form (to be provided to MCESA before testing) with all requested information. Include screenshots of defect if possible.
2. Submit form to the MCESA Support at
[email protected] with the subject line: PoC Track A Defect for ADE QA Personnel
3. ADE QA will log the defect into TFS and as a “bug” task type under the current project and assign to the Technical Lead.
Defect Triaged
Once a defect has been logged within the defect tracking system, it is then triaged and assigned. This process is in part a simple re-creation of the error and also a research effort. It is important for non-functional errors to be researched to ensure that what is being requested is actually a part of the current application functionality.
In other words, a defect cannot state that a particular feature X is missing. That is not a defect of the currently operating application, but rather a new feature request.
Defect Approved or Rejected
After the triage process is complete and the defect has been verified as part of the applications current feature set and is a reproducible event, the defect is approved as a Bug and assigned to the development team for remediation.
Defect Task Created
A new Bug task type is created within TFS. This should be created within the project for which the source application is built from.
Defect Task Assigned
The new defect/bug task is assigned to the remediation team as a part of normal development policy (i.e., as an item of the current sprints backlog for an AGILE team or as another task for the iteration for a waterfall team)
2.5.7. Required Defect Documentation
In order for a team to successfully manage a defect task and address the problem, they will need information about the error. The following information should be provided by the ADE QA or MCESA team and attached to the TFS Defect/Bug task type as either meta data entered directly into the task, or as attachments to the task.
Data Description
Time and Date The time of day of the defect. As close to the second as is possible to get [yyyy-mm-dd hh:mm:ss]
Source Application The application that is generating the error Inputs to Service Any data that used as an input to the feature
that is throwing the error
Outputs of Service Any expected outputs of the feature, including error test (if any)
Expected Behavior The normal happy path of the feature that is the source of the defect
Any trace logs (application, server, DB, etc.) that time-box the event
These should be as complete as possible – server logs, application logs, database logs, event logs
Who logged the initial defect
The name of the person who initially reported the defect (not the QA resource logging the defect)
2.5.8. Problem Resolution and Escalation
If a problem/issue has not been resolved within the allocated QA period for the task- the escalation of the problem/issue should be done to ensure that the project manager and quality assurance manager are fully briefed on the problem and steps to resolve to date.
2.6.
Test Tools
Test Tool Name
and Version # Description and Usage Location/Link
JMeter Performance Testing
3. Testing Types
3.1. Secured Login Access to Scorecard
3.2. ODCT Regression Testing
T e s t i n g t h e L i n B
Description / Scope Validating the login access to scorecard Test Environment testweb40.az.gov
Test Schedule 7/16/12
Participants ADE QA
Type and Source of Test
Data Test data source is “mcesapoc” on testdata40.azed.gov
Requirements / Test Scenarios / Test Cases / Test Scripts
MCESA REILize Decision Support System Project Definition Document
Track A Project
Description / Scope Validating the basic functionalities work as required Test Environment testweb40.az.gov
Test Schedule 7/10/12 – 7/13/12 ODCT Regression Testing 7/16/12 – 7/23/12 QA Test Case Execution
Participants ADE and MCESA QA
Type and Source of Test
Data Test data source is “mcesapoc” on testdata40.azed.gov
Requirements / Test Scenarios / Test Cases / Test Scripts
MCESA REILize Decision Support System Project Definition Document
3.3. Roster link to BFK on the REIL Performance Scorecard
3.4. V
Verify that the REIL Performance Scorecard is getting updated from BASIS
Description / Scope Validating the link at scorecard takes user to the BFK-Link application
Test Environment Testweb40.az.gov Test Schedule 6/21/12- 6/22/12
Participants ADE QA
Type and Source of Test Data
Trainingdata40 Roster data will be uploaded to the BFK-Link application by the customer once the external URL is opened
Description / Scope Validating the scores at REIL Performance scorecard getting updated
Test Environment Testweb40.az.gov
Test Schedule 7/21/12
Participants ADE QA
Type and Source of Test Data
3.5. Verify that the View Observations link on the Scorecard screen takes the Evaluate user to a view of the Observation Cycle Report, and takes the Evaluator user to the ODCT Teacher Selection screen
Description / Scope
Verify that the View Observations link on the Scorecard screen takes the Evaluate user to a view of the
Observation Cycle Report, and takes the Evaluator user to the ODCT Teacher Selection screen
Test Environment Testweb40.az.gov
Test Schedule 7/21/12
Participants ADE QA
Type and Source of Test Data
4. Test Plan Review & Approval
Project Number/Name: Track A Project- Proof of Concept Test Plan Deliverable Name: Track A – Proof of Concept
System/Application Name: ODCT and BFK Link
Date: June 14th 2012
Instructions:
1. Please review the attached Test Plan, annotating with corrections, suggestions and recommendations as necessary.
2. If you are providing a hardcopy approval, print these pages of the document and complete and sign the Review/Approval Results section on the last page. The shaded text boxes will not print.
If you are providing your approval electronically (via email), check the appropriate box in the
Review/Approval Results section and type your name within the text box on the Signature Line. (To mark an “X” in a check box electronically: (a) Place the cursor over the desired check box; (b) Double-click the left mouse button; (c) Choose “Checked” under “Default Value” in the “Check Box Form Field Options” window; (d) Click “OK”.)
3. Return or email this page form to the person named below with your summary of corrections, suggestions and critical recommendations. You will be notified of any critical recommendations that will NOT be incorporated and the rationale.
Test Plan Review & Approval
The names, departments, and phone numbers of individuals who will be approving the Test Plan
Name Department Phone #
David Barcai ADE MCESA REIL Project
Manager 602-542-5819
Gary Kerekes ADE Lead QA Analyst 602-542-5295
Christa Thompson ADE Sr. Program Manager 602-542-3028
Al Dullum REIL Data Management
System Project Director 602-826-5638
Review/Approval Results
Approved without revisions Approved with revisions Not approved Signature Date Signature Date Signature Date Signature Date
