9.1 General
1. The Bidder is required to perform tests for the system as specified in this Section. These tests form part of the acceptance requirements by which the Bidder shall demonstrate to the satisfaction of TELCO PROV CO that the system as installed fulfils the specified requirements of TELCO PROV CO
2. As part of the project, the Bidder shall submit a certification and acceptance test (CAT) plan and execute this plan in the presence of TELCO PROV CO staff. TELCO PROV CO must approve the CAT test plan
9.2 Types of Tests to be Included in the CAT Test Plan
9.2.1.1 The Bidder shall include system tests to verify that all requirements of the relevant SMS specification have been met. These tests shall be designed to cover the total system including the operational software.
9.2.1.2 The acceptance test cases should include but not limited to the following:
a) Automated heartbeat/monitor for any IPC/RPC communications.
b) Availability of visual and audible alarms on the various hardware platform(s).
c) Availability of visual and audible alarms at a remote site, e.g. 24-hour Network Control Center. Connectivity to NMS via SNMP/HP-OVO traps, events must be present
d) Backup and recovery of the system. e) Shutdown of the system.
f) Warm/Cold/Reload of the system from root disk or tape.
g) Verification of the redundancy functionality.
h) Verification of hot-swapping and load-sharing functionality.
i) Verification of interconnection to external nodes. j) Verification of end-user interfaces.
k) Verification of database configuration and environmental variables.
l) Verification of operating system setup, including the automatic purging of audit and error log entries. m) Verification of security and access controls.
n) Breakage and resumption (synching) of disk mirroring, etc.
o) Elimination of database overflow pages, fragmentation, etc.
p) Freeing of system, database resources (e.g. semaphores, monitors, flags) upon application shutdown. (e.g. ipcrm)
q) Software patching process.
r) Verification on the accuracy of statistics, performance measurements and other counters, pegs generated on the system.
s) Automation of all O&M tasks. This can be achieved either via cron-Unix or the Schedule Service-Win NT/2000.
9.2.1.3 TELCO PROV CO may include additional test items to the acceptance test plans submitted by the Bidder. The Bidder should assist TELCO PROV CO to produce test procedures and test plans for those additional items submitted by TELCO PROV CO.
9.2.1.4 As part of the system test on site, a combined stability and subscriber test (including testing of software) shall be performed at the conclusion of all other items of site acceptance tests. All the system adjustments (including software) shall be made prior to the start of the test and no further adjustments will be allowed for the duration of the stability test. During the stability test, the system shall be subjected to normal operation and traffic capacity. All malfunctions, errors and failures shall be recorded. The length of time for this test shall be a minimum of 21 days continuously.
9.2.1.5 The system shall satisfactorily operate continuously without
a) adjustment, changes or part replacement of equipment,
b) changes to computer software
c) interruptions, errors or malfunctions of equipment d) Computer crashes due to software or operational
inputs.
9.2.1.6 If any unsatisfactory result is obtained within this 21-day period, the system shall be adjusted properly and accordingly and the test shall be re-started from then for a further 21-day period.
9.2.2 Maintainability Test
9.2.2.1 Tests shall be designed to prove the suitability of the diagnostics and applicability of all instructions in the technical manual provided by The Bidder. In particular, tests shall be designed to demonstrate the capability of failure isolation, repair and confidence checking to assure that the malfunction has been corrected and that the unit is available for operational use.
9.2.2.2 The tests shall also be designed to demonstrate the practicability and suitability of the preventive operation maintenance aspect of the system
9.2.3 Test Plans and Procedures
9.2.3.1 The test plans and procedures shall include, at a minimum, the following:
a) Description and objective of the test b) Scope of tests
c) Expected or desired results
d) Criteria for successful acceptance
e) Designation of all inputs that are required to test each function
f) Test output records including a description of required outputs, the types of equipment used to observe or measure the outputs, etc. As far as practicable, the input/output shall be expressed in measurable terms and the acceptable tolerances for each measurement shall be provided.
9.3 Facilities and Set-Up Requirements
9.3.1 Test Facilities
9.3.1.1 The Bidder shall provide all test equipment, tools, services and all other facilities required for the preparation, conducting and recording of all tests at its expense.
9.3.1.2 The Bidder should propose to supply a test system that allows the testing of patches and new software loads so as to minimize service disruption to the live system.
9.3.1.3 The Bidder should propose a list of tools and test equipment (hardware and software) for TELCO PROV CO to acquire for daily O&M activities
9.3.1.4 The Bidder should highlight which tools and test equipment are part of the offer and which are optional 9.3.1.5 Where applicable, the Bidder shall propose discounts for
TELCO PROV CO to purchase the tools and test equipment.
9.3.1.6 The Bidder should highlight the possible non-intrusive points on the LAN/WAN interfaces for TELCO PROV CO to probe for IP traffic (SMTP, SMPP, etc) without causing system outage in any way.
9.3.2 Responsibility of Quality Assurance
9.3.2.1 Notwithstanding any inspection, witnessing or observation of test by TELCO PROV CO representative(s) or any of its staff at any stage of the site acceptance tests, for full turnkey implementation, the Bidder shall be solely responsible for the quality of the system. For semi-turnkey implementation, the Bidder shall be responsible for the quality of the parts of the system that TELCO PROV CO is not responsible for.