LISA - Virtualized Services
Capgemini India Private Limited July 2013
Prepared by: Srikumar Bose
Table of Contents
1.
Abstract ...2
2.
Background ...4
3.
General Testing Challenges to be addressed by Virtualization ...5
4.
Anticipated Benefits of Virtualization ...6
5.
LISA Virtualize Feasibility Study ...7
6.
Test Conduct ...8
7.
LISA Virtualize Test Evaluation ...9
8.
Additional Potential Benefits Identified ...10
9.
Protocols Demonstrated ...11
10.
Challenges Identified ...12
11.
Practical Applications ...13
11.1
Testing the Transformation Service (TS) ...13
11.2
Testing the Rate Process Service (RPS) ...14
11.3
Third Party Vendor Testing ...14
12.
Areas for Further Investigation ...15
12.1
Skill Set Development ...15
12.2
Cross Domain Support ...15
12.3
Unsupported/Evolving Capabilities of Tool under Test ...16
12.4
Cost ...16
13.
Conclusion ...17
14.
References ...18
1.
Abstract
In the effort to make testing more cost effective and efficient, one set of expenses stand out – the costs to create and maintain testing environments. An inadequate test environment reduces the effectiveness of testing thereby increasing risk to the project. However, the resources required to maintain replicas of the complex production environment with its plethora of internal and external endpoints constrain development efforts, extend timelines and raise costs of development.
An analysis that looked at the feasibility of virtualized test environments within the existing computing infrastructure was commissioned. One of the issues identified was the minimum number of tools in house that would help support a virtual test environment. Thus, a project team was assembled to begin to identify tools that could possibly facilitate this task.
IT section of one of the leading NA Insurance Company currently has a Web Services Testing tool called LISA which was marketed by the iTKO Corporation. They offered another tool that models Virtual Services. LISA Virtualize is a virtual server environment that supports many of the techniques that would purportedly facilitate many of our virtualization ambitions. In collaboration with its vendor, the project team hosted a proof of concept (POC) to determine whether such a tool could be used to assist our efforts. This whitepaper documents those efforts.
Virtualization is needed when a live service does not exist, when a live service is underperforming, when there is restricted access to the service or to avoid 3rd Party data discrepancies/inconsistencies across tiers.
LISA Virtualization creates reusability throughout the SDLC and allows us to easily add and grow to an existing virtual service. All that is needed to Virtualize is one request-response set in the form of two separate XML’s.
2.
Background
LISA Virtualize (formerly LISA Virtual Service Environment) decouples the dependency on live or unfinished systems and components throughout the lifecycle of building and testing SOA applications.
LISA Virtualize models the expected functionality of any service within SOA applications, whether it’s a Web Service, WSDL, an integration layer such as an ESB, or an implementation layer such as an EJB, mainframe or database, and represents that service virtually, with a high level of dynamic input and output capabilities and performance characteristics that simulate the real-world behaviors of complex systems.
3.
General Testing Challenges to be addressed by Virtualization
There is a need to identify how to reduce the complexity and resulting costs of creating and maintaining pre-production test environments in specific areas:
Data Management - The creation and maintenance of a synthetic data to mitigate the considerable costs associated with the current reliance of mainframe data even for test efforts not involving changes to mainframe data.
Quality - The ability to isolate test efforts from the considerable volatility inherent in test regions shared by multiple test efforts, resulting in more consistent and meaningful test results.
Test Construction – The ability to sidestep the limitations on regression test construction and automation imposed by fragile data, region specific data and endpoints under construction.
Support for New Products and Initiatives – The ability to model new products, new interactions among systems not yet developed.
Efficiency – Ability to execute the same tests using fewer resources. Reduce expenditures for third party services on the pre-production tiers.
4.
Anticipated Benefits of Virtualization
The ways we anticipate that virtualization can assist the test effort are:
Ability to “Virtualize” sub-systems - after recording a set of data exchanges and substituting the “virtual endpoint” for the actual endpoint, the application under test interacts with this “virtual endpoint” in all respects as though it were conversing with an actual endpoint. The advantage being that the request/response expected results are known and invariable, isolating the testing to the application/service under test from all the subsystems required to support the application/service. This is also a principle advantage when third party vendor information exchanges are required for the application or service to work, but are ancillary to the actual test. Date Manipulation - the tester can capture interactions with an existing subsystem and then instruct
the LISA product to “age” the data to whatever date is required to exercise the business rules under test without altering the underlying data which facilitates the reuse of data.
“Magic String” isolation - when a requesting application or service sends one of these values to the LISA product, unique key values, such as request reference numbers, p2cid’s etc. will be “echoed” back to the requestor, as the requestor expects, eliminating a great deal of data maintenance and the ability to move test cases among test regions (so long as all tests use the same universal data set).
WSDL data set generation – the tester can generate the tags and a data set for a proposed WSDL which enables testing an existing application or service’s interaction with a proposed application or service facilitating parallel development among applications that will eventually have to interact.
Support for other messaging protocols – any support for asynchronous messaging protocols such as JMS, MQ, etc. would support any emergent technologies.
5.
LISA Virtualize Feasibility Study
LISA Virtualize was tested for four days and exercised both client specific Virtual Service Environments (VSE’s) to exercise client specific conditions as well as demonstrated LISA example projects that featured specific capabilities.
6.
Test Conduct
The test team identified several services and applications where constrained resources or complex data requirements result in considerable setup costs, data acquisition costs and data verification. Such projects are perceived as being the biggest potential benefactors of this product.
Transactions were to be recorded and replayed as “virtual” components to assess the product’s ability to mimic our subsystems and assess the usefulness of such transactions.
ITKO provided a technician who assisted the client team in setting up the test environment. There were a number of challenges in set up that required number of technical and development I/S areas. That being said, the entire setup of the virtual environment took far less time and resources (albeit different resources) than typically required of a test region. First time setup took less than 12 hours, future setups should take significantly less time.
After setup was complete, the recording of virtual transactions was no more complex than executing a LISA test and was quickly accomplished. Using recorded transactions involved resetting the endpoint in the configuration files – and this was a challenge only because some applications stored configurations inside .jar files rather than exposing them in configuration files, but after those issues were resolved, using the virtual data requires no special expertise beyond the ability to use LISA.
7.
LISA Virtualize Test Evaluation
The POC was considered successful because the product under test demonstrated the following capabilities:
Creation of a virtual service from a WSDL. A WSDL can be exposed easily to create a spreadsheet for a non-technician to use to create a dataset. That dataset can then be used as a substitute for the actual service.
Data capture from existing systems through the use of listeners inserted in front of the application or service to be virtualized. The captured communications are then used as “virtual” versions of the system that is not under test system under test to work.
Simulation of interactions with a virtualized application or service created during the capture operation. The application under test was able to interact with the static set of transactions recorded and return the expected values as though it were communicating with the actual subsystem. We used a set of previously recorded XML requests and responses from RPS.
Simulation of interaction with future services based on architectural specifications, schemas and service configuration definitions. The LISA harness was able to expose the data fields in a spreadsheet format so that valid values could be supplied. The test continued as those valid values were then used to simulate a virtual subsystem to exercise a data exchange between the system under test and the future system that is not yet available.
8.
Additional Potential Benefits Identified
In addition to the major functionalities identified, there are a number of additional abilities that would be particularly valuable in our current environment:
Date Manipulation –with this product, the tester can capture interactions with an existing subsystem and then instruct the tool to “age” the data to whatever date is required to exercise the business rules under test without altering the underlying data which facilitates the reuse of captured data.
The ability to age reports for GRS. Current methodology to keep CBR and MVR reports within the date range required to create the desired test condition involves manipulation of the report dates on the mainframe – a labor intensive and time consuming task that must be repeated frequently for various test efforts in various regions. Creating a virtual GRS for all but GRS testing would eliminate this task, allowing the tester to age the reports in the virtual environment relative to requirements of the test.
The ability to “freeze” birth dates, ONICC dates, etc.
Many test conditions rely on specific dates relative to the effective date of the transaction in the test condition. (i.e.). The driver must be exactly 18 years old; the NGCC date must be at least 3 years from effective date, etc. This particular tool would allow the tester to specify the relationship that these dates must have to the effective date of the policy.
“Magic String” isolation - when a requesting application or service sends one of sets of values to the tool, unique key values, such as request reference numbers, p2cid’s etc. will be “echoed” back to the requestor, as the requestor expects, eliminating a great deal of data maintenance and the ability to move test cases among test regions (so long as all tests use the same universal data set).
The ability to virtualize region specific data. Values such as P2CID, report reference number, etc., tie current test efforts to a specific test region and when any test effort in that mainframe region requires “fresh” data, the existing carefully constructed test conditions are destroyed. The LISA tool would enable the tester to use a virtual data set where the region specific data could be mimicked such that the test efforts would become less dependent on region specific data.
9.
Protocols Demonstrated
SOAP/HTTP – We were able to observe actual Client RES services recorded and played back feeding transactions to the RPS service imitating the RES returning rates.
XML/HTTP (REST) – Demonstrated a REST service Asynchronous (JMS MG, etc.)
10.
Challenges Identified
Cross domain support – Setting up a virtual test set requires quite a bit of developer, TPS and other Base Tech support.
Project documentation – Identifying service endpoints and application/service process flows was not easy. As a best practice or standard, detailed architectural documentation should be required for all solutions that are built and maintained within I/S.
CTG – not supported. We had a WSDL wrapped CTG call that we could not execute because of the proprietary nature of CTG. The vendor indicated the willingness to work with us to develop the ability to use CTG but it is not available now.
HTTPS – LISA Virtualize struggles with SSL. The passing of trust certificates seems to be a significant issue. We were able to work around the problem in a very convoluted way, but the vendor is working on a simpler solution.
11.
Practical Applications
Some practical applications of the virtualization technology came immediately to mind with several members of the team:
11.1
Testing the Transformation Service (TS)
Many times we are involved into new projects, either with services under development or services, which are already developed as far as WSDL goes. One of service that could benefit a lot from this tool is the Transformation Service (TS), which is a service being developed for the Integrated Claims Application (ICS).
This service lies in between Product Service (PS) and Commercial Lines Policy Service (CLPS). It converts the V1 schemas coming from CLPS to ICS compatible V2 schemas. In order to test the complete integrity of the workflow, it would be useful test CLPS for both, functional as well as performance; test PS for functional as well as performance and TS for both functional and performance.
At one point in the development process, where CLPS and PS were coded completely but TS wasn’t. The desired test condition would be to use the same set of test data to test CLPS, TS and PS to make sure there was consistency in the results and to confirm the operational and functional accuracy of these services, but due to unavailability of the TS, only CLPS and PS could be tested.
With the LISA Virtualize tool, the testing effort could include creating a Virtual Service out of the available WSDL and the corresponding schemas and could have inserted a virtual service between PS and CLPS thus completing the entire path.
Also with LISA Virtualize, testing could have setup a virtual TS service to test the validity of the WSDL and the schemas and also the various operations
Without LISA Virtualize, the functional as well as performance testing of the TS was delayed by at least a month until the code was ready and deployed to INT and PERF Test environments for the test team to test. The testing thereafter took almost a week to complete since we had to setup the test from scratch since we had nothing before. VSE could have saved us at least a week’s effort in setting up the service and validation. Also basic testing could have completed without the actual code being present.
11.2
Testing the Rate Process Service (RPS)
RPS regression testing involves a complex data set including persons from the CIM database, reports from GRS. Neither of these subsystems are under test during RPS testing and both of them tie RPS to a specific test region because the Social Security Number of the P2CID of the rating request must match the SSN of both the CIM record and the credit score record stored in GRS.
The data setup in GRS requires a complex set of manipulations so that the reports returned match the scores and dates required for the RPS tests.
Test data in one region cannot be used in another because the P2CID’s will not match and the test will return data not found conditions.
The setup of this data can take more than 80 hours to setup, manipulate and verify before any test can be completed.
The date and magic string manipulation capabilities of LISA Virtualize would enable a “universal test data set” for RPS that would make that testing independent of any mainframe region, greatly reducing the test costs to that project.
11.3
Third Party Vendor Testing
Any test effort that requires third party vendor data could substitute a virtual test data set for the interaction with the actual vendor, reducing the test setup costs for the project. In addition, any “per use” or monthly subscription costs required to support test efforts could be either eliminated or greatly reduced.
12.
Areas for Further Investigation
The POC was essentially a demonstration of a product by a vendor representative very familiar with the product but not at all familiar with the client architecture, personnel and procedures. So, there was some simplification of some tests that we would have liked to more fully explore. In addition, many resources had brought in from other domains and if this will be an on-going requirement, this could impact current test conduct.
12.1
Skill Set Development
This product shows great promise in supporting Agile environments. However, because of its ability to simulate the response for an application call to a service, database or other component, technical knowledge and support for this product is not trivial.
Test Engineers –Although existing LISA scripts can be incorporated into the virtualized environment relatively easily, and executing a test in a virtualized environment is no more complex than standard LISA testing, creating the initial virtualized environment is another matter. Knowing what to Virtualize and how to use the “magic string” and “date manipulation” capabilities of the product effectively will require a tester pool of some considerable experience with LISA in general and the Virtualize product in specific.
Administration – once these virtualized environments are created, there will be a need to promote and administer their proper use, so that the investment is best leveraged.
12.2
Cross Domain Support
Although the tool tested makes some of this easy, proper positioning of listeners and the incorporation of the resulting virtualized components will require significant understanding existing pre-production environments and deployment protocols. The nature of virtualization almost ensures that similar cross domain support will be required of any virtualization effort
There are several cross-domain activities of particular interest:
MQ –Queues must be setup for the tool. Capturing data has to be done via logging data as it enters the queue that is to be simulated. Currently, this activity is handled by the MQ administrators, some consideration should be given to allowing the virtual team to create and administer their own queues in the virtualized environments.
Database – Although it is possible to capture the virtual data exchanges in a spreadsheet, ideally, the LISA virtualized data should live in databases for ease of maintenance and administration. Current practice would require DBA support. Some consideration of allowing the virtual team to create their own queues in the virtualized environments.
12.3
Unsupported/Evolving Capabilities of Tool under Test
The tool did not have a flawless debut. There were some problems with playback noted and we struggled around some issues with security. The product clearly is still under development but the issues surfaced did not prevent the majority of our testing.
CTG Is not supported at this time with this tool. We noticed some CTG calls “wrapped” in WSDL in our test applications. Based on a discussion with the POC technician – this type of service wrapper will not work with the Lisa product. Because direct connection via CTG is not the preferred method of exchanging data at client and generally confined to Websphere applications that are being phased out, this is not a major issue.
HTTPS is not supported elegantly. The setup was awkward at best, and a more honest assessment would put it at a hack. The vendor is addressing this issue.
12.4
Cost
A complete TOC and ROI analysis would be required for any virtualization solution considered. The TOC/ROI analysis should include the costs of acquisition, maintenance and administration over 1, 3 and 5 year periods.
13.
Conclusion
Although this type of product has shown great potential in the demonstration, a further assessment is required to determine utility and cost effectiveness of virtualization in the American Family test process. Properly assessing the projected ROI from implementing virtualization must be based on a solid understanding of client’s testing goals and objectives, how the product may help the testing effort contribute to those goals, and development of a sound plan to effectively implement the product in our current complex environment.
During the proposal project, the team would identify and address issues each of these categories:
Architectural – Determine how the tool is to be implemented and the on-going vendor and
non-testing support required to keep the product usable.
Organization - Determine how this product meets business goals and objectives and priorities.
Financial - Define the benefits, know where the real costs lie, and define the budget.
Implementation - Prescribe an incremental improvement approach starting in projects where the
product would be both least intrusive and most beneficial.
Monitor and measure - Develop a consistent set of metrics for measuring and monitoring the new
product.
Other testing goals and objectives important to assess:
Ability to provide sufficient training and build technical expertise for test team Ability to leverage the investment over the long term
Ability to decrease time-to-market for in-house development Ability to optimize the reusability of tests and test automation Ability to improve test output and coverage
Ability to increase managerial control over quality and testing Visibility into quality and testing effectiveness
14.
References
http://en.wikipedia.org/wiki/Service_virtualization http://en.wikipedia.org/wiki/Itko http://www.itko.com/solutions/index.jsp http://www.ca.com/us/products/detail/CA-LISA.aspx http://www.ca.com/us/collateral/product-briefs/na/CA-LISA-Service-Virtualization.aspx http://www.ca.com/us/service-virtualization.aspx15.
Author’s Biography
Srikumar Bose has nearly 10 years of experience in Insurance, Cards as well as Retail Banking Domains. Worked extensively in CICS, DB2, Visionplus, Cardpac, e-Banking & Guidewire and is currently working in “AMFAM” account for P&C Insurance with Guidewire v7.0.6.
Performed as an SME for E-banking, Automation Expert (QTP & LISA) as well as provided Data Support & Environment Support along with Quality Assurance / User Acceptance / Base / Compliance / Product Upgrade / System Testing / ETL Testing & Data Migration Testing for GE UK (Hyderabad Unit), FDI (Chennai Unit), RBS [Gurgaon, London (UK) & Chennai Units] and CNA (Chennai Unit) Projects in Capgemini India Pvt. Ltd.
Expertise in CICS, TSO, File Aid (DB2), Oracle SQL, UNIX & JCL. Extensive knowledge in Batch processing, report generations & processing’s, Running Jobs using Control Cards as well as ETL process.
