QA in DevOps. Why a great development team isn t enough? Vinai Gangireddy, Director & Practice Leader for QA & Testing

Why a great development team isn’t enough?

Reimagine technology to accelerate your business

Vinai Gangireddy,

Director & Practice Leader for QA & Testing

QA in DevOps

DevOps Introduction

DevOps is a set of practices that combines development, QA, and IT operations to reduce time to market. It leverages processes and tools that automate the software development life cycle and support accelerated project delivery goals. The popularity of a DevOps approach is due to its ability to build and improve products at a faster pace, compared to traditional software development methodologies.

Key outcomes of DevOps include:

• Shorter development cycles with continuous delivery and innovation

• Reduced implementation failures

• Tighter feedback loops and quicker recovery time • Less complexity to manage change

• Improved communication and collaboration

Traditional QA vs DevOps QA

Traditionally, the QA team receives a product build from the development team. The build is placed in the designated testing environment where the QA team conducts functional and regression testing. When the build has been verified by the QA team, it is released to production. This process typically takes between a few days and a few weeks to complete.

In the DevOps methodology, throughout the life cycle,

developers and quality analysts share responsibility for properly functioning software and timely delivery. Quality functions are integrated in a collaborative approach to development, augmented by tools and infrastructure that ensure a fast and seamless process. Builds are delivered continuously to the testing environment, where developers and testers work collaboratively to increase product quality.

Critical rules for QA in a DevOps process:

• QA analysts must validate requirements during the design process and before coding is started as the clarification process will almost certainly change the code and may also change the architecture.

• QA tools, templates, and processes must be customized to align to the DevOps methodology.

• Automation of both unit and functional test cases must be employed to the maximum possible code coverage, minimizing manual testing.

• Deployment to all environments is automated

• Automated test cases should be properly aligned with the Continuous Integration Cycle

• The continuous planning aspect of DevOps requires continuous engagement with product owners

A key aspect of DevOps is the ability to automate the build, deploy, and smoke test process. This eliminates human errors we frequently see in these steps, while also significantly increasing the speed. Specialized automation and continuous integration tools help achieve this goal.

In addition, QA functions are integrated throughout the

development life cycle, from design to deployment. QA analysts validate the clarity of requirements by decomposing them into test scenarios, collaborate with developers to create automated unit tests, prepare functional and regression automated and manual tests, and coordinate with developers to include smoke tests in automated build/deploy processes.

The specialized training, experience, and mindset of QA analysts/testers enable critical success factors:

• Understanding exactly where requirements may be misinterpreted or assumptions made by development or if there are any ambiguities that need clarification

• Since it is impossible to test every business scenario, knowing where to focus, what to validate, and what test data provides the highest risk coverage are some of the key skills leveraged by quality professionals.

• There can be no DevOps without automation, and great

and limitations of various frameworks and know intuitively which one to choose based on different application factors. They also understand how to maximize the value of

automation and minimize future maintenance, what should be automated, and where automation should be avoided. • A big risk in fast-paced DevOps development is that

releases which should focus on new functionality may include rework from previously missed requirements and defect fixes, which will ultimately slow the pace of overall development. Employing the skills of quality professionals throughout the life cycle not only protects against this possibility, but also supports the concept of ongoing improvements so that a DevOps team can become increasingly more efficient and more effective in producing high quality software at a very rapid pace.

Ciber’s Approach for QA in DevOps

The key QA aspect to consider in DevOps is that the testers are teamed with the developers. In this way, the QA team ensures quality throughout the development life cycle, defining tangible aspects of quality that are expected from the application. This helps to achieve timely results.

Another important action is defining quality metrics, which

UAT

QA

DEV

Team

Utilizing Tools and Frameworks

Business/User Acceptance

Use Case Review Test Estimations Test Planning Acceptance Criteria HLD Tech Specs Use Cases Test Plans Test Reports Test Sign Off

Solution Design Design Architecture Process Flow

Automated Acceptance Test Functional & Regression End -to-End Validation Build Automation Suite Performance Testing Test Execution Defect Logging Defect Reviews Defect Triage Root Cause Analysis Test Reports Metrics Test Sign Off

Intake Process

Interface/API CR Details

Code Progression Through Environments

Code Reviews Code Analysis Code Coverage Test Data Prep Automated Unit Tests Test Framework Automation Scripts Early Performance Tests Integration Tests

User Acceptance Testing Defect Reviews Defect Triage & Metrics Root Cause Analysis Test Reports & Sign Off

PROD

QA in DevOps

Shift-Left

Defects/ Change Requests

Requirements Backlog/Intake

While testers can see the developer's perspective, they maintain their tester mindset that supports objective testing and avoids the limitations of author bias. Working directly with the developer, the tester can fix the code or assist the developer with technical insights.

By working together, both developers and testers are able to start end-to-end testing long before all of the code is written. The developers create special drivers that simulate missing code, while the testers are mentored to simulate real-life business scenarios in pre-production environments. This enables bug detection much earlier in the development process — long before the full product development is completed.

Performance, load, and security testing is also started early in the development cycle. The non-functional testing team operates as a shared service, integrating with the various application teams to identify performance, load, and security issues early in the development process. Members of Ciber's Enterprise Quality Assurance and Testing Practice are cross trained with automation skills and DevOps tools, which are extensively implemented in all our DevOps projects. Our library of CI/CD frameworks enables fast setup for our customers and quick onboarding of team members. In addition, Ciber has partnerships with leading vendors of quality assurance analysis and

automation software.

Conclusion

Ciber embraces the idea that ‘DevOps is a culture, not a team.’ While addressing customer demands and

shortening lead time between design and deploy, DevOps QA has a larger role to play. DevOps can speed up development but having a solid QA strategy helps prevent defect creation and creates a better production product. At the same time, building a self-sufficient QA team with strong skills to support the DevOps process at all levels is an important objective. Automation is used throughout the life cycle and a proven framework is required to support continuous integration and a continuous delivery model. In one case study, Ciber partnered with a leading

hospitality provider in an enterprise initiative to migrate all their applications to a micro services platform. Ciber implemented leading CI/CD tools and customized CI/CD flow involving development, code repository and builds, and deployments. Today, they have executed over 15 production releases, selectively activating code with feature flags depending on application readiness. Bug introduction is very low and virtually all identified bugs are fixed the same day they are found, significantly reducing development costs.

Vinai Gangireddy

About The Author

Vinai Gangireddy leads the Enterprise Quality Assurance and Testing Practice (EQAT) at Ciber Global with over 20 years of testing experience. He has effectively executed multiple software testing projects. Working in different technology stacks and domains, he has assisted numerous clients in implementing quality best practices and testing solutions that have resulted in high-quality software.

3270 West Big Beaver Road | Troy MI 48084

www.htcinc.com www.caretech.com www.ciber.com