The Significance Of Testing Throughout The Software
Development Life Cycle-A Roadmap
Asma Bhat*, Prof. S.M.K. Quadri
*Research Scholar, Department of Computer Science, University of Kashmir. Director, Department of Computer Science,University of Kashmir
* [email protected] A B S T R A C T
Software testing is an incredibly complex and imperative element of any software development lifecycle. This paper discusses it in broad-spectrum without taking any particular model of software development into consideration. The matter of the fact is that testing should start as early as possible in the developmental cycle as possible to keep the time and monetary issues under control. It is to tackle these issues that testing is to be performed at every step of the software development life cycle and that is what has been thoroughly discussed in this paper and a framework is presented to provide an initial roadmap for this very matter. A series of steps have been presented which take us through the whole classical software development life cycle meticulously and the tests possible at each step have been acknowledged as well.
Index Terms: Software testing, Software development life cycle (SDLC), Requirement specification, Static testing, Functional testing, Structural testing, Performance Testing, White box testing, Black box testing.
I. INTRODUCTION
A software development lifecycle is a series of steps which are followed in order to develop a software in a proper manner. There are various models present and anyone of them can be followed to develop software. Software testing is a key ingredient in software development lifecycle and a deficiency of testing has many a times resulted in many software associated inconveniences in the past [1] [2]. There are basically three types of testing mechanisms that can be used in the software development life cycle, where the first is static testing wherein we only do the examining, the second is functional and structural testing where actually tests are performed and the third one is the performance testing mechanism. Prime objectives of testing are as discussed further. First we have to discover the measure and sources of development risk reducible by testing and then we have to carry out testing to reduce identified risks. After that we have to know when testing is complete as also to administer testing as a standard project inside the development project. When we know we cannot test the whole system we check those parts which were deemed most important by the customer and also we test those parts of the system where the possibility of finding the error is maximum [3]. Testing is done primarily for the purpose of verification and validation. Verification is the course of action confirming that the software meets its specifications whereas validation is the process that confirms that the software meets the users requirements. Then again quality assurance and software testing are intertwined jobs because the main aim in quality assurance is the creation and enforcement of standards and methods to improve the development process and to prevent bugs from ever occurring [4].
Looking at quality in a naïve context it means that when the customer uses the software he or she encounters zero errors or defects. It is agreed upon here that quality testing is a subjective issue and that it is best to start testing early in the SDLC. The first reason why it should be started early is that the more the delay the more the cost involved in solving the particular defect [5] [6]. Basili and Boehm proved the axiom “test early and test often”. The cost that has to be incurred if we ignore testing at the earlier phases goes on increasing and it has been supported by Capers and has been proved statistically as well [7]. Now why software testing should be done throughout the software development life cycle beginning in the early stages? Basically both positive as well as negative software testing is useful. Positive testing is verification that the software works as it has been advertised or known to work whereas negative software testing is where we try to break the software. Functional, structural and performance testing is done in different stages of SDLC. Where performance is to be tested and manual testing is not feasible we use automated testing tools. When we have less time for testing we can very well go for the 80-20 rule wherein we test the software for those areas, usually that 20 percent of the software which is mostly used and if it works properly mostly the software works correctly but exceptional cases which aren’t that much used need to be tested properly as well but when there is scarcity of time and resources we know we have to follow this rule. Most of the defects are found in the specification and can be rectified there only even though the code is written later. It is because mostly errors are introduced in the specifications only as also in the designs of the process, data, interface, database, platform, connectivity, etc. and its better to rectify them in these stages only [5]. The cost to rectify the very same errors increases many fold in the later stages as well as the time needed to detect and rectify increases as well.
III. SOFTWARE DEVELOPMENT LIFECYCLE AND HOW TO GO ABOUT TESTING IN ALL ITS STAGES A software development life cycle is a series of steps which are followed when developing a software in a proper manner which is disciplined as well. Classic software life cycle model is today called the waterfall model. It basically has the following stages- planning, analysing, design and implementation phase. Each stage freezes and can’t be returned to in this type of model as per the classical rules. But this rule was always in need of being broken and in fact it was due to the changes that needed to be incorporated as well the time required to complete each step that other models were introduced. Then, a prototyping model was introduced wherein the basic rule was to develop the system through a series of iterations and to keep on bringing the changes in any stage and discard the prototypes until proper and acceptable software is fully developed. A prototype can be based on either evolutionary or requirement basis. Then there is rapid application development or RAD which is used in large scale environments. It has the unchanged stages as the orthodox waterfall model but supports it with heavier consumer association, computer based advanced tools as well as with advanced skilled tools. Then there is phase development life cycle which again is a slight variation of the classical model as well. We have incremental model, reuse oriented model and many others but the basic stages are the same in all of them [8]. Overall the different phases of software development life cycle support testing in the different stages as follows [3] [9].
Table 1.Tesing in different stages of SDLC
STAGE OF SDLC TYPE OF TESTING TESTCASE DESIGN
Preliminary investigation of requirements
Devise test plan for post implementation
Analysis Static testing of requirements
Pre-design phase Database design and application logic design tested through static testing Design Static testing of design documents like
entity relationship diagrams, data flow diagrams etc
Devise test case for final software
Preliminary model and its initial development
Static testing of code Functional testing White box testing Black box testing Grey box approach Performance testing
Initial administrative testing Recovery testing
Back up testing Initial security testing
Execute test cases for preliminary investigation of errors
Final model Static testing
Formal structural and functional testing
Performance test Regression testing Load test
User acceptance test Installation test
Execute test cases for final release of software
Installation Compliability testing Conformance testing Installability testing
Execute test cases for implementation
Post implementation Monitor performance within prescribed boundaries
Performance testing Pre release Alpha testing
Beta testing Acceptance testing
The nearer the testing and the making of the software is done, the better the software turns out to be. Herein is presented in a nutshell, a review of the case that has been studied in [3] as also various other sources were consulted as cited below to prepare a roadmap as to where the different testing techniques should be used in the software development life cycle and as shown in the table 1 above. And this is irrespective of the plethora of the models available. This roadmap is a flexible one and can be modified to suit most of the models available.
1. First and foremost, static testing is done to check the specifications and requirements and that they have been totally met. This obviously is the analysis phase. Herein we can formally either go for
specification testing or model based testing [6]. The attributes that we have to test while checking the specifications are completion, accuracy, precision, unambiguity, clarity, consistency, feasibility, testability and relevance [4].It reduces defects in the documentation and is basically of three types-Desk checking, inspection and walk-through. A little time spent in correcting and testing the documentation can go a long way in delivering a proper software and can save a significant amount of time and capital which would be required to solve the very same crisis in the later stages of the SDLC.
2. This step also is a part of the analysis phase and pre design phase. The test modelling is complete to a level and we know that 85% of the errors have roughly been reduced because of the testing of specification and now we can move on to the design phase. The databases design is tested here as is the application logical design and the cross validation of the logical design is done through the static testing. User interface defects although unearthed in the initial analysis phase itself must be retested in the design stage with for example graphical user interface testing. GUI testing can take help from equivalence class partitioning specially to find errors in data entry [3].
3. In the design phase, the first blueprint of test cases is for the use cases. Another category of test cases that can be checked and used over here are for the user interface and the architectural design which are not related with the use cases at all, at least in a direct way. Not to forget the testing strategy here must pay interest to the hardware and the software concerned apart from the application software being developed [3].
4. Requirement specification, system specification and system design which are accessible herein are helpful in designing acceptance test arrangement, system integration test plan, sub system integration test preparation, which are obviously required in the last stages of the SDLC [8].
5. Initial development of the software starts now. Meticulous specifications are written over here as also the coding begins. Static testing like walkthrough is advocated in this stage. When the testing begins in this stage only not only lesser time is needed to develop the test cases but better testing is done as also lesser the cost involved. Equivalence class testing is used in this stage to a fairly large extent. The initial preparation for performance and functional testing begins here.Black box testing is done whilst only the executable program and the data necessary is at hand and these requirements are typically met in about the core of the developmental life cycle of the software when chunks of code begin to work in tandem with one another. On the whole we can say as the design stage of the software development life cycle goes on the details of which areas need testing in isolation and which areas need to be tested in tandem become available and help design test cases as well
6. Now, the next stage is where our source code is available. So, herein white box testing techniques like statement coverage, branch coverage, path coverage, loop coverage etc can be used. As also we can go for adequacy assessment, data flow testing, domain testing, mutation testing, path testing, structural testing, test minimization through coverage. White box testingto be worked on in the SDLC now, as the availability of software requirements, use cases, executable program, its data as well as its source code is met. White box testing is an extension of code debugging activity to be carried out usually early in the developmental life cycle. As Capers puts it in, the earlier the defects are found the lesser the cost to rectify them. And if source code is not available we can always go for black box testing techniques like equivalence class testing and boundary value analysis as mentioned above or follow a grey box approach. Black box testing which can be used here to test the informal requirements are the ad-hoc testing, boundary value analysis, category partition, classification tree, cause-effect graphing, partition testing, predicate testing, random testing [10][11][12].
7. Administrative testing is now done to validate whether all the processes that the software was supposed to accomplish are in fact achieved.
8. Next in line is interface testing.
9. Now comes back up, recovery testing and smoke testing. Backup and recovery testing along with smoke testing are done on the last phases of SDLC. But for back up testing to work we have to keep taking backups on the first place at regular intervals [3].
10.Lastly we do the performance testing. Compatibility testing belongs to this stage and herein we check whether our software works correctly when it interacts and shares information with other softwares. This task can be achieved with the help of basic static testing as well as both white and black box testing where we thoroughly analyse the specifications of the product [4]. The basic function of performance testing is to check the software for performance during peak hours as well as the response time of the system which in the terms of a layman can be called the speed of the system. It is performed after the functional testing mostly, as also when the system is relatively stable. This testing is possible only after the areas and transactions which will have to bear the brunt of peak hours work load have been identified. For component performance testing the software must be functionally sound but the software development need not be finished.
11.Security testingtoo should be encouraged as early on in the SDLC as possible from the performance as well as the regression viewpoint.
12.Here we exit the initial development of the software. Any changes that are to be brought about are discussed between the testers and the developers in here and this is possible if both were working in tandem throughout the software development life cycle.
13.Final development of the software. The static testing here is the same as in the design and the initial development of the software. The testing begins by checking out the new use cases and the process flows in the architectural design.
14.Next comes the static testing of the program specifications which are derived from the requirements and predicate analysis can be used here to check the specifications. Generation of test from finite state model specifications helps in the later stages to check conformance to say the least [6]. Here we better the sub system integration plan as well as the unit and module test cases are designed [8].
15.Then static testing of program code written from specifications is carried out.
16.After this we move on to proper code debugging and code testing. The structural and the functional testing are done here but the approach is way more critical than in the initial development phase of SDLC. As a more formal model is built where we have mathematical and graphical specifications we can go for state chart testing, pair wise testing, finite state machine testing as also syntax testing. The interface is built almost by now and we can go for interface testing, interface mutation and also again pairwise testing. As also it would be worthwhile to mention here that unit testing is the way to follow here. Then as we go on combining the pieces of code, integration testing is possible, then later complete system testing [6]. Integration testing is done either by bottom up approach or top down approach or simply by integrating the two methods [3]. They together fall under the incremental approach methodology. In top down model all the driver modules are present (the calling modules) but stub development can be complicated i.e. the modules that need to be called whereas in the bottom up approach the stubs are present but the drivers need to be created to run them. Overall we use the mixed up sandwich approach [10] [13] [14].
17.Now we move on to performance regression testing to check that the new or changed or redesigned code doesn’t bring about any adverse reaction and it forms part of the overall maintenance as well. Here we also go for overall system testing which is to a huge extent supported by mutation testing. Regression testing is carried out commencing from version to version. This means that it is done in the later stages when new functionality is added or at least in the last phases of the software development life cycle in order to check and make sure that these new additions don’t bring about any problem in the software and that it works smoothly as it was working before the changes. Regression testing is nowadays an important part of the continuous process of software development and it plays a central role in software testing [6]. Regression testing means testing the software again and again and that can be quite a tedious task and its here that we understand the need to use automation tools for testing. They are faster, more efficient, relentless, accurate and way more precise. Testing tools don’t replace the testers but only supplement their work. As also they are of two types. Either they can be non-invasive where they only scrutinize and examine the software without modifying it or else they can be invasive where they manipulate the code and adjust the operating environment [4]. Regression testing is divided into three categories: Operational, organizational, and strategy [15].
18.Next we check the overall performance of the system as well as penetration testing, load testing, volume testing, operations testing, documentation testing and stress testing to gauge the overall security of the system. Compliance testing or conformance testing, etc are a part and parcel of checking the overall performance as well. Herein we check whether the deliverables of each stage of SDLC meet all the standards and guidelines. Compliance checking is simply where the standards and procedures are documented in each phase of SDLC and then we have an inspection where we find out whether everything was properly done or not. Documentation testing helps improve usability and reliability. We can use stress and load tools as automation tools for stress and load testing. Interference injectors and noise generators can be used here too. They are similar to load and stress testing tools but are more random in what they do. Having advocated the use of automation tools doesn’t mean we will go about putting all the work load on them because software specifications and features keep changing, there is no substitute for human eye and intuition and perception and basically the way human intervention is needed to keep an eye on the overall progress is indisputable [4].
19.Implementation stage gives way to installation testing and compatibility testing as well as configuration testing is also done herein where we check out the hardware features, modes and operations which are possible and the different software features which will work well with the different hardware combinations. Bug bashing can be done here wherein for a stipulated period of time all the testers concentrate on one buggy piece of code and when many minds meet many new bugs emerge as well as their solutions [4]. Installation testing is done in the last phases of the SDLC to check whether the software product is deliverable and will be properly installed in the environment where it is supposed to work. Configuration testing comes close to installability testing wherein the focus is to check the software as on which configurations it works on correctly from the many possible configurations it comes under.
20.Release testing can also be done here which is done by a different group which is not concerned in the development of the software at all and gives an unbiased opinion whether the software is ready to be brought out or not and follows a black box approach. Scenario testing supports release testing [8].
22.Alpha testing and beta testing are a part of the pre release and post maintenance phase. These help us to check whether the software will be accepted by the customer or not. Beta testing can contribute to many types of testing [4] [6].
23.Acceptance testing. This is the closing step of testing after which the software is customary for functioning use.
No stage is rigid as we can go on to the previous stages as and when we feel that there is an error that needs to be rectified. Developers as well as testers should work together in the SDLC lest the phenomenon- developers’ defects had the chief fix rate and expert testers’ defects had the least fix rate, might come up. So if they work together this negative effect can be lessened a bit [16]. One more thing that is realized in these steps is that the requirements that are identified in the earliest sections of the SDLC are the ones that actually help us in deciding whether or not we want to accept the software or still more work need be done. Testing should be a part of the SDLC, to a small extent agile software development process supports this view. Even though, they are mostly well-matched to application development where the system requirements typically alter quickly during the development procedure. They are intended to deliver operational software quickly to customers, who can then propose fresh and transformed requirements to be incorporated in later iterations of the system. Agile software development advocates continuous creation of workable deliverable software to the customer which is obviously well tested and then changing the requirements as need be but here the aim we have is not to keep changing the requirement as a necessity but we bring about the changes and test the requirement using different techniques initially only so that then in the later stages we don’t have to create an altogether new software and that whatever we deliver in the end is the best possible and hopefully the final version of the software.[8]
Overall, now we look at the the crux of these steps of SDLC and the testing involved in each steps . Initially we go for requirement study, then in the design phase we go for component specification and test specification design and review, then code review, performance and simulation testing is done and finally for deployment we do test closure and test process analysis to check the overall performance of the software under development[14]. The simple matter of fact here is that we are actually trying to integrate testing with the development life cycle wherein test planning, writing and designing test cases, executing the test cases and bug tracking and defect analysis report are a part of the classical SDLC [9]. At the different steps the opacity can be black or white box testing , some steps require high level design, and some low level design , as also some require requirement specification and some code structuring as explained in the steps mentioned above [5][17]. A relatively new concept of ZDLC i.e. zero deviation lifecycle has been introduced wherein where there is minimum deviation from the requirements and there is as little defect injection as possible between the different stages of SDLC. This method helps in controlling the defects in the earlier stages of SDLC only so that we don’t have to incur the heavier cost of rectifying the fault in the later stages of SDLC and can be adopted in this framework as well[18].
IV. CONCLUSION AND FUTURE SCOPE
The framework presented in this paper not only covers the plethora of the software testing techniques but also presents a detailed framework which explains how testing can easily be carried in the initial stages as well and how it eases the test case formation. Not just that but since the errors are caught up earlier only, the cost and effort to rectify them is lesser as well. As also the testing team and the developmental team can work together like this and with more mutual co-operation. This paper can serve as a starting point and using the frame work herein case studies can be formulated to use test testing techniques throughout the software development lifecycle. And since no particular model has been followed the framework can easily be modified to suit which ever model suits the developmental
team as the steps have been divided really painstakingly and meticulously. Then as the testing goes on throughout the SDLC the end result that we get is guaranteed to be of a better quality as well.
V. REFERENCES
[1] S.V. Naga Srinivasu, Dr. I. Ramesh Babu, “A new approach for estimating software testing techniques for reliability”,International Journal of Engineering & Science Research (Published by IJESR.), vol. 3, no. 27, pp. 2538-2542, Feb 2013.
[2] Nishu Rastogi ,“An empirical study of software testing approaches “, International journal of system and software engineering (publishingindia.com), vol. 1,issue no. 1, pp.38-42, June 2013. [3] Gerald D.Everett, McLeod Raymond,Jr, Software testing: testing across the entire software
development life cycle,( John Wiley & Sons,Inc., Hoboken), New Jersey, 2007.
[4] Ron Patton, Software testing. (Sams Publication) 800 E. 96th St., Indianapolis, Indiana, 46240,
USA. , 2006.
[5] Yogesh Singh, Software Testing, (Cambridge University Press), New York, USA, 2012. [6] Aditya P.Mathur, Foundations of Software Testing, 2/e.( Pearson Education ), India, 2008. [7] Elias M.Awad, Systems analysis and design, (Galgotia publications pvt.ltd), Delhi, India, 2008. [8] Ian Sommerville, Software Engineering. Edition 9,(Addison-Wesley, Pearson) , USA., 2009.
[9] T.Rajani Devi, “Importance of testing in software development lifecycle” , International journal of scientific and engineering research(Published by ijser), vol. 3, issue no. 5, pp. 01-05, May 2012. [10] Glenford J. Myers, The art of software testing, (Published by John Wiley & Sons, Inc.) Hoboken,
New Jersey, 2004.
[11] Mohd. Ehmer Khan, Farmeena Khan. "A Comparative Study of White Box, Black Box and Grey Box Testing Technique”, International Journal of Advanced Computer Science and Applications (Published by IJACSA), vol. 3, no. 6, pp.12-15, 2012.
[12] Mohd.Ehmer Khan,” Different Approaches to White Box Testing Technique for Finding Errors”, International Journal of Software Engineering and Its Applications (Published by IJSEA), Vol. 5 , No. 3,pp.01-14, July, 2011.
[13] B. B. Agarwal, S. P. Tayal, M. Gupta, Software engineering and testing-An introduction,(Jones and Bartlett publishers), Sudbury, Massachusetts, 2010.
[14] D. Meenakshi, J.Swami Naik, M. Raghavendra Reddy, “ Software testing techniques in software development lifecycle“, International journal of computer science and information Technologies(Published by IJCSIT), vol. 5, no. 3, pp. 3729-3731, 2014.
[15] David Parsons, Teo Susnjak, and Manfred Lange, "Influences on regression testing strategies in agile software development environments." Software Quality Journal (Springer Science+Business Media New York) ,vol. 22, no. 4 ,pp. 717-739, 2014.
[16] Mika V. Mäntylä, Juha Itkonen, Joonas Iivonen. "Who tested my software? Testing as an organizationally cross-cutting activity." Software Quality Journal (Springer Science+Business Media New York ),vol. 20, no. 1 ,pp. 145-172, 2012.
[17] Srinivas Nidhra, Jagruthi Dondeti. "Black Box And White Box Testing Techniques–A Literature Review." International Journal of Embedded Systems and Applications (Published by IJESA), vol 2, no. 2 pp. 29-50, June 2012.
[18] Y. K. Makoondlall, S. Khaddaj, B. Makoond. "ZDLC for the Early Stages of the Software Development Life Cycle." In Distributed Computing and Applications to Business, Engineering and Science (DCABES), 2014, 13th International Symposium on, pp. 6-12. IEEE, 2014.
