Many advocate the independence of VV&T pertaining to simulation, particularly since independence provides an objective assessment platform, adds a new analytical per-spective, brings its own set of tools and techniques, introduces intermediate users, and significantly enhances testing as well as discovers design flaws and coding errors (Arthur & Nance, 2000). VV&T clearly crystallizes this need when the system is real-time critical software, the program has a high cost of failure, or the cost of error detection, maintenance, or modification exceeds the cost of VV&T. This highlights the fact that with independence we mean: technical, managerial, and financial independence (Arthur &
Nance).
Future Trends
Indeed testing is a tedious, mind-numbing, lackluster, repetitive, and long process. Just like compilers started years ago to locate syntax errors, VV&T tools are becoming well known. In fact, as seen in Table 2, there are many tools that test software. Whether the
Product Web site Use For vectorCAST www.vectorcast.com for c/c++, Delphi,
java, Visual Basic,
visual c++ —
testcomplete 3 www.automatedqa.com for win32, java, .NET — The Grinder http://sourceforge.net/projects/grinder Java load-testing 5/25/ 2005
Mantis http://www.mantisbt.org/ Php, MySQL,
Web-based bug tracking 5/25/ 2005 OpenSTA http://opensta.org/download.html Scripted HTTP and
HHTPS load test 5/16/ 2005 TestMaker http://www.pushtotest.com/ solves bugs in the
HTTP protocol
VV&T technique is static or dynamic, formal or informal, many tools have been developed to do the grubby work for the developer. Some tools, like ProofPower, go as far as offering supporting proof in higher order logic (HOL) and in the Z notation. As such, those mentioned hereinafter will be part of our life in validation, verification, accreditation, and surly independence.
Conclusion
In conclusion, this chapter gave a full-fledged overview regarding the simulation of VV&T techniques, whether they may be procedural or object-oriented simulations, putting forth various taxonomies for both types of VV&T techniques. Furthermore, the chapter highlighted the fact that the use of UML in simulation can be tuned to serve the purpose of simplifying the VV&T process.
In addition, mapping best practices of software engineering to simulation process, thirteen best practices recommended to be integrated within the simulation system development.
Finally, the chapter pinpointed the significance of the independence of VV&T, identify-ing the reasons behind advocatidentify-ing for it, the most important of which would be capability to provide an objective assessment platform.
References
Arthur, J., & Nance, R. (2000, December 10-13). Verification and validation without independence: A recipe for failure. In J. A. Joines, R. R. Barton, K. Kang, & P. A.
Fishwick (Eds.), Proceedings of the Winter Simulation Conference (pp. 859-856).
San Diego, CA: Society for Computer Simulation International.
Balci, O. (1994). Validation, verification, and testing techniques throughout the life cycle of a simulation study. Annals of Operations Research, 53, 215-220.
Balci, O. (1995). Principles and techniques of simulation validation, verification, and testing. In C. Alexopoulos, K. Kang, W. R. Lilegdon, & D. Goldsman (Eds.), Proceedings of the 1995 Winter Simulation Conference (pp. 147-154). New York:
ACM Press.
Balci, O. (1997, December 7-10). Verification, validation and accreditation of simulation models. In S. Andradóttir, K. J. Healy, D. H. Withers, & B. L. Nelson (Eds.), Proceedings of the Winter Simulation Conference, Atlanta, GA ( pp. 135-141).
Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Balci, O. (2003). Verification, validation, and certification of modeling and simulation applications. ACM Transactions on Modeling and Computer Simulation, 11(4), 352-377.
Balci, O., Glasow, P. A., Muessig, P., Page, E. H., Sikora, J., Solick, S., & Youngblood, S.
(1996, November). Department of defense verification, validation and accredita-tion (VV&A) recommended practices guide. Alexandria, VA: Defense Modeling and Simulation Office. Retrieved May 25, 2005, from http://vva.dmso.mil/
Mini_Elabs/VVtech-dynamic.htm
Balci, O., Nance, R., Arthur, J., & Ormsby, W. (2002, December 8-11). Expanding our horizons in verification, validation, and accreditation research and practice. In E.
Yücesan, C. H. Chen, J. L. Snowdon, & J. M. Charnes (Eds.), Proceedings of the 2002 Winter Simulation Conference (pp. 653-663). Piscataway, NJ: IEEE.
Balci, O., & Sargent, R. G. (1981, April). A methodology for cost-risk analysis in the statistical validation of simulation models. Communications of the ACM, 24(11), 190-197.
Banks, J. (1999, December 5-8). Introduction to simulation. In P. A. Farrington, H. B.
Nembhard, D. T. Sturrock, & G. W. Evans (Eds.), Proceedings of the 1999 Winter Simulation Conference (pp. 7-13). New York: ACM Press.
Banks, J., Carson, J., Nelson, B., & Nicol, D. (2001). Discrete-event system simulation (3rd ed.). Upper Saddle River, NJ: Prentice Hall International Series.
Baradhi, G., & Mansour, N. (1997, September 29-October 2). A comparative study of five regression testing algorithms. In Proceedings of the Software Engineering Con-ference, Australia (pp. 174-182).
Binder, R. V. (1995). State-based testing. Object Magazine, 5, 75-78.
Carr, J., & Balci, O. (2000, December 10-13). Verification and validation of object-oriented artifacts throughout the simulation model development life cycle. In A. Joines, R.
R. Barton, K. Kang, & P. A. Fishwick (Eds.), Proceedings of the Winter Simulation Conference (pp. 866-871). Piscataway, NJ: IEEE.
Doong, R., & Frankl, P. (1994). The ASTOOT approach to testing object-oriented programs. ACM Trans. on Software Engineering and Methodology, 3(2), 101-130.
Freedman, D. P., & Weinberg, G. M. (2000). Handbook of walkthroughs, inspections, and technical reviews: Evaluating programs, projects, and products (3rd ed.). New York: Dorset House Publishing Co.
Gannon J., McMullin, P., & Hamlet, R. (1981, July). Data-abstraction implementation, specification, and testing. ACM Transactions on Programming Languages and Systems, 3(3), 211-223.
Harrold, M., McGregor, J., & Fitzpatrich, K. (1992). Incremental testing of object-oriented class structures. In Proceedings of the 14th International Conference on Software Engineering, Melbourne, Australia (pp. 68-80). Retrieved April 2004, from http:/
/citeseer.ist.psu.edu/harrold92incremental. html
Hoffer, J. A., George, J. F., & Valacich, J. S. (2005). Modern systems analysis and design (4th ed). Upper Saddle River, NJ: Prentice Hall.
Hughes, M., & Stotts, D. (1996). Daistish: Systematic algebraic testing for OO programs in the presence of side-effects. In Proceedings of the ACM SIGSOFT Int. Symp. On Software Testing and Analysis (pp. 53-61). Retrieved June 12, 2005, from http://
rockfish.cs.unc.edu/pubs/issta96.pdf
IBM. (n.d.). Retrieved June 12, 2005, from http://www-106.ibm.com/developerworks/
rational/library/2797.html
Joines, J., & Roberts, S. (1998, December 13-16). Fundamentals of object-oriented simulation. In D. J. Medeiros, E. F. Watson, J. S. Carson, & M. S. Manivannan (Eds.), Proceedings of the 1998 Winter Simulation Conference (pp. 141-149). Los Alamitos, CA: IEEE Computer Society Press.
Kleignen, J., Bettonvil, B., & Gmenendahl, W. (1996). Validation of trace-driven simula-tion models: Regression analysis revisited. In J. M. Ckrnes, D. J. Morrice, D. T.
Brunner, & J. J. Swain (Eds.), Proceedings of the Winter Simulation Conference (pp. 352-359). New York: ACM Press.
Labiche, Y., Thévenod, P., Fosse, H., Waeselynck, M., & Durand, H. (2000). Testing levels for object-oriented software. Proceedings of the 22nd International Confer-ence on Software Engineering, Limerick, Ireland (pp. 136-145). New York: ACM Press.
Leemis, L. (2003 December 7-10). Input modeling. In S. Chick, P. J. Sánchez, D. Ferrin, &
D. J. Morrice (Eds.), Proceedings of the 2003 Winter Simulation Conference, New Orleans, LA (pp. 14-24). Retrieved April 2003, from http://www.informs-sim.org/
wscpapers.html
Lendermann, P., Gan, B., & McGinnis, L. (2001, December 9-12). Distributed simulation with incorporated aps procedures for high-fidelity supply chain optimization. In B.
A. Peters, J. S. Smith, D. J. Medeiros, & M. W. Rohrer (Eds.), Proceedings of the Winter Simulation Conference, Arlington, VA (pp. 805-812). Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Linzhang, W., Jiesong, Y., Xiaofeng, Y., Jun, H., Xuandong, L., & Guoliang, Z. (2004).
Generating test cases from UML activity diagram based on gray-box method.
Apsec, 284-291.
Mansour, N., & Brardhi, B. R. (2001). Empirical comparison of regression test selection algorithm. The Journal of System and Software, 57, 79-90.
Manta. (n.d.). Retrieved June 12, 2005, from http://manta.cs.vt.edu/cs6204
Offutt, J., & Abdurazik, A. (1999, October). Generating tests from UML specifications.
In Second International Conference on the Unified Modeling Language (UML99), Fort Collins, CO.
Orso, A., Shi, N., & Harrold, M. J. (2004, October 31-November 6). Scaling regression testing to large software systems. SIGSOFT’04/FSE-12, Newport Beach, CA (pp.
241-251).
Page, H., & Nance, R. (1997, July). Parallel discrete event simulation: A modeling methodological perspective. ACM Transactions on Modeling and Computer Simulation, 7(3), 88-93.
Paul, R. J., & Balmer D.W. (1998). Simulation modelling. Lund, Sweden: Chartwell-Bratt Student Text Series.
Pezz, M., & Young, M. (2004). Testing object oriented software. In Proceedings of the 26th International Conference on Software Engineering (ICSE’04).
Pidd, M. (1998). Computer simulation in management science (4th ed.). Chichester, UK:
John Wiley & Sons.
Pressman, R. (2005). Software engineering: A practitioner’s approach (6th ed.). New York: McGraw Hill.
Pretschner, A. (2005, May 15-21). Model based testing. In Proceedings of ICSE’05 (pp.722-723). St. Louis, MO: ACM.
Ramakrishnan, S., Lee, S., & Wysk, R. (2002, December 8-11). Implementation of a simulation-based control architecture for supply chain interactions. In E. Yücesan, C. H. Chen, J. L. Snowdon, & J. M. Charnes (Eds.), Proceedings of the Winter Simulation Conference, San Diego, CA. Retrieved April 2003, from http://
www.informs-sim.org/wscpapers.html
Richardson, D. J., O’Malley, O., & Tittle, C. (1989, December). Approaches to specifica-tion-based testing. In Proceedings of ACM SIGSOFT Symposium on Software Testing, Analysis and Verification (pp. 86-96). New York: ACM Press.
Robinson S. (1999, December 10-13). Three sources of simulation inaccuracy (and how to overcome them). In P. A. Farrington, H. B. Nembhard, D. T. Sturrock, & G. W.
Evans (Eds.), Proceedings of the 1999 Winter Simulation Conference, Orlando, FL (pp. 1701-1708).
Sadowsk, R. P. (1991). Avoiding the problems and pitfalls in simulation. Winter Simula-tion Conference Proceedings of the 23rd Conference on Winter simulaSimula-tion (pp.
48-55), Phoenix, AZ. Washington, DC: IEEE Computer Society.
Sargent, R. (1998, December 13-16). Verification and validation of simulation models. In D. J. Medeiros, E. F. Watson, J. S. Carson, & M. S. Manivannan (Eds.), Proceedings of the Winter Simulation Conference, Washington, DC (pp. 121-130). Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Sargent, R. (1999, December 10-13). Validation and verification of simulation models. In P. A. Farrington, H. B. Nembhard, D. T. Sturrock, & G. W. Evans (Eds.), Proceedings of the Winter Simulation Conference, Orlando, FL (pp. 39-48).
Sargent, R. (2003, December 7-10). Verification and validation of simulation models. In S. Chick, P. J. Sánchez, D. Ferrin, & D. J. Morrice (Eds). Proceedings of the Winter Simulation Conference, New Orleans, LA (pp. 39-48). Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Sebesta, R. W. (2003). Concepts of programming languages (6th ed.). Addison-Wesley.
Shannon, R. (1998, December 13-16). Introduction to the art and science of simulation.
In D. J. Medeiros, E. F. Watson, J. S. Carson, & M. S. Manivannan. Proceedings of the Winter Simulation Conference, Washington, DC (pp. 7-14). Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Smith, R. (1998, December 13-16). Essential techniques for military modeling & simula-tion. In D. J. Medeiros, E. F. Watson, J. S. Carson, & M. S. Manivannan. Proceed-ings of the Winter Simulation Conference, Washington, DC (pp. 805-812). Re-trieved April 2003, from http://www.informs-sim.org/wscpapers.html
Sommerville, I. (n.d.). Software engineering (6th ed.). Upper Saddle River, NJ: Addison-Wesley, Pearson Education.
Wikipedia. (n.d.). Retrieved June 12, 2005, from http://en.wikipedia.org/wiki/
Lambda_calculus
Withers, D. (2000, December 10-13). Software engineering best practices applied to the modeling process. In J. A. Joines, R. R. Barton, K. Kang, & P. A. Fishwick (Eds.), Proceedings of the Winter Simulation Conference, Orlando, FL (pp. 432-439).
Retrieved April 2003, from http://www.informs-sim.org/wscpapers.html
Xu, D., Xu, W., & Nygard K. (2005, July 14-16). A state-based approach to testing aspect-oriented programs. In Proceedings of the 17th International Conference on Software Engineering and Knowledge Engineering (SEKE’05), Taiwan. Re-trieved April 2004, from http://cs.ndsu.edu/~dxu/publications/SEKE05-xu.pdf Yilmaz, L. (2001). Automated object-flow testing of dynamic process interaction models.
In B. A. Peters, J. S. Smith, D. J. Medeiros, & M. W. Rohrer (Eds.), Proceedings of the Winter Simulation Conference (pp. 586-594). Retrieved April 2003, from http:/
/www.informs-sim.org/wscpapers.html
Zammit, J. (2005). Correct cystem, Web site for information systems engineering.
University of Malta. Retrieved June, 18, 2005, from http://www.cis.um.edu.mt/
~jzam/vv.html