Structured Testing in VDM
Kenneth Lausdahl and Peter Gorm Larsen
May 2011
1
Introduction
The Vienna Development Method was initially developed with proof of property in mind. However a number of test principles known for traditional software development is also supported by VDM such as unit-, integration- and system-test. Tool support exists for Unit testing and Combinatorial Testing, the latter is a part of the language where regular expressions can be used to bring a system into a specific state and from there generate tests automatically to check for violation of invariants, pre-, and post-conditions in the VDM model.
The exercise is divided into three assignments:
1. Specify a number of functions to be tested.
2. Create Unit tests for all functions. Include positive and negative tests.
3. Write a trace for the provided Treemodel and see if the trace can detect any errors.
2
VDM Unit testing
Unit testing is primary used to test small parts of a model such as a single function/-operation or class. VDM-Unit is a port of the well known JUnit for Java, it enabled a structured test through TestCases and TestSuites. The VDM Unit framework is de-scribed in more detail in chapterThe VDMUnit Framework of [1], The VDM-Unit library provided for this exercise is a newer version than presented in [1], thus naming and the internal class structure is different.
2.1
Specify functions to be tested
1. Write a function that can reverse a sequence of integers.
2. Write a recursive power function for integers: xne.g.210= 1024. (Remember
thatx0= 1and thatx(n+1)=x∗xn)
3. Write a function which creates an “onion” structure. An onion is build of a small onion with a onion around thus creating a layers structure. In VDM we can see that as an empty list as the middle (inner onion) a list of numbers as the outer layers such that:
makeOnion([1,2,3,4]) =[1,[2,[3,[4,[],4],3],2],1]
4. Write a function which creates an “onion dual” structure:
makeDualOnion([1,2,3,4]) =[4,[3,[2,[1,[],1],2],3],4]
-- Use language version = vdm10
functions
public reverse : seq of int -> seq of int reverse(list) == is not yet specified; public power : int * int -> int power(x,n) == is not yet specified; types
public T = nat1 | seq of T; functions
public makeOnion : seq of int -> seq of T makeOnion(list)== is not yet specified; public makeDualOnion : seq of int -> seq of T makeDualOnion(list)== is not yet specified; public static add : nat * nat -> nat add(a,b) == a+b;
Listing 1: Signatures for the functions to test.
2.2
Create Unit tests
Now develop test for each of the new functions. Produce at least one positive and one negative test of each function. So if we had anaddfunction it could look like:
class TestCaseAdd1 is subclass of TestCase operations
runTest()== (
assertTrue(Functions‘add(2,3)=5); );
end TestCaseAdd1
class TestCaseAdd2 is subclass of TestCase operations protected runTest : () ==>() runTest()== ( assertFalse(Functions‘add(5,5)=11); ) end TestCaseAdd2
Listing 2: Example of a TestCase for the add function.
TheTestCase’s can then easily be executed trough aTestSuiteas shown in list-ing 3 (Remember to include theIOlibrary).
let ts : TestSuite = new TestSuite(), result = new TestResult()
in ( ts.addTest(new TestCaseAdd1()); ts.addTest(new TestCaseAdd2()); ts.run(result); IO‘println(result.toString()); )
Listing 3: RunningTestCases through aTestSuiteand printing the result.
2.2.1 Simplified test structure
From Overture IDE version 1.0.1 it is possible to run Unit tests in a similar way as JU-nit, where the VDM interpreter is able to search for test operations within aTestCase
or a sub class here of, and run all of tests found. This is possible by instantiating a
TestSuitewith a single or a set of classes which are sub classes ofTestCase. class TestCaseFunctions operations protected testAdd235 : () ==>() testAdd235()== ( assertTrue(Functions‘add(2,3)=5); );
protected testAdd5511 : () ==>() testAdd5511()== ( assertFalse(Functions‘add(5,5)=11); ) end TestCaseFunctions
Listing 4: A singleTestCasewhere all operations prefixed “test” will be extracted and executed.
let ts : TestSuite = new TestSuite(new TestCaseFunctions()), result = new TestResult()
in ( ts.run(result); IO‘println(result.toString()); )
Listing 5: Running all operations prefixed “test” in TestCases through a
TestSuiteand printing the result.
3
Combinatorial Testing in VDM
Combinatorial Testing is a good alternative to normal Unit testing for VDM mod-els. Since formally specified VDM models often are accompanied by pre- and post-conditions and also invariants the allowed behaviour of the VDM model is restricted. This can be utilised through combinatorial testing since tests can be generated and vali-dated against these invariants, pre- and post-conditions. Combinatorial testing in VDM is build to allow a modeller to get a VDM model into a particular state and then specify a scenario by the use of constructs such asalternativesandrepetitionwhich then can be expanded to all possible combinations. This will in most cases generate a large number of tests even for small test traces.
Exercise:ATreeVDM model is on purpose provided with at least one error. Specify a trace which uses the different search mechanisms of the tree. Through this identify existing errors. When a trace test fails then try to send it to the interpreter and debug the problem and fix the error(s). So your tasks can be summarised as:
1. Extend theUseTreeclass with a more advanced trace definition. 2. Try out theCombinatorial Testingperspective (CT)1.
3. Try to run a full test.
4. Try to see of any of the reduced options is enough to detect the problem. (Faster executions.)
5. Try to send a test case to the interpreter. (When doing so this should be similar to running a manually specified Unit test.
6. Fix the errors you might discover.
References
[1] John Fitzgerald, Peter Gorm Larsen, Paul Mukherjee, Nico Plat, and Marcel Ver-hoef.Validated Designs for Object–oriented Systems. Springer, New York, 2005.