Quality Assurance in Software Development
51
0
0
Full text
(2) Institute for Software Technology. t. Agenda. I. Mutation Testing. I. Model-based Mutation Testing. I. Test-Case Generation with ioco checking. B. Aichernig. Quality Assurance in Software Development 2 / 19.
(3) Institute for Software Technology. t. Some Bugs Hide for a Long Time! Binary search bug in Java I. JDK 1.5 library (2006). I. out of boundary access of large arrays. I. due to integer overflow. I. 9 years undetected. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19. public static int binarySearch ( int [] a , int key ) { int low = 0; int high = a . length - 1; while ( low <= high ) { int mid = ( low + high ) / 2; int midVal = a [ mid ]; if ( midVal < key ) low = mid + 1; else if ( midVal > key ) high = mid - 1; else return mid ; // key found } return -( low + 1); // key not found }. “Beware of bugs in the above code; I have only proved it correct, not tried it.” [Knuth77] B. Aichernig. Quality Assurance in Software Development 3 / 19.
(4) Institute for Software Technology. t. Some Bugs Hide for a Long Time! Binary search bug in Java I. JDK 1.5 library (2006). I. out of boundary access of large arrays. I. due to integer overflow. I. 9 years undetected. Algorithm was proven correct! I. Programming Pearls [Bentley86, Bentley00]. I. assuming infinite integers :(. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19. public static int binarySearch ( int [] a , int key ) { int low = 0; int high = a . length - 1; while ( low <= high ) { int mid = ( low + high ) / 2; int midVal = a [ mid ]; if ( midVal < key ) low = mid + 1; else if ( midVal > key ) high = mid - 1; else return mid ; // key found } return -( low + 1); // key not found }. “Beware of bugs in the above code; I have only proved it correct, not tried it.” [Knuth77] B. Aichernig. Quality Assurance in Software Development 3 / 19.
(5) Institute for Software Technology. t. Some Bugs Hide for a Long Time! Binary search bug in Java I. JDK 1.5 library (2006). I. out of boundary access of large arrays. I. due to integer overflow. I. 9 years undetected. Algorithm was proven correct! I. Programming Pearls [Bentley86, Bentley00]. I. assuming infinite integers :(. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19. public static int binarySearch ( int [] a , int key ) { int low = 0; int high = a . length - 1; while ( low <= high ) { int mid = ( low + high ) / 2; int midVal = a [ mid ]; if ( midVal < key ) low = mid + 1; else if ( midVal > key ) high = mid - 1; else return mid ; // key found } return -( low + 1); // key not found }. “Beware of bugs in the above code; I have only proved it correct, not tried it.” [Knuth77] B. Aichernig. Quality Assurance in Software Development 3 / 19.
(6) Institute for Software Technology. t. Some Bugs Hide for a Long Time! Binary search bug in Java I. JDK 1.5 library (2006). I. out of boundary access of large arrays. I. due to integer overflow. I. 9 years undetected. Algorithm was proven correct! I. Programming Pearls [Bentley86, Bentley00]. I. assuming infinite integers :(. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19. public static int binarySearch ( int [] a , int key ) { int low = 0; int high = a . length - 1; while ( low <= high ) { int mid = ( low + high ) >>> 1; int midVal = a [ mid ]; if ( midVal < key ) low = mid + 1; else if ( midVal > key ) high = mid - 1; else return mid ; // key found } return -( low + 1); // key not found }. “Beware of bugs in the above code; I have only proved it correct, not tried it.” [Knuth77] B. Aichernig. Quality Assurance in Software Development 3 / 19.
(7) Institute for Software Technology. t. Observations I I. Verification failed (wrong assumption) Established testing strategies failed: I I I I. I. statement coverage branch coverage fails multiple condition coverage MC/DC: standard in avionics [DO-178B/ED109]. Long array needed: int[] a = new int[Integer.MAX_VALUE/2+2]. Lesson I Concentrate on possible faults, not on structure. I. Generate test cases covering these faults. I. Mutation Testing [Lipton71, Hamlet77, DeMillo et al.78]. B. Aichernig. Quality Assurance in Software Development 4 / 19.
(8) Institute for Software Technology. t. Observations I I. Verification failed (wrong assumption) Established testing strategies failed: I I I I. I. statement coverage branch coverage fails multiple condition coverage MC/DC: standard in avionics [DO-178B/ED109]. Long array needed: int[] a = new int[Integer.MAX_VALUE/2+2]. Lesson I Concentrate on possible faults, not on structure. I. Generate test cases covering these faults. I. Mutation Testing [Lipton71, Hamlet77, DeMillo et al.78]. B. Aichernig. Quality Assurance in Software Development 4 / 19.
(9) Institute for Software Technology. t. What Is Mutation Testing?. Originally: Technique to verify the quality of test cases. “There is a pressing need to address the, currently unresolved, problem of test case generation.” [Jia&Harman11]. B. Aichernig. Quality Assurance in Software Development 5 / 19.
(10) Institute for Software Technology. t. What Is Mutation Testing?. Originally: Technique to verify the quality of test cases. “There is a pressing need to address the, currently unresolved, problem of test case generation.” [Jia&Harman11]. B. Aichernig. Quality Assurance in Software Development 5 / 19.
(11) Institute for Software Technology. t. How Does It Work? Step 1: Create mutants. Mutation Process. Mutant. Source Code. Mutation Operator. B. Aichernig. Quality Assurance in Software Development 6 / 19.
(12) Institute for Software Technology. t. Example: Scala Program. I. Kind of triangles: I I I. I. equilateral isosceles scalene. Create mutants I. I. mutation operator == ⇒ >= creates 5 mutants. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. object triangle { def tritype ( a : Int , b : Int , c : Int ) = (a ,b , c ) match { case _ if ( a <= c - b ) = > " no triangle " case _ if ( a <= b - c ) = > " no triangle " case _ if ( b <= a - c ) = > " no triangle " case _ if ( a == b && b == c ) = > " equilateral " case _ if ( a == b ) = > " isosceles " case _ if ( b == c ) = > " isosceles " case _ if ( a == c ) = > " isosceles " case _ = > " scalene " } }. Source code in Scala. B. Aichernig. Quality Assurance in Software Development 7 / 19.
(13) Institute for Software Technology. t. Example: Scala Program. I. Kind of triangles: I I I. I. equilateral isosceles scalene. Create mutants I. I. mutation operator == ⇒ >= creates 5 mutants. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. object triangle { def tritype ( a : Int , b : Int , c : Int ) = (a ,b , c ) match { case _ if ( a <= c - b ) = > " no triangle " case _ if ( a <= b - c ) = > " no triangle " case _ if ( b <= a - c ) = > " no triangle " case _ if ( a >= b && b == c ) = > " equilateral " case _ if ( a == b ) = > " isosceles " case _ if ( b == c ) = > " isosceles " case _ if ( a == c ) = > " isosceles " case _ = > " scalene " } }. Mutant. B. Aichernig. Quality Assurance in Software Development 7 / 19.
(14) Institute for Software Technology. t. Example: UML Model. I. I I I. I. AlarmSystem_StateMachine. Car Alarm System. Unlock. OpenAndUnlocked. event-based controllable events observable events. Open. Close. ClosedAndUnlocked. Mutate the model. Unlock. Lock. Lock. Alarm Unlock. OpenAndLocked Close. Activate Alarms /entry Deactivate Alarms /exit FlashAndSound. Open 30 / Deactivate Sound. I. I. mutation operator ⇒ 17 mutants. ClosedAndLocked Flash 20 Close. Armed. SilentAndOpen. Unlock. Show Armed /entry Show Unarmed /exit. 300. Open. State machine model in UML. B. Aichernig. Quality Assurance in Software Development 8 / 19.
(15) Institute for Software Technology. t. Example: UML Model. I. I I I. I.
(16) . Car Alarm System. . . event-based controllable events observable events.
(17) .
(18). . Mutate the model. . . .
(19).
(20)
(21) .
(22)
(23)
(24) . I. I. mutation operator ⇒ 17 mutants. . .
(25). . . . .
(26) . Mutated UML model. B. Aichernig. Quality Assurance in Software Development 8 / 19.
(27) Institute for Software Technology. t. How Does It Work? Step 2: Try to kill mutants. A test case kills a mutant if its run shows different behaviour.. B. Aichernig. Quality Assurance in Software Development 9 / 19.
(28) Institute for Software Technology. t. Example: Scala Program. I. I. Mutant survives path coverage (MC/DC): tritype(0,1,1) tritype(1,0,1) tritype(1,1,0) tritype(1,1,1) tritype(2,3,3) tritype(3,2,3) tritype(3,3,2) tritype(2,3,4) Mutant killed by tritype(3,2,2). B. Aichernig. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. object triangle { def tritype ( a : Int , b : Int , c : Int ) = (a ,b , c ) match { case _ if ( a <= c - b ) = > " no triangle " case _ if ( a <= b - c ) = > " no triangle " case _ if ( b <= a - c ) = > " no triangle " case _ if ( a >= b && b == c ) = > " equilateral " case _ if ( a == b ) = > " isosceles " case _ if ( b == c ) = > " isosceles " case _ if ( a == c ) = > " isosceles " case _ = > " scalene " } }. Mutant. Quality Assurance in Software Development 10 / 19.
(29) Institute for Software Technology. t. Example: Scala Program. I. I. Mutant survives path coverage (MC/DC): tritype(0,1,1) tritype(1,0,1) tritype(1,1,0) tritype(1,1,1) tritype(2,3,3) tritype(3,2,3) tritype(3,3,2) tritype(2,3,4) Mutant killed by tritype(3,2,2). B. Aichernig. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. object triangle { def tritype ( a : Int , b : Int , c : Int ) = (a ,b , c ) match { case _ if ( a <= c - b ) = > " no triangle " case _ if ( a <= b - c ) = > " no triangle " case _ if ( b <= a - c ) = > " no triangle " case _ if ( a >= b && b == c ) = > " equilateral " case _ if ( a == b ) = > " isosceles " case _ if ( b == c ) = > " isosceles " case _ if ( a == c ) = > " isosceles " case _ = > " scalene " } }. Mutant. Quality Assurance in Software Development 10 / 19.
(30) Institute for Software Technology. t. Example: UML Model. I. I I I. I.
(31) . Mutant survives. Killed by Lock(); Close(); After(20);. . . function coverage state coverage transition coverage.
(32) .
(33). . . . .
(34).
(35)
(36) .
(37)
(38)
(39) . . .
(40). . . . .
(41) . Mutated UML model. B. Aichernig. Quality Assurance in Software Development 11 / 19.
(42) Institute for Software Technology. t. Example: UML Model. I. I I I. I.
(43) . Mutant survives. Killed by Lock(); Close(); After(20);. . . function coverage state coverage transition coverage.
(44) .
(45). . . . .
(46).
(47)
(48) .
(49)
(50)
(51) . . .
(52). . . . .
(53) . Mutated UML model. B. Aichernig. Quality Assurance in Software Development 11 / 19.
(54) Institute for Software Technology. t. From Analysis to Synthesis State of art:. Research:. Analysis of test cases. Synthesis of test cases. How many mutants killed by test cases?. Find test cases that maximise mutation score.. mutation score =. #killed mutants #mutants. Idea: I. Check equivalence between original and mutant. I. Use counter-example as test case.. Problem: equivalent mutants. Problem: equivalence checking is hard (undecidable in general). Solution: review of surviving mutants. Solution: generate from models (abstraction) → model-based mutation testing. B. Aichernig. Quality Assurance in Software Development 12 / 19.
(55) Institute for Software Technology. t. From Analysis to Synthesis State of art:. Research:. Analysis of test cases. Synthesis of test cases. How many mutants killed by test cases?. Find test cases that maximise mutation score.. mutation score =. #killed mutants #mutants. Idea: I. Check equivalence between original and mutant. I. Use counter-example as test case.. Problem: equivalent mutants. Problem: equivalence checking is hard (undecidable in general). Solution: review of surviving mutants. Solution: generate from models (abstraction) → model-based mutation testing. B. Aichernig. Quality Assurance in Software Development 12 / 19.
(56) Institute for Software Technology. t. From Analysis to Synthesis State of art:. Research:. Analysis of test cases. Synthesis of test cases. How many mutants killed by test cases?. Find test cases that maximise mutation score.. mutation score =. #killed mutants #mutants. Idea: I. Check equivalence between original and mutant. I. Use counter-example as test case.. Problem: equivalent mutants. Problem: equivalence checking is hard (undecidable in general). Solution: review of surviving mutants. Solution: generate from models (abstraction) → model-based mutation testing. B. Aichernig. Quality Assurance in Software Development 12 / 19.
(57) Institute for Software Technology. t. From Analysis to Synthesis State of art:. Research:. Analysis of test cases. Synthesis of test cases. How many mutants killed by test cases?. Find test cases that maximise mutation score.. mutation score =. #killed mutants #mutants. Idea: I. Check equivalence between original and mutant. I. Use counter-example as test case.. Problem: equivalent mutants. Problem: equivalence checking is hard (undecidable in general). Solution: review of surviving mutants. Solution: generate from models (abstraction) → model-based mutation testing. B. Aichernig. Quality Assurance in Software Development 12 / 19.
(58) Institute for Software Technology. t. Model-Based Testing. Test Case Generator. SUT. B. Aichernig. Test Driver. Quality Assurance in Software Development 13 / 19.
(59) Institute for Software Technology. t. Model-Based Testing. Model. Test Case Generator. SUT. B. Aichernig. Test Driver. Quality Assurance in Software Development 13 / 19.
(60) Institute for Software Technology. t. Model-Based Testing. Model. Test Case Generator. Abstract Test Case. SUT. B. Aichernig. Test Driver. Quality Assurance in Software Development 13 / 19.
(61) Institute for Software Technology. t. Model-Based Testing. Model. Test Case Generator. Abstract Test Case. SUT. B. Aichernig. Test Driver. pass / fail. Quality Assurance in Software Development 13 / 19.
(62) Institute for Software Technology. t. Model-Based Testing. Model. Test Case Generator. if conforms Abstract Test Case. SUT. B. Aichernig. Test Driver. then pass. Quality Assurance in Software Development 13 / 19.
(63) Institute for Software Technology. t. Model-Based Testing. Model. Test Case Generator. if ¬conforms Abstract Test Case. SUT. B. Aichernig. Test Driver. then pass/fail. Quality Assurance in Software Development 13 / 19.
(64) Institute for Software Technology. t. Model-Based Mutation Testing. Model. Mutation Tool. Test Case Generator. Abstract Test Case. SUT. B. Aichernig. Test Driver. Quality Assurance in Software Development 13 / 19.
(65) Institute for Software Technology. t. Model-Based Mutation Testing. Model. Mutation Tool. Model Mutant. Test Case Generator. Abstract Test Case. SUT. B. Aichernig. Test Driver. Quality Assurance in Software Development 13 / 19.
(66) Institute for Software Technology. t. Model-Based Mutation Testing. Model. Mutation Tool. Model Mutant. Test Case Generator. if ¬conforms Abstract Test Case. SUT. B. Aichernig. Test Driver. then pass/fail. Quality Assurance in Software Development 13 / 19.
(67) Institute for Software Technology. t. Model-Based Mutation Testing. Model. Mutation Tool. Model Mutant. Test Case Generator. if ¬conforms Abstract Test Case. if conforms. SUT. B. Aichernig. Test Driver. then fail. Quality Assurance in Software Development 13 / 19.
(68) Institute for Software Technology. t. Model-Based Mutation Testing then ¬ conforms Model. Mutation Tool. Model Mutant. Test Case Generator. if ¬conforms Abstract Test Case. if conforms. SUT. B. Aichernig. Test Driver. then fail. Quality Assurance in Software Development 13 / 19.
(69) Institute for Software Technology. t. Reactive Systems AlarmSystem_StateMachine. ClosedAndUnlocked. Servers and Controllers. Unlock. Unlock. FlashAndSound. Close. Lock. Open 30 / Deactivate Sound. Events: controllable and observable communication events. ClosedAndLocked Flash 20 Close. SilentAndOpen. Armed. 0. 4. obs AlarmArmed_SetOn. 5. Open. ctr Open. 6. obs AlarmArmed_SetOff. 7. obs OpticalAlarm_SetOn. 8. obs AcousticAlarm_SetOn. 9. Show Armed /entry Show Unarmed /exit. obs after(30). 10. 300. Unlock. obs AcousticAlarm_SetOff. 12. obs after(270). 13. obs AcousticAlarm_SetOff. 14. obs OpticalAlarm_SetOff. 15. ctr Close. 16. obs AlarmArmed_SetOn. 11. ctr Unlock. 17. obs pass. Test cases: sequences of events. obs AlarmArmed_SetOff. I. Activate Alarms /entry Deactivate Alarms /exit. OpenAndLocked. 1. I. Unlock. Alarm. Lock. Close. ctr Close. I. Open. 2. Do not terminate. ctr Lock. I. OpenAndUnlocked. 3. React to the environment. obs after(20). I. Adaptive test cases: trees branching at non-deterministic observations. B. Aichernig. Quality Assurance in Software Development 14 / 19.
(70) Institute for Software Technology. t. Semantics I. 10. Operational semantics e.g. Labelled Transition Systems. ctr Unlock. 11 obs AcousticAlarm_SetOff. ctr Unlock. 2 obs after (c_waittime: 30 ). I. Input-output conformance (ioco) [Tretmans96]. 8. obs after (c_waittime: 270 ). obs AcousticAlarm_SetOn 7 obs OpticalAlarm_SetOn. SUT ioco Model =df. 4. 15 obs AlarmArmed_SetOff. ∀σ ∈ traces(Model) :. obs AcousticAlarm_SetOff. 14. obs AcousticAlarm_SetOff. out(SUT after σ) ⊆ out(Model after σ). ctr Open 13. ctr Unlock. out ... outputs + quiescence after ... reachable states after trace. 1. obs AlarmArmed_SetOn 12. 5. obs OpticalAlarm_SetOff. obs AlarmArmed_SetOff. obs after (c_waittime: 20 ) 16. ctr Lock. ctr Close ctr Open. ctr Unlock 6. ctr Lock. ctr Close 9. B. Aichernig. ctr Close 17. ctr Open obs OpticalAlarm_SetOff. ctr Unlock 0. ctr Unlock. 3. Quality Assurance in Software Development 15 / 19.
(71) Institute for Software Technology. t. Semantics I. 10. Operational semantics e.g. Labelled Transition Systems. ctr Unlock. 11 obs AcousticAlarm_SetOff. ctr Unlock. 2 obs after (c_waittime: 30 ). I. Input-output conformance (ioco) [Tretmans96]. 8. obs after (c_waittime: 270 ). obs AcousticAlarm_SetOn 7 obs OpticalAlarm_SetOn. SUT ioco Model =df. 4. 15 obs AlarmArmed_SetOff. ∀σ ∈ traces(Model) :. obs AcousticAlarm_SetOff. 14. obs AcousticAlarm_SetOff. out(SUT after σ) ⊆ out(Model after σ). ctr Open 13. ctr Unlock. out ... outputs + quiescence after ... reachable states after trace. 1. obs AlarmArmed_SetOn 12. 5. obs OpticalAlarm_SetOff. obs AlarmArmed_SetOff. obs after (c_waittime: 20 ) 16. ctr Lock. ctr Close ctr Open. ctr Unlock 6. ctr Lock. ctr Close 9. B. Aichernig. ctr Close 17. ctr Open obs OpticalAlarm_SetOff. ctr Unlock 0. ctr Unlock. 3. Quality Assurance in Software Development 15 / 19.
(72) Institute for Software Technology. t. Semantics I. Operational semantics e.g. Labelled Transition Systems. I. Input-output conformance (ioco) [Tretmans96]. Model: !soundOn !flashOn. !soundOn !flashOn. SUT ioco Model =df. SUT:. ∀σ ∈ traces(Model) : out(SUT after σ) ⊆ out(Model after σ) out ... outputs + quiescence after ... reachable states after trace. !soundOn !flashOn. ?unlock. SUT ioco Model. B. Aichernig. X. Quality Assurance in Software Development 15 / 19.
(73) Institute for Software Technology. t. Explicit Conformance Checking I. Model and Mutant → LTS. I. Determinisation. Model:. I. Build synchronous product modulo ioco. I. If mutant has additional I. !soundOn. I. !flashOn. !output: → fail sink state ?input: → pass sink state. Model ×ioco Mutant: !soundOn. !soundOn !flashOn !flashOn. !soundOff. Mutant: !soundOff. !soundOn pass. !flashOn. pass. fail. ?unlock pass. ?unlock. B. Aichernig. I. Extract test case covering fail state Quality Assurance in Software Development 16 / 19.
(74) Institute for Software Technology. t. Explicit Conformance Checking I. Model and Mutant → LTS. I. Determinisation. Model:. I. Build synchronous product modulo ioco. I. If mutant has additional I. !soundOn. I. !flashOn. !output: → fail sink state ?input: → pass sink state. Model ×ioco Mutant: !soundOn. !soundOn !flashOn !flashOn. !soundOff. Mutant: !soundOff. !soundOn pass. !flashOn. pass. fail. ?unlock pass. ?unlock. B. Aichernig. I. Extract test case covering fail state Quality Assurance in Software Development 16 / 19.
(75) Institute for Software Technology. t. Explicit Conformance Checking I. Model and Mutant → LTS. I. Determinisation. Model:. I. Build synchronous product modulo ioco. I. If mutant has additional I. !soundOn. I. !flashOn. !output: → fail sink state ?input: → pass sink state. Model ×ioco Mutant: !soundOn. !soundOn !flashOn !flashOn. !soundOff. Mutant: !soundOff. !soundOn pass. !flashOn. pass. fail. ?unlock pass. ?unlock. B. Aichernig. I. Extract test case covering fail state Quality Assurance in Software Development 16 / 19.
(76) Institute for Software Technology. t. Applications of Explicit Conformance Checking I. HTTP Server (LOTOS). I. SIP Server (LOTOS). I. Controllers (UML). I. Hybrid Systems (Action System). Scalability: abstractions for data-intensive models. Bernhard K. Aichernig and Corrales Delgado. From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems, FASE 2006.. Martin Weiglhofer, Bernhard K. Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2-3):375-411, 2009. Chinese Academy of Science.. Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, and Willibald Krenn. Efficient mutation killers in action, ICST 2011.. Harald Brandl, Martin Weiglhofer, and Bernhard K. Aichernig. Automated conformance verification of hybrid systems, QSIC 2010. B. Aichernig. Quality Assurance in Software Development 17 / 19.
(77) Institute for Software Technology. t. Applications of Explicit Conformance Checking I. HTTP Server (LOTOS). I. SIP Server (LOTOS). I. Controllers (UML). I. Hybrid Systems (Action System). Scalability: abstractions for data-intensive models. Bernhard K. Aichernig and Corrales Delgado. From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems, FASE 2006.. Martin Weiglhofer, Bernhard K. Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2-3):375-411, 2009. Chinese Academy of Science.. Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, and Willibald Krenn. Efficient mutation killers in action, ICST 2011.. Harald Brandl, Martin Weiglhofer, and Bernhard K. Aichernig. Automated conformance verification of hybrid systems, QSIC 2010. B. Aichernig. Quality Assurance in Software Development 17 / 19.
(78) Institute for Software Technology. t. Applications of Explicit Conformance Checking I. HTTP Server (LOTOS). I. SIP Server (LOTOS). I. Controllers (UML). I. Hybrid Systems (Action System). Scalability: abstractions for data-intensive models. Bernhard K. Aichernig and Corrales Delgado. From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems, FASE 2006.. Martin Weiglhofer, Bernhard K. Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2-3):375-411, 2009. Chinese Academy of Science.. Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, and Willibald Krenn. Efficient mutation killers in action, ICST 2011.. Harald Brandl, Martin Weiglhofer, and Bernhard K. Aichernig. Automated conformance verification of hybrid systems, QSIC 2010. B. Aichernig. Quality Assurance in Software Development 17 / 19.
(79) Institute for Software Technology. t. Applications of Explicit Conformance Checking I. HTTP Server (LOTOS). I. SIP Server (LOTOS). I. Controllers (UML). I. Hybrid Systems (Action System). Scalability: abstractions for data-intensive models. Bernhard K. Aichernig and Corrales Delgado. From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems, FASE 2006.. Martin Weiglhofer, Bernhard K. Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2-3):375-411, 2009. Chinese Academy of Science.. Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, and Willibald Krenn. Efficient mutation killers in action, ICST 2011.. Harald Brandl, Martin Weiglhofer, and Bernhard K. Aichernig. Automated conformance verification of hybrid systems, QSIC 2010. B. Aichernig. Quality Assurance in Software Development 17 / 19.
(80) Institute for Software Technology. t. Applications of Explicit Conformance Checking I. HTTP Server (LOTOS). I. SIP Server (LOTOS). I. Controllers (UML). I. Hybrid Systems (Action System). Scalability: abstractions for data-intensive models. Bernhard K. Aichernig and Corrales Delgado. From Faults via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems, FASE 2006.. Martin Weiglhofer, Bernhard K. Aichernig, and Franz Wotawa. Fault-based conformance testing in practice. International Journal of Software and Informatics, 3(2-3):375-411, 2009. Chinese Academy of Science.. Bernhard K. Aichernig, Harald Brandl, Elisabeth Jöbstl, and Willibald Krenn. Efficient mutation killers in action, ICST 2011.. Harald Brandl, Martin Weiglhofer, and Bernhard K. Aichernig. Automated conformance verification of hybrid systems, QSIC 2010. B. Aichernig. Quality Assurance in Software Development 17 / 19.
(81) Institute for Software Technology. t. Agile Development 6$1*7+")& 345%8&. !"#$%&. '$($)*+$&& ,$-+&.*-$-&. 345%$4$(+& ,$-+&.*-$-&. /$)012&,$-+& .*-$-&. I. Model-driven development. I. Formal verification. I. Model-based test case generation. I. Test-driven development. B. Aichernig. Quality Assurance in Software Development 18 / 19.
(82) Institute for Software Technology. t. Summary. I. Model-based Testing + Mutation Testing. I. Test case generation via ioco check. I. Industrial applications in EU projects MOGENTES, MBAT, CRYSTAL Testing cannot show the absence of bugs [Dijkstra72]. Testing can show the absence of specific bugs [Aichernig12].. B. Aichernig. Quality Assurance in Software Development 19 / 19.
(83) Institute for Software Technology. t. Summary. I. Model-based Testing + Mutation Testing. I. Test case generation via ioco check. I. Industrial applications in EU projects MOGENTES, MBAT, CRYSTAL Testing cannot show the absence of bugs [Dijkstra72]. Testing can show the absence of specific bugs [Aichernig12].. B. Aichernig. Quality Assurance in Software Development 19 / 19.
(84) Institute for Software Technology. t. Summary. I. Model-based Testing + Mutation Testing. I. Test case generation via ioco check. I. Industrial applications in EU projects MOGENTES, MBAT, CRYSTAL Testing cannot show the absence of bugs [Dijkstra72]. Testing can show the absence of specific bugs [Aichernig12].. B. Aichernig. Quality Assurance in Software Development 19 / 19.
(85)
Related documents
Starting with the case of L = 8 projections, the table demon- strates that, for all GEMM computations, and under the same recognition accuracy as the conventional (non
Equally you might find that the ownership issue of social customer service between Marketing, PR and Customer Services provokes a broader debate about collaboration using the rich
A proposta do Interacionismo Sociodiscursivo para o ensino de línguas estrangeiras propõe que os conteúdos linguísticos-discursivos devem ser trabalhados a partir
It introduces a new approach to handling these outliers using a method based on robust statistics: namely, the median multitaper method for power spectral estimation.. It
HMI Operator Panel: Setting the Interface for the Project Transfer Transfer Start Control Panel Transfer Settings Channel Directories X OK Advanced Enable Channel
A similar analysis was presented in Hutchinson, Wright, and Mi- lan ( 2011 ) for Solar Cycle 23 but concluded that storm main phase duration increases with intensity to a point,
The markup <html> Starts the document <head> Opens the header </head> Closes the header <body> Opens the body </body> Closes the body
that you are using is successful. If it’s not then, actually you should go back and to find some other way to do some lesson, but it’s also to find out what the kids
