Scenario Execution

As discussed for the dependency graph creation (Chapter 4), the dy- namic nature of web applications means that the scenario generation process must be based on dynamic rather than static analysis. For this reason, we execute and analyze each scenario. The scenario execution is performed with our custom-made browser simulator (Chapter 8) which has a JavaScript interpreter capable of performing both concrete and symbolic execution of web applications, and building a dependency graph (Chapter 4). On top of this, the browser simulator can be instantiated with different internal browser states, in that way mimicking different browsers.

Example. The first scenario being executed is the scenario that con-

tains only the default loading event: so=e0d, where e0d is the default

loading event. In the example from Listing 5.1, this means constructing the DOM of the page (based on lines 1 – 9) and executing JavaScript statements in lines 10 – 14, 26. At this point, no user-generated events are executed. The loading of the example page does not depend on the internal browser state. After the page is loaded, the browser is aware of

52 Chapter 5. Automatic Scenario Generation

The UI of the application is composed of two squares, one corre- sponding to the div element with the ID fc, and the other to the div element with the ID sc, line 8, Listing 5.1. The example application has two features: Feature 1, that manifests on the first square (the HTML node with ID fc, line 8), which consists of two behaviors: i) when the user clicks on the square with the left mouse button, the application subscribes to the mouse move events which change the color of the first square background depending on the position of the mouse, ii) it counts the number of middle mouse button clicks on the first square, and out- puts whether this number is even or odd; and Feature 2, that manifests on the second square (the HTML node with ID sc, line 8), with the fol- lowing behavior: i) when the user clicks on the second square it outputs the current mouse position. This is an example of an event-driven ap- plication where code coverage depends both on the events raised by the user, and the properties of the raised events (e.g. which mouse button was clicked). Throughout this section, we will show how the process generates scenarios that target the first feature.

5.2.2

Selecting Scenarios

Our method creates new scenarios by systematically exploring the event and value space of the application. This means that the number of gen- erated scenarios grows considerably with application complexity, and the procedure by which the next scenario for analysis is picked could influ- ence how fast the scenario generation method achieves good coverage. For this reason, we have considered several prioritization functions that determine the order in which the scenarios will be analyzed. The pri- oritization functions are the following: i) FIFO (First In, First Out), where scenarios are picked based on their order of creation; ii) Random, where the next scenario is randomly picked from the pool of non ana- lyzed scenarios; iii) Event Length, where the next scenario for analysis is the scenario with the shortest sequence of events. We also use two more complex prioritization functions: iv) Coverage and a v) Custom prioritization function.

The Coverage prioritization function is based on the intuition that executing scenarios with events that have already achieved high code coverage is likely to be less useful than executing scenarios with events with low coverage [5]. After the execution of every scenario, for every

function visited during the evaluation of each event ei, we recalculate

5.2 Detailed process description 53

the coverage achieved so far. Based on this coverage, each scenario is assigned a weight (or a priority). We use the following formula [5]:

P (si) = 1− cov(e0)· cov(e1)· ... · cov(em),

where si = e0, e1, ..., em is the scenario whose weight is being de-

termined, e0, e1, ..., emthe event chain of si, and cov (ej) a function that

calculates statement coverage of all functions visited during the execu- tion of an event achieved so far by the whole process (i.e. by all previ- ously analyzed scenarios) in all functions executed during the handling

of event ej. The next scenario for analysis is then chosen with weighted

random selection.

The Custom prioritization function is based on the following intu- ition: if there is an unexecuted scenario created by exploring the value space (i.e. modifying the input parameters), or a scenario whose last event has not yet been executed in the scenario generation process, then the process selects such a scenario based on the order of creation. If there are no such scenarios, i.e. only the scenarios created by extending the event chain with events already executed in the process, then select the next scenario by using the Coverage prioritization function.

5.2.3

Scenario Execution

As discussed for the dependency graph creation (Chapter 4), the dy- namic nature of web applications means that the scenario generation process must be based on dynamic rather than static analysis. For this reason, we execute and analyze each scenario. The scenario execution is performed with our custom-made browser simulator (Chapter 8) which has a JavaScript interpreter capable of performing both concrete and symbolic execution of web applications, and building a dependency graph (Chapter 4). On top of this, the browser simulator can be instantiated with different internal browser states, in that way mimicking different browsers.

Example. The first scenario being executed is the scenario that con-

tains only the default loading event: so=e0d, where e0d is the default

loading event. In the example from Listing 5.1, this means constructing the DOM of the page (based on lines 1 – 9) and executing JavaScript statements in lines 10 – 14, 26. At this point, no user-generated events are executed. The loading of the example page does not depend on the internal browser state. After the page is loaded, the browser is aware of

54 Chapter 5. Automatic Scenario Generation

two event registered event handlers (onmousedown in line 14 and onclick in line 26). This information will be used by the process to generate new scenarios.