Experimental plan

We defined an experimental plan by considering different combinations of the levels and factors presented in the previous subsection. In turn, the experimental plan can be divided in three sets. The full experimental plan includes 72 experiments, for a total of 18 days of experimentation. All the experiments are listed in Table4.2and sorted from the oldest to the newest Android version. The three test plans are blocked full-factorial designs, in which one factor is fixed (i.e., the Android device or version, according our research questions), while we vary all the other parameters.

TABLE4.2: Experimental plan of the case study

ID DEV VER APP EVENTS STO

EXP1 HUAWEIP8 ANDROID5 EU MIXED1 NORMAL

EXP2 HUAWEIP8 ANDROID5 EU MIXED1 FULL

EXP3 HUAWEIP8 ANDROID5 EU MIXED2 NORMAL

EXP4 HUAWEIP8 ANDROID5 EU MIXED2 FULL

EXP5 HUAWEIP8 ANDROID5 EU MIXED3 NORMAL

EXP6 HUAWEIP8 ANDROID5 EU MIXED3 FULL

EXP7 HUAWEIP8 ANDROID5 CHINA MIXED1 NORMAL

EXP8 HUAWEIP8 ANDROID5 CHINA MIXED1 FULL

EXP9 HUAWEIP8 ANDROID5 CHINA MIXED2 NORMAL

EXP10 HUAWEIP8 ANDROID5 CHINA MIXED2 FULL

Table 4.2: Experimental plan of the case study – continued from previous page

ID DEV VER APP EVENTS STO

EXP11 HUAWEIP8 ANDROID5 CHINA MIXED3 NORMAL

EXP12 HUAWEIP8 ANDROID5 CHINA MIXED3 FULL

EXP13 HUAWEIP8 ANDROID6 EU MIXED1 NORMAL

EXP14 HUAWEIP8 ANDROID6 EU MIXED1 FULL

EXP15 HUAWEIP8 ANDROID6 EU MIXED2 NORMAL

EXP16 HUAWEIP8 ANDROID6 EU MIXED2 FULL

EXP17 HUAWEIP8 ANDROID6 EU MIXED3 NORMAL

EXP18 HUAWEIP8 ANDROID6 EU MIXED3 FULL

EXP19 HUAWEIP8 ANDROID6 CHINA MIXED1 NORMAL

EXP20 HUAWEIP8 ANDROID6 CHINA MIXED1 FULL

EXP21 HUAWEIP8 ANDROID6 CHINA MIXED2 NORMAL

EXP22 HUAWEIP8 ANDROID6 CHINA MIXED2 FULL

EXP23 HUAWEIP8 ANDROID6 CHINA MIXED3 NORMAL

EXP24 HUAWEIP8 ANDROID6 CHINA MIXED3 FULL

EXP25 HTCONEM9 ANDROID6 EU MIXED1 NORMAL

EXP26 HTCONEM9 ANDROID6 EU MIXED1 FULL

EXP27 HTCONEM9 ANDROID6 EU MIXED2 NORMAL

EXP28 HTCONEM9 ANDROID6 EU MIXED2 FULL

EXP29 HTCONEM9 ANDROID6 EU MIXED3 NORMAL

EXP30 HTCONEM9 ANDROID6 EU MIXED3 FULL

EXP31 HTCONEM9 ANDROID6 CHINA MIXED1 NORMAL

EXP32 HTCONEM9 ANDROID6 CHINA MIXED1 FULL

EXP33 HTCONEM9 ANDROID6 CHINA MIXED2 NORMAL

EXP34 HTCONEM9 ANDROID6 CHINA MIXED2 FULL

EXP35 HTCONEM9 ANDROID6 CHINA MIXED3 NORMAL

EXP36 HTCONEM9 ANDROID6 CHINA MIXED3 FULL

EXP37 LGNEXUS ANDROID6 EU MIXED1 NORMAL

EXP38 LGNEXUS ANDROID6 EU MIXED1 FULL

EXP39 LGNEXUS ANDROID6 EU MIXED2 NORMAL

EXP40 LGNEXUS ANDROID6 EU MIXED2 FULL

EXP41 LGNEXUS ANDROID6 EU MIXED3 NORMAL

EXP42 LGNEXUS ANDROID6 EU MIXED3 FULL

EXP43 LGNEXUS ANDROID6 CHINA MIXED1 NORMAL

EXP44 LGNEXUS ANDROID6 CHINA MIXED1 FULL

EXP45 LGNEXUS ANDROID6 CHINA MIXED2 NORMAL

EXP46 LGNEXUS ANDROID6 CHINA MIXED2 FULL

EXP47 LGNEXUS ANDROID6 CHINA MIXED3 NORMAL

EXP48 LGNEXUS ANDROID6 CHINA MIXED3 FULL

EXP49 SAMSUNGS6EDGE ANDROID6 EU MIXED1 NORMAL EXP50 SAMSUNGS6EDGE ANDROID6 EU MIXED1 FULL

Table 4.2: Experimental plan of the case study – continued from previous page

ID DEV VER APP EVENTS STO

EXP51 SAMSUNGS6EDGE ANDROID6 EU MIXED2 NORMAL EXP52 SAMSUNGS6EDGE ANDROID6 EU MIXED2 FULL EXP53 SAMSUNGS6EDGE ANDROID6 EU MIXED3 NORMAL EXP54 SAMSUNGS6EDGE ANDROID6 EU MIXED3 FULL EXP55 SAMSUNGS6EDGE ANDROID6 CHINA MIXED1 NORMAL EXP56 SAMSUNGS6EDGE ANDROID6 CHINA MIXED1 FULL EXP57 SAMSUNGS6EDGE ANDROID6 CHINA MIXED2 NORMAL EXP58 SAMSUNGS6EDGE ANDROID6 CHINA MIXED2 FULL EXP59 SAMSUNGS6EDGE ANDROID6 CHINA MIXED3 NORMAL EXP60 SAMSUNGS6EDGE ANDROID6 CHINA MIXED3 FULL EXP61 SAMSUNGS6EDGE ANDROID7 EU MIXED1 NORMAL EXP62 SAMSUNGS6EDGE ANDROID7 EU MIXED1 FULL EXP63 SAMSUNGS6EDGE ANDROID7 EU MIXED2 NORMAL EXP64 SAMSUNGS6EDGE ANDROID7 EU MIXED2 FULL EXP65 SAMSUNGS6EDGE ANDROID7 EU MIXED3 NORMAL EXP66 SAMSUNGS6EDGE ANDROID7 EU MIXED3 FULL EXP67 SAMSUNGS6EDGE ANDROID7 CHINA MIXED1 NORMAL EXP68 SAMSUNGS6EDGE ANDROID7 CHINA MIXED1 FULL EXP69 SAMSUNGS6EDGE ANDROID7 CHINA MIXED2 NORMAL EXP70 SAMSUNGS6EDGE ANDROID7 CHINA MIXED2 FULL EXP71 SAMSUNGS6EDGE ANDROID7 CHINA MIXED3 NORMAL EXP72 SAMSUNGS6EDGE ANDROID7 CHINA MIXED3 FULL

The first set (EXP13∼EXP60) covers all of the DEV levels, and keeps the VER factor to ANDROID6, since Android 6 Marshmallow is the only version that can be installed on all the devices, allowing us to study the impact of software aging across devices from different vendors (and all other factors with the same level). The second set of experiments (EXP1∼EXP24) fixes the DEV factor to HUAWEIP8, and varies the VER factor between ANDROID5 and ANDROID6. The third set (EXP49∼EXP72), instead, fixes DEV to SAMSUNGS6EDGE and the VER to either ANDROID6 or ANDROID7. These last two sets are used to study the impact of software aging across different versions of the Android OS. In each set, with the sole exception of the fixed factor, we consider every possible combination of the levels, leading to a full factorial design. Based on our preliminary experiments [60],

USB CABLE

Workstation

Database

Commands: monkey, logcat Test Configuration: app, storage

Activity Launch Time PSS Measurements GC Measurements Task-level Measurements

FIGURE4.1: The Experimental Android Testbed

we calibrated the duration of each experiment to 6 hours, as this duration suffices to point out software aging effects.

The devices are controlled and monitored using the Android Debug Bridge (ADB) utility (which is a non-intrusive, dedicated channel through the USB port for debugging purposes). User inputs are provided with the monkey tool, which is a workload generator that randomly generates UI events. The events are generated at a high frequency (500ms) to stress the device, and follow the random profile of the EVENTS factor. The experimental testbed is showed in Figure4.1.