Threads to Validity

Besides the testing code, different devices and hardware configurations that can influence energy profiling results, further threats possibly influencing the profiling results presented above are discussed in the following.

Continuous Powering via USB Connection

The test cases have been executed on a DUT via a USB connection from a test runner PC. Unfortunately, the USB connection provides the DUT with energy during each test run which cannot be disabled while running the tests. Thus, a second power meter has been used, profiling the power rate at the USB cable during all test runs and added the power rate to the DUT’s profiled energy consumption. It can be assumed that the quantitative results of the measurements may be marginally affected by the variance of this power supply. However, all test runs were issued with a fully-charged battery to reduce the noise for battery charging and to ensure comparable measurements. Besides, for comparative measurements—which are in the major focus of our work—the influence of the continuous powering via the DUT’s USB connection is rather low and, thus, can be ignored.

Application Behavior depending on USB-Connection

Within the Android OS, applications are able to access battery information via the Android battery manager (cf. Sect. 3.3.2), including information whether the battery is currently charged via USB or an external power supply. Thus, applications can be designed as adapting their behavior to the current battery state, to avoid higher energy consumption rates when running on battery power. This effect has not been addressed by the case studies conducted in this work, but could be evaluated for the respective applications in future work.

Influence of Application Settings

Some of the applications evaluated in the case studies allow several settings to configure them w.r.t. the users’ needs (e.g., for some of the browsers, caching can be enabled or dis- abled). Within this work, applications have been tested with their default settings only. Of course, adapting their settings could have led to more energy-efficient results. How- ever, most users will use applications under their default settings. Thus, these settings should be considered for the comparative test runs. For investigating the influence of application and device parameter settings on energy consumption, the interested reader is referred to related work, for example the work of Palit et al. [94].

9.2. Energy Profiling and Derivation of Energy Models

Influences from Helper Applications

In some cases, Android applications delegate services to other applications via so-called Intents. For example, downloaded files are opened by another application, depending on the file type (e.g., by an image or PDF viewer). Thus, the energy consumption of respective use cases depends on the application to open the files as well. As a conse- quence, for the tests conducted within the evaluation of this thesis, a dummy application has been realized that simply receives the open file intent but does not do anything (except terminating itself) to exclude this effect from the test runs.

Indeterminism of Profiled Use Cases

Considering the executed use cases, some of them behave indeterministicly. First, net- work communication can vary for each test run as data packages can be lost and have to be resent again. Besides, the used WiFi router and network hardware can influence the throughput and, thus, the applications’ runtime behavior as well. Furthermore, the content of real web pages such as Google or the New York Times can change between the individual test runs (e.g., by new search results or news to appear over time). These factors can influence test results, resulting in higher variances for the profiled power rates and execution times, as for example for the Firefox browser and the New York Times weg pages (cf. Fig. 9.6). Nevertheless, the presented profiling results can be considered as being quite representative, as test cases have been executed 50 times for each appli- cation. Furthermore, the significance of the measured differences between the evaluated applications has been confirmed by a Kruskall-Wallis test, leading to significant measur- able differences for all test cases (except for the mail client background tests). However, for future benchmarks comparing applications depending on external content, such as websites, it should be considered to use copied versions of the websites deployed on a local server instead, to reduce the indeterminism causes by varying website content. Representativeness of the Selected Use Cases

The use cases of the evaluated usage domains have been designed manually and it can be questioned, whether or not they do represent realistic and average usage scenarios of the tested applications. A survey conducted in October and November 2012 with 132 cell phone and smart phone users showed that web browsing and emailing are actually the most typical use cases for today’s mobile devices, as 18% and 16% of all users said that they would use these kinds of applications more than ten times on each day. 42% and 48% of all users said that they used these applications between two and ten times each day (cf. Fig. 9.17) [67]. Questioned for their three most important applications, 52% of the users named email clients and 48% of the users named web browsers belonging to their most important mobile applications (cf. Fig. 9.18).

9. Evaluation 0% 10% 20% 30% 40% 50% 60% Web Browser Email Clients Social Networks

Calendars Games Dictionaries News Apps Navigation

more than 10 times per day 2 to 10 times per day up to once per day

less or never

Figure 9.17.: Self evaluation of 114 smart phone users on the question how often they use a given set of types of mobile applications [67].

52% 48% 39% 36% 36% 35% 34% 32% 18% 12% 11% 10% 9% 6% 19%

Figure 9.18.: Percentage of smart phone users naming these categories of applications belonging to their three most important mobile applications [67].