SOA Based Mobile Application Software Test
Framework
LIU Zhi-fang, LIU Bin
Dept. of System Engineering of Engineering Technology Beihang University
Beijing, China
GAO Xiao-peng
School of Computer Science and Engineering Beihang University
Beijing, China
Abstract—The most important issues of the test framework are its generality, reusability and scalability in a specific area. Mobile application software that constantly updates and improves makes it especially difficult for these points. This paper first proposes the definition of mobile application software, as well as its classification, and summarizes the mobile application software testing technology. Then we introduce the thought of service-oriented architecture into this area, and adopt COM technique, aims at designing a mobile application software test framework. During the design, we focus on the technical implementation and interface definition. Finally a test platform is built based on the framework and used to test a typical software of mobile phone. The test results show that the test framework greatly improved the testing flexibility and provided a good foundation for implementation and expansion of mobile application test.
Keywords-SOA; test framework; mobile application software; component reuse; COM
I. INTRODUCTION
Mobile phone Test mainly includes hardware-related stress test, anti-drop test, high/low temperature test, key life test, electrostatic test [1], as well as software-related protocol test [2-3] and application software test, as in [4-5]. In this paper major research subjects are the mobile application software test. Some related work includes log-based test [6], test method based on the script-driven [7-9], based on the AT command test [5], etc. However, whichever testing method we choose, a framework would be required to implement the strategies, and building a test framework is facing the following problem.
• Mobile phone system is not stable. Different mobile phone manufacturers have different opinions in the platform selection and R&D, resulting in the market diversity of mobile platforms. Even the same phone vendor may cover multiple platforms to create their different positioning of products. In addition, a platform is also constantly upgrading. Mobile phone system as a mobile application software running platform directly restricted the means of tests implementation, thereby affecting the overall framework design.
• Mobile application software updates frequently. Changes in mobile phone system are one of the reasons for application updates. But lacking of a universal and unified architecture for mobile application software development is the major reason,
and more important this type of update leads to structure, data flow differences between versions and moreover the substantial adjustments of testing methods, test tools and test cases which can often be on the basis of them.
• For different applications to vary the demand for testing. Different applications have their own unique business flow and data flow, this difference inquires a sophisticated testing framework.
Service-oriented architecture (SOA) is a service concerned distributed architecture, which is interoperable, easy to be integrated, and with high scalability and security access features [10] , and which is the trend of software engineering technological development, as in [11-12]. Service interface design is considered to be one of the key issues of constructing SOA [13], and the other significant issue is the division of service components and the cooperation between them, which is also related to the quality and scalability of the architecture.
The purpose of this paper is to take the advantages of SOA to solve the problems in mobile application software testing field, with the goal of proposing a more general framework, and also bring forward a new way of thinking and technology for the implementation of field testing.
II. SOA BASED DESING
The mobile phone applications software described in this paper refers to the software running on the intelligent mobile phone system with operable graphical user interface (GUI). Intelligent mobile includes: Symbian, Windows Mobile, Linux, Android and other open systems.
Some popular usage methods in the mobile application software test are as follows, as in [14-15].
• Record/Playback Technique: Recording the events occurred in GUI operation through user's mouse and keyboard input and by recycling recorded events. But if any change to application occurred, all the influenced test cases should be re-framed.
• Capture/Playback Technique: User uses mobile application software to test by setting an example, and then writing scenario
• Particulars-Based Test Technique: The particulars are necessary to be explicitly described, to be perfect and to be summarized and designed. To get these particulars makes constant efforts.
• Beta Test Technique: Beta test is the most popular test method. It is used very much as a method carrying out test to common users by merchandising the software of beta version. But it is difficult to get common users to educate and understand which will affect the test effect.
In this paper, we choose script-driven test method, and at the same time Capture/Playback and Particulars-Based Test Technique are combined and adopted. So the test method will be first design test cases and write test script based on the Screen Particulars, and then run the test.
A. SOA Based Overall Design
According to SOA, the architecture contains four main proportions: Service, Service users, Service providers, Register Service Center, as in [16].
• Service: Services adhere to a communications agreement, as defined collectively by one or more service description documents. Mobile test oriented services design needs to summarize the functions in the form of services in accordance with the mobile software requirement, and define specific services according to its implementation and invoking behavior. After analysis, the main test platform services are divided into two categories: user interaction services, testing-related services. User interaction service is responsible to provide GUI, but testing services differ in their ways of interaction with users, and for which it is difficult to propose a generic model for them. Therefore we need to provide a service, which manage all GUI element of testing services. Testing-related services design depends on the features of mobile software testing. These services include test management, test execution, process control, testing implementation, communications, results analysis, data management and some other services.
• Service client/Service provider: Service providers publish services to Register Service Center through its contract and interfaces in order that service clients can query and access the service. Mobile application software test services based on the above test design can be roughly divided into: service providers such as user interaction service, test execution, communications, data management, and service clients such as test management, test flow control, results analysis. Service components are not necessarily very strict identity, such as test execution component, which may also use the user interaction services to show the test process information, while it has dual identity. Therefore service implementation does not have to rigidly adhere to this as it is just a conceptual division.
• Register Service Center: Services query supporter, in which a repository is built in order to store a variety of services and interfaces in a unified form, allows service clients to query and access available services. Based on the above idea of SOA analysis, the concept of the testing framework has been formed, and the architecture
detailed design will be mainly from the following two key issues: (1) Services component division and design (2) Service component interface design.
B. Component Division and Design Based On COM
COM is a cross-application and the language of binary code-sharing method [17], which is chosen for achieving SOA in this paper in the consideration of generality. We build independent components for different services using COM technology, which makes the component location in the platform transparent since the self-registration feature. Because of the unified specification of component design, any service can be called through the same interface.
Register Service Center Component: It needs to maintain a dynamic list of services and realize a query mechanism for querying the appropriate service interface based on the service description and returning it to service users. Generally, services registration and list initialization happens when the test platform is starting, but service registration or un-registration is also permitted during the platform’s running.
User Interaction Service Component: Providing the basic framework of GUI and interfaces for other service components in order to generate a suitable frame in which the components are able to create their own screen elements. It gives the centralized GUI management.
Other Service Components: Implementing the specific services and interfaces in accordance with the unified interface specification.
Component-based framework makes the upgrade and improvement more flexible, be able to import new modules continuously. At the same time, because of the independence of the functional modules and standardized interface, the test platform is easy to transplant to web application mode.
C. Framework Interface Design
Register Service Center module provides interface IEventDispatch in order to achieve registration and inquiries. Through the interface, this module receives service register information, inserts them into a designated list, and also accepts service request and return back the appropriate service interface.
In order to achieve a unified interface between the service calls, Register Service Center module contains a virtual interface IChildComInterface, which should be derived from when building a service provider component and realizing the service functions. The interface parameter design with reference to that of IDispatch, has mutable number and type of variable.
Figure 1. Service Component Interface Design
Meanwhile, the interface IChildComInterface inherits in the ICategory, in which way, Register Service Center module only needs to record a Category GUID, then system platform will be able to get the location of service components and create the services, which facilitate the expansion of service component. Design was shown in Fig. 1.
IChildComInteface is the standard interface from which Service component interface should be derived. Interface function Initial () will be called by the founders after the service is created for the initializing the Register Service Center interface address, and returning the service identification information. A service will be executed when calling the function SendEvent(), which is responsible for the service providing. Interface function design is as follows:
Initial(IEventDispatch* pEventDispatch, ULONG * nComID)
IEventDispatch* pEventDispatch [in] Register Service Center Interface Address
ULONG * nComID [out] Identity of current Service Component
SendEvent(LONG nComID, ULONG nFlag, LONG pDispParams, VARIANT FAR* pVarResult, long* nErrorCode)
LONG nComID [in] Identity of origin Service Component ULONG nFlag [in] Flags
LONG pDispParams [in] Parameter Set VARIANT FAR* pVarResult [out] Return Value long* nErrorCode [out] Error Code
Interface IEventDispatch is used for Service Registration and Query. Function SendEvent() traverses the service list and return the specified interface. Function CreateAllChileCom will be called after the Register Service Center is created for instantiating testing-related service components.
SendEvent(LONG nComID, ULONG nFlag, LONG pDispParams, VARIANT FAR* pVarResult, long* nErrorCode)
LONG nComID [in] Identity of origin Service Component ULONG nFlag [in] Flags
LONG pDispParams [in] Parameter Set VARIANT FAR* pVarResult [out] Return Value long* nErrorCode [out] Error Code
CreateAllChildCom ()
During the running of the Register Service Center module, it only accepts the data from function SetEvent() through
inte
Im ication software testing
anagement and org
d implementation, as well as the def
ents for
ned functional components, to achieve the effect of:
A. Example selection
W S function on the phone of Symbian
st object, because its features are fixed, mo
MSFUNCTIONAL TEST DESIGN
rface IEventDispatch, then queries services interface and returns it to the Service Client. Because Service Client may require multi-services and use them collaboratively, the process of inquiry services inquiries to do a complete traversal algorithm so that all the service interfaces can be matched back to the Service Client.
D. Implementation of the Framework
plementation of the mobile appl
framework needs to build SOA modules of m
anization, and design basic service components, which supports the test flow.
The former mainly includes: Registration Service Center component design an
inition of virtual interfaces for the service components. The latter mainly include management-related components such as project manager, test monitor; and the core compon
driving testing: user interaction framework, resource manager, the virtual phone, script editor, database manager, script interpreter, result display and communication manager components.
In this paper, it cost about 12 person-months to prepare the above-mentio
being able to enter test script and run them to drive the phone, and at the same time observe the real-time results.
III. EXAMPLE
e selected the SM S60 2nd edition as the te
de of operation is relatively unified, and thereby it’s more easily to test via a platform. The function under test and cases design is shown as TABLE I.
TABLE I. MOBILE S Test
Item Test Point
Cases Number Editing SM Characters empty message, lo abnormal mixed message, ng message, numbers sent; insert phrases, contact, pictures and save; input switch; call incomes when editing
43
Inbox 35
View, reply, forward, observe the message signs; delete, save number
Outbox View, reply, forward; delete 21 Drafts Save drafts, forward; delete, delete multiple 18
Others
Receive message while stand by, talking, or writing message;
ing mailbox capacity testing; message capacity test
67
B. Construction
Based on t ments, we utilized the test
llowing components in order to con
of platforms
he above test require framework and expanded the fo
struct the test platform: Schedule editor, and image comparison. New testing service components were built using COM technology, and packaged in accordance with the framework interfaces. We spent around one person-month for
the preparation of new components and add them into the platform. Dependences between the components are shown in Fig. 2.
Figure 2. Dependences between the Service Components
C. Run
The p of testing SMS function via mobile application platform includes the creation of test project, mo
Figure 3. Process of Running Test using Test Platform
D. Data Ana
W of manual testing and that of
testing as TableⅡ (unit: person-hour).
TES test
rocess software test
bile configuration, test script edit, running test and analyzing the results. Test steps and the platform screenshot are in Fig. 3.
lysis
e compared the workload the test via platform
TABLE II. COMPARISON BETWEEN THE WORKLOAD OF MANUAL
TESTING AND THAT OF THE T VIA PLATFORM
Test Method Case n Case Selection Script Edit Run Test Regression Test Total Desig Manual Testing 24 0 0 32 16 72 Via Platform 24 4 24 16 8 76 Ac
includescordin tw xtra st s: Case Tab , althou the test via test platfolectio nd Sc t Edit,
ove
re wo
d test is practical and eff
ONCLUSIONS
This paper first de cation software and
classified them based th and test strategies, wh
y, and the imp
ication software, which separated the
[1] Zhang, S.Y. (2006), Design of automatic testing system on SC660X platform, Beijing Univer elecommunications
-3,Computer
g to
o e eple 2 Segh n a rip therm
rall workload is nearly the same as that of the manual testing. Especially in the phases of Run Test and Regression Testing, apart from the significant workload reduction, using the platform also makes the implementation of test cases more accurate and greatly facilitates the error locating via recording the entire testing process in detail. According to another statistics, 150 test cases in 184 can be run on this platform, which means test automation rate is as high as 81.5%, thereby the test quality is guaranteed under an acceptable efficiency.
Something to highlight: it is the first time to test this type of software using the test framework, so there are relatively mo
rk we need to do, such as preparation of the necessary component for constructing the platform which cost one person-month approximately, test case selection and case script edit and debugging. But when meeting some similar type of function test, the platform is almost no longer necessary to carry out changes, and the test cases only need to be modified from the existing work. Thus these two steps will greatly decrease the workload, coupled with the high-efficiency in steps Run Test and Regression Test, the advantages of using test platform for testing will be reflected.
To conclude, the method of using the framework described in this paper to construct test platform an
ective.
IV. C
fined the mobile appli on eir architecture
ich made a preparation for the framework theory research. Then SOA theory was applied in the mobile application test field which greatly improved the flexibility of testing and provided a good foundation for implementation and expansion of mobile application test. Finally a function-specific test platform implementation was demonstrated to prove the effectiveness of the SOA based framework theory.
The advantages of using COM as the technical implementation is the versatility and scalabilit
lemented services can be maximally reused, whose process is transparent to the callers.
This paper described the use of SOA based software test framework for mobile appl
test design and implementation, and from an overall perspective divided the mobile phone software test into different levels. Also from test development perspective, this method improved the test continuity and scalability, thereby increased the overall efficiency of the test.
REFERENCES sity of Posts and T
[2] Sun, J. Zhao, H.Q. Ma, Y.F. (2006), Mobile Phone Game Testing Research Based On TTCN, Symposium on Chinese test
[3] Wang, Y.R. (2008), Research on GSM-R mobile terminal testing methods and testing system, Beijing Jiaotong University
[4] Shang, S.C. Zhao H.Q. Chen, P. (2008), Study on method of mobile phone network application software testing based on TTCN
[5] Yang, Z.H. Hu, Q. Ling, J.R. (2008) The Studying of SMS system’s testing on Smartphone Platform, Embedded software applications, 24(1-2), 92-93, 277
[6] Xu, T.F. (2007) Mobile Phone Test Method Research Based on Log Analysis, Nanjing University of Technology and Engineering
[7] Zhao, Q. (2008) The Design and Development of Semi-automatic Test Tool for Cell-phone, Beijing Jiaotong University
[8] Zhang, Z.M. (2008) Design and Realization of Test Automation for Mobile Phone Software, Beijing Jiaotong University
[9] Jiang, B. Long, X. Gao, X.P. (2007), MobileTest: A Tool Supporting Automatic Black Box Test for Software on Smart Mobile Devices, Second International Workshop on Automation of Software Test [10] Guo, Y. Deng, H. Yi, S.B. (2006) Service Oriented Grid Software
Testing Environmen, Journal of Software, 17(11): 2335-2340
[11] Yang, F.Q. (2005), Thinking on the Development of Software Engineering Technology. Journal of Software, 16(1):1−7.
[12] Cervanted H, Hall RS. (2005), Technical concepts of service orientation. In: Zoran S, Ajantha D, eds. Service-Oriented Software System Engineering: Challenges and Practices. Idea Group Publishing, 1−47. [13] Chen, Z.B. Wang, J. Dong, W. Qi, Z.C. (2006), An Interface Model for
Service-Oriented Software Architecture, Journal of Software, 17(6):1459-1469
[14] Hwang, S.M. Chae, H.C. (2008), Design & Implementation of Mobile GUI Testing Tool, International Conference on Convergence and Hybrid Information Technology, 704-707
[15] Kwon, O.H. Hwang, S.M. (2008), Mobile GUI Testing Tool Based on Image Flow, Seventh IEEE/ACIS International Conference on Computer and Information Science, 508-512
[16] http://en.wikipedia.org/wiki/Service-oriented_architecture [17] BOX, D. (2001) essential COM, China Electric Power Press
