One of the disadvantages of using Timestamp Transfer approach is the inability of keeping track of the status of a specific record. As described, the challenge for implementing the
Timestamp Transfer strategy on both platforms, iOS and Android, is to manage properly the delete operation.
To determine that an item has been deleted, a Boolean flag is used. For this purpose a new field is added, as shown on Figure 12, which indicates whether the item has been deleted or not.
Figure 12-Deletion
When synchronization occurs, the mobile client receives a response from the server with the newly modified records. The deleted field is used to mark a record so if a deletion is indicated, the mobile client would remove it from the local repository.
Scalability and future problems
With the growth of the system, and an analysis ofthe current Skonto data synchronization a few problems are possible to occur in the near future. First and foremost, the system currently does not support upload from a mobile device to server. This was not an issue in the early stages of the project, but as the apps mature it may become unavoidable. Furthermore, a large part of the data that is synchronized from the server to the app is not actively used. This combined with the expected growth of shop’s attributes to be stored may lead to a high level of redundancy.
Additionally, the present synchronization scheduling is not optimal, which generates unnecessary requests to the server.
Solution
Considering the problems described in the previous chapter, only the issues most valuable from stakeholder perspective will be analyzed. In this part of the project the focus will be on
optimizing the locally stored data and introducing data upload from mobile client to server. The solution will be designed, but not implemented. The main reason is the fact that the system is in very early stages and it is not known how it will advance in future. The risk is too high and possible implementation will not be practical, considering time and resources. Additionally, if the project evolves, the requirements and priorities may drastically change, which could eliminate the need for this particular implementation.
Problem Analysis
Initially, Skonto started primarily with discounts at fast food joints only. However, the owner quickly diversified and introduced other businesses. As a result, many new shops and categories were added to the app, which highly increased the amount of inessential information locally stored on the mobile device. Moreover, extra attributes such as comments and photo albums are expected to appear in future versions. Hence, it will not be practical to store them on the phone. Additionally, if the system proves to be successful extended functionality such as comments and ratings may have to be included.
After reflecting on those problems, the following requirements were identified as critical for Skonto’s near future:
Data upload - include support for upload from mobile client to server in order to allow ratings and comments
Reduce locally stored data – eliminate unnecessary information stored in the local storage of the mobile device
Design
This part focuses on building solutions for the analyzed problems. It is logically divided into two parts, each one with emphasis on a particular requirement from the previous chapter.
Data Upload
Sending data from the apps to the server has to be implemented in order to support a rating system and user comments for every shop. The user selected rating will be uploaded to the server through a web service. In cases of no internet connection the chosen rating will have to be stored in the local database until synchronization with the server is possible. When the user wants to leave a comment a similar approach will be adapted.
In order to implement this new functionality a few things have to be changed in the system at general. To begin with the backend has to be extended to store the user ratings and comments. Afterwards, a service has to be exposed in order to allow submitting ratings and comments from the mobile clients. The rating calculation and storage on the server side are outside the scope of this project. However, the solutions for iOS and Android will be discussed in detail in the following two parts.
DATA UPLOAD IN ANDROID
In order to support upload functionality, the Android architecture has to be modified, as shown on Figure 13. After a field, which indicates that a record is updated, has been added to the local SQLite database, the Content Provider must expose a method, which queries the dataset, selects and returns all modified records since the last successful synchronization. Additionally, the web service that connects the Skonto Android application with the remote system has to be modified so it requires a timestamp of the last successful data transfer, along with a list of updated shops. The rest of the components are unaffected.
Figure 13-Android Data Upload architecture
The process of synchronization, consisting of upload and download operation, between a Skonto Android application and Skonto server can be visualized as a sequence diagram as shown on Figure 14. It is initialized every time the user opens the application. When the activity is started, an event, which runs the Sync Adapter, is triggered. Before sending a network request to the Skonto server, the timestamp of the last successful synchronization, stored in the Shared Preferences, is retrieved and used for comparison for getting all newly modified records. Then the body of the POST Request is constructed, and the fetched timestamp and items are sent to the Skonto server. The server processes the information and returns a response, which contains new or modified shops for the mobile device. The shops are parsed, stored into the database via the
Figure 14-Android Data Upload SD
Data Upload in iOS
Similarly to the Android implementation the iOS architecture will have to be changed, as visualized on Figure 15, in order to support uploading. A modified attribute will have to be included in the local database in order to indicate when the last time the particular record has changed. When the Sync Engine is triggered it will get all the records that have changed since the last synchronization and it will send them as parameters to the Networking Layer. The changed data is included in the request to the web service and the response is handled without any difference.
Figure 15-i0S Data Upload architecture
The following sequence diagram, Figure 16, represents a detailed visualization of how the logic will flow when synchronization is triggered. The SyncEngine is still responsible for controlling the synchronization process; the first thing it does is taking the timestamp of the last
synchronization from NSUserDefaults and loading all the shops that have local changes waiting for upload. Once the shops are loaded from the database, through the Core Data framework, the
SyncEngine calls the networking class responsible for making POST Requests and parsing the server response. The networking class sends the updated data to the web service and in return receives new or updated records from the server. The records are being sent back to the
SyncEngine, which updates the local database and stores a new timestamp in the
Local data reduction
The following diagram, Figure 17, presents all the shops’ attributes currently stored on the mobile devices.
Figure 17
Most of the attributes are self-explanatory, however there are a few which should be described in more detail.
Isfavourite is an integer representation of Boolean, which shows whether the shop is marked as favorite on this particular device or not.
Notificationsareenabled is an integer representation of Boolean, which indicates if notifications related to this shop are allowed on the particular phone. The user can activate and deactivate the notifications for every shop.
Modified is a timestamp, which represents the time of last data change. This is used for uploading only the data that has changed since the last synchronization.
Notification_sent is a timestamp, which indicates when the last time a notification related to that shop was sent. A requirement for the Skonto app was to not send more than one notification per shop in 24 hours, as a result this field was introduced to keep track on the notification timing.
Rating and selected_rating are used to store the general rating of the shop and the one selected by the user. The selected _rating field is important because if the user picks a rating while he or she is offline it will be locally stored until synchronization occurs.
With the inclusion of new attributes, such as comments and photo albums, it will not be practical to store everything locally since it may result in a high usage of storage and data transfer
resources. Downloading and accessing the required information from the server on demand could avoid this. Similarly, the current set of attributes could be decreased.
After careful consideration and discussion with the owner it was decided, that if data has to be excluded and not available offline the local database could look the following way, Figure 18:
One of the main changes compared to the previous diagram is the exclusion of opening and closing hours, which were represented in many attributes since a part of the requirements was to show in the view of every shop if it is open or closed at the moment. Additionally, also the shop description was removed, as it usually consists of a long text, which occupies larger space than any other piece of information. A disadvantage of this reduction is the inability of providing working hours information and shop description when the phone is offline.
In extreme cases an even more drastic data reduction could be needed, hence another exclusion of locally stored attributes was considered. The following diagram, Figure 19, is the shop information that is regarded as absolutely essential.
Figure 19
Each one of these attributes is necessary for the proper functionality of the app. The flags for notifications, favorites and rating selection are used for storing user preferences; therefore they have to be stored locally. The title, category, city and region are essential for sorting the shops in the app. The coordinates are used for delivering notifications based on geofencing, even when no internet connection is available.
With the attributes reduction, a part of the information will be accessed on demand. This leads to many network request to the Skonto server which can be reduced with the help of caching. Nevertheless, caching is a complex subject and it is outside the scope of this project. Therefore it will not be elaborated upon.
Another data reduction strategy is to reduce the amount of shops synchronized on the mobile device. Since the Skonto application has discount at shops through whole Denmark, it is unreasonable for a user located in Aalborg or other city to have the data for the whole country saved on his/her device. Accordingly, the data set of synchronized shops could be based on user location, region or city. This way the locally stored information will be reduced significantly. However, handling frequent travelers and users living in between two major areas will be a challenge. Considering the desire for simplicity and the currently limited budget of the owner, such a solution will not be rational, hence the focus will be on attribute exclusion strategy.
LOCAL DATA REDUCTION IN ANDROID
After a careful consideration, the set of attributes, stored locally, has to be defined, so that some of them will be kept on the mobile device and the rest received via network call to the remote system. This is visualized on Figure 20. The data stored on the Android device will be accessed through a Content Provider, where an AsyncTask, which will be added to the
RestServiceAdapter,willhave the responsibility of sending a request to the Skonto server and visualizing the response. It will perform the network call asynchronously and publish the result to the interface only after the task is finished.
Figure 20-Android Local data reduction Architecture
With this solution, there are two ways of accessing the information, both of which are visualized and discussed accordingly:
Data stored locally – When a information is saved on the mobile devices,the Cursor Loader communicate with the Content Provider,and desired data is returned.The process is visualized on Figure 21.
Figure 21-Data stored locally
Data not stored locally – When data is accessed via network call, a new background thread, which does not block the UI thread, is started. A request is sent to the Skonto service and the response is published as a result on the user interface. The technique can be seen on Figure 22.
Figure 22-Data not stored locally
LOCAL DATA REDUCTION IN IOS
Similarly to the Android implementation, the iOS architecture has to be modified in order to support loading a set of shop attributes on demand. The bold parts of Figure 23 visualize which components of the current implementation will have to be changed. The UIViewController will have to start an asynchronous synchronization event on a separate thread with the help of the
Grand Central Dispatch (GCD). Afterwards the Networking Layer will load the additional attributes from the server and the process will return the data to the controller, which will update the view.
Figure 23-iOS Local data reduction
The following sequence diagrams visualize how the app will operate, depending on the data availability. The locally available attributes will be loaded from Core Data, Figure 24, while the additional shop attributes will be asynchronously downloaded from the server, Figure 25.