Multimedia Sharing over the Internet
from a Mobile Phone
Rui Trindade1,2, Paulo Correia1and Miguel Santos2
1Instituto Superior Técnico – Instituto de Telecomunicações, Av. Rovisco Pais, 1049-001 Lisboa, Portugal
2Mobi2do, Sala 332, Núcleo Central, Taguspark, 2740-122 Oeiras, Portugal
rui.trindade@ist.utl.pt, plc@lx.it.pt, miguels@mobi2do.com
Abstract. Nowadays mobile applications are among the most popular services in the telecommunications world. The increased capabilities of mobile phones and the new mobile networks providing high data rates have created the possibility to exploit the development of connected and fully featured mobile applications. In this context, there are great opportunities for applications that go beyond voice or text transmissions, such as multimedia sharing. This paper proposes a set of solutions to share multimedia content from a mobile phone, exploring the capabilities and the constraints of the existent technology. This system supports several multimedia modalities that can be selected and switched according to a set of application scenarios and usage conditions. The proposed solutions include a multimedia uploading application for BlackBerry OS phones and a photo streaming application for Windows Mobile OS phones.
Keywords: Mobile, Multimedia, Uploading, Streaming, Upstream
1 Introduction
Based on the technological improvements in the mobile area and on the social impact of multimedia sharing applications, this work aims to develop a solution to share multimedia content from a mobile phone, exploring the current capabilities and constraints of the existent technology. The usage environment is considered during the applications execution. A set of application scenarios is defined, according to the audiovisual nature of the content to share and to the operation environment constraints. The proposed applications also perform a constant monitoring of the network conditions to help the user in selecting the most adequate set of multimedia modalities to represent the desired contents, according to the scenarios previously defined. The operating systems chosen to develop the applications were RIM’s (Research in Motion) BlackBerry, and, due to limitations in the available BlackBerry APIs (Application Programming Interface), also Windows Mobile has been used.
The main contributions of this work include the proposal of solutions for sharing multimedia content from a mobile phone, as well as making available some guidelines and code samples for the development of mobile multimedia sharing applications.
In section 2 an overview of the literature related to multimedia sharing from a mobile device is presented. Section 3 proposes a set of relevant application scenarios and multimedia modalities that may be provided by a mobile phone. The mobile networks typically available for the majority of mobile phones and the existing methods for transferring data between a mobile phone and a web server are presented in section 4. The proposed system for sharing multimedia content from a mobile phone is described in section 5, the performance analysis of the proposed applications is presented in section 6 and conclusions are drawn in section 7.
2 Related Work
In this section some of the available literature related to multimedia content sharing over mobile networks from a mobile phone is briefly reviewed, focusing on video streaming, which has been receiving considerable attention by the academic community.
Akkanen [1] and Xie [2] proposed systems supporting peer-to-peer live video streaming over Wi-Fi, for handheld devices based on Symbian OS (operating system). In the system described in [1], the video stream generated by the mobile phone camera is encoded using the MPEG-4/H.264 standard format and
then transmitted, using the UDP transport protocol, over Wi-Fi, to listening phones within the same wireless LAN, where it can be viewed. In the system proposed by Xie, although the mobile devices exchange video streaming data among them, the content is generated and firstly delivered by a local media server, using the TCP transport protocol. The media server also runs the tracker, managing the communication between peers.
Faichney [3] and Yu [4] developed several video codecs to be used in videoconferencing for mobile devices running Windows Mobile OS. Some interesting solutions related to sharing content to (not from) a mobile device were proposed by Lundan [5][6], Haratcherev [7], Basso [8], Weber [9] and Kyriakidou [10].
Given the existing limitations shown by the BlackBerry APIs, which present several camera related constraints such as not allow accessing a video file while it is being recorded, for the OS version available to develop this work, video streaming is not one of the multimedia sharing modalities considered. As we shall see in the next section, several other multimedia sharing modalities are proposed in this paper.
3 Multimedia Modality Switching and Application Scenarios
Mobile devices are terminals for which the ability to provide different presentations for the same content may be crucial, given their operational constraints and limited resources. Moreover, a user may desire to switch among the different content presentations available on the mobile phone, e.g., selecting the most appropriate multimedia modalities given the operation conditions, to maximize the Quality of Experience (QoE) [11]. Hence, the factors that may influence the decision to provide a given content presentation, can be clustered into five main classes [12][13]:
1) User preferences, reflecting the user’s interest in using different modalities;
2) Terminal capabilities and limitations, reflecting the device’s display, capture, processing, memory and software capabilities;
3) Network characteristics, reflecting the limitations often found in wireless channels, such as interference, fading, multipath, mobility and traffic load, which lead to constrained transmission power (error rate) and bandwidth (data rate), and influence Quality of Service (QoS) parameters such as latency, jitter and packet loss;
4) Surrounding environment, reflecting the influence of the usage environment, such as noise or illumination, on the modality selection;
5)Content characteristics, reflecting the content suitability to be represented using alternative modalities. To identify the modalities that can be used in each situation and to ease their selection for representing the contents, a set of application scenarios are proposed in this paper, depending on a series of dimensions related to the content characteristics and to the operation environment constraints.
Table 1 presents the proposed scenarios classified according to the identified dimensions. Table 2 establishes a relationship between the application scenarios and the modalities that may be selected for usage.
Table 1. Proposed scenarios.
Scenario Event Duration Real-Time
Transmission Visual Content Audio Content Application Example
A One time - Yes - Photography
B Continuous Yes Yes Yes Sports match
C Continuous Yes Yes No Surveillance
D E F G Continuous Continuous Continuous Continuous Yes No No No No Yes Yes No Yes Yes No Yes VoIP Social networking Slideshow Audio interviews Table 2. Relationship between the scenarios and the modalities.
Modality A B C D E F G Video Streaming Photo Streaming Audio Streaming Video Uploading Photo Uploading Audio Uploading x x x x √ x √ x x x x x √ √ x x x x √ x √ x x x x x x √ x x x x x √ √ x x x x √ x √
Streaming and uploading represent two different ways of sharing multimedia content. Streaming involves constantly capturing and sending content to the end users, while uploading refers to capturing the whole content and then sending it to the end users at once. Notice that the best modality to represent a specific scenario may not be the most resource consuming one. For example, in scenario C, photo streaming is probably the most adequate, although video streaming can be used as well.
4 Mobile Networks and Protocols
Transferring data between a mobile phone and a backend server is one of the main actions performed in mobile communications nowadays. The networks allowing data services on mobile phones include GPRS, EDGE and UMTS, specifically designed for mobile phones, and wireless LANs as Wi-Fi/IEEE 802.11.
For the applications targeted in this paper, data rate is the main distinguishing factor between mobile networks. For comparison purposes, Table 3 summarizes the theoretical peak downlink and uplink rates for the above mentioned mobile networks.
Table 3. Comparison of mobile networks data rates [14].
Mobile Network Downlink [Mbit/s] Uplink [Mbit/s]
GPRS 0.171 0.171
EDGE 0.474 0.474
UMTS 2 0.768
Wi-Fi 54 54
According to Table 3 it is clear that Wi-Fi has an advantage when compared to the other mobile networks, notably for upstream communications, given that it provides much higher data rates in the uplink direction, at a lower cost. However, Wi-Fi also has several disadvantages, such as a more limited coverage area and the fact that it was designed for still or slow moving users, while the remaining networks support fast moving users.
The most common methods for transferring data between a mobile phone and a web server are: 1) Streaming protocols, such as RTP (Real-time Transport Protocol) and RTSP (Real-time Streaming Protocol); 2) HTTP Post (HyperText Transfer Protocol); 3) Socket connections; 4) Remote procedure calls(RPCs), such as Java Remote Method Invocation (RMI); 5) Web services.
HTTP Post and Web Services, unlike socket connections and RPCs, establish connections typically over port 80, which usually is not blocked by firewalls, and unlike the streaming protocols mentioned, have APIs available to support the development of applications where mobile phones act as data sources. For these reasons, HTTP Post and Web Services were selected to transfer data in the uplink direction, for the mobile multimedia sharing solutions proposed in this paper.
5 Sharing Multimedia from a Mobile Phone
The mobile multimedia sharing solutions proposed in this paper support four multimedia modalities: 1) Video uploading; 2) Photo uploading; 3) Audio uploading; 4) Photo streaming.
These solutions are implemented using a multimedia uploading application – MobiShare – developed for the BlackBerry OS [15], and a photo streaming application – MobiStream – developed for the Windows Mobile OS [16]. Figure 1 illustrates the relationship between the multimedia modalities implemented and the application scenarios supported in the proposed system.
Fig. 1.Relationship between modalities and application scenarios supported.
5.1 MobiShare: Multimedia Uploading Application
The MobiShare application involves three main elements: the mobile phone, the web server and the client web browser, as illustrated in Figure 2.
Fig. 2.MobiShare multimedia uploading application architecture. In the mobile phone, the main application components are:
1) Modality selection – within the application home screen, the user can choose among the available multimedia uploading modalities: video, photo or audio;
2) Start Camera/Microphone – according to the user’s choice, the device’s embedded recorder application is started;
3) Record and encode– the multimedia file is encoded and saved using as file format jpg, 3gpor amr, whether it is a photo, a video or an audio clip, respectively;
4) File Access– the file is detected in the file system and accessed;
5) Upload– the file is uploaded to the web server, using either HTTP post or web services, over one of the available mobile networks: Wi-Fi, UMTS, EDGE or GPRS.
Once the upload is complete, the mobile user is notified of the file’s URL and the application returns to the home screen.
In the web server, the received file is saved into a shared folder, whose contents are publicly available. Finally, the client can access the file from any device connected to the internet, through its URL, using a compatible web browser.
Figure 3 includes snapshots of the Blackberry screen while running the application, as well as of a web browser playing a video captured using MobiShare.
a) b)
c) d)
e) f)
Fig. 3.MobiShare’s home screen (a). Dialog box to input a name of the file to share (b). Video camera recorder (c). Upload progress bar (d). Web server’s file URL(e). Web browser playing a video file captured by MobiShare (f).
MobiShare constantly monitors the available networks, launching a dialog box whenever the network conditions change, notifying the user that different multimedia modalities can be selected, according to the application scenario, as shown in Figure 4.
a) b)
Fig. 4. MobiShare’s dialog boxes displayed when network conditions change and, as a consequence, some modalities are not recommended (a) or now supported (b).
5.2 MobiStream: Photo Streaming Application
The MobiStream application supports the streaming of a sequence of photos. This application architecture involves the mobile phone, the web server and the client application, as illustrated in Figure 5.
Fig. 5.MobiStream photo streaming application architecture. MobiStream’s mobile phone main components are:
1) Create Home Screen – The camera panel, the start button and the stop buttons are created and a streaming ID is generated and displayed;
2) Start Camera– using the start button, the video camera starts running;
3) Capture and Encode Frame – frames are captured and encoded, at a frame rate depending on the available mobile network;
4) Upload – the file is uploaded to the web server using HTTP post over one of the available mobile networks: Wi-Fi, UMTS, EDGE or GPRS.
In the web server the file is made publicly available in a shared folder. Finally, the client application’s main components are:
1) Ask for the Streaming ID– a windows form is displayed asking the client to input the desired content’s Streaming ID.
2)Open Web Browser– a web browser is used to open the URL associated to the Streaming ID; 3) Display Image– the image is displayed on the web browser;
4)Refresh Web Browser– the web page is reloaded to retrieve the most recent image available on the server.
MobiStream constantly monitors the available mobile networks, adapting the frame rate to the current network characteristics, in order to provide the end user with the best QoE possible. Figure 6 illustrates the MobiStream application interface on the mobile device and on the client’s computer.
a) b) c)
Fig. 6. MobiStream mobile phone’s screen (a). Windows form asking for the Streaming ID (b) and containing the web browser where images are displayed (c).
6 Performance Analysis
A set of tests was performed to evaluate each of the applications, taking into account their different functional structures and supported scenarios. The main characteristic evaluated in MobiShare is the upload time, i.e., the time interval from the beginning of a file transmission until the moment when that file is completely received in the web server and made available to be accessed by a client. In
MobiStream, the main characteristic evaluated is the average frame period, expressed as the average time interval between two consecutive photos being stored in the web server. Additionally, MobiStream’s streaming latency, computed as the average photo upload time, is also estimated.
MobiShare tests were conducted using a BlackBerry Curve 8900 with the 4.6.1 OS version and GPRS/EDGE/Wi-Fi enabled. Table 4 presents the average upload times of photos, audio clips and video clips over the mentioned networks.
Table 4. MobiShare’s upload times. Mobile Networks Photo
(55 kB) Audio clip(3min 30s, 328 kB) Video clip(20 s, 1.3 MB)
GPRS 29 s 2 min 45 s
-EDGE 18 s 1 min 40 s
-Wi-Fi 4 s 15 s 1 min 17 s
The main conclusion drawn from the tests is that MobiShare’s performance over Wi-Fi is much better than over GPRS and EDGE, and this difference increases as the size of the file shared increases. Moreover, the performances using HTTP Post or Web Services are globally similar, regardless of the selected multimedia modalities, file sizes or mobile networks considered. The attributes of the recorded content influence the size of the files, and consequently, the upload time, especially for photos and video. In fact, given the characteristics of the photo and video codecs used to compress the files, JPEG and MPEG-4 (which is used in the 3gpfile format), respectively, the size of the file increases as the details of an image increase and as the motion of a video increases. The observed upload times are acceptable according to the requisites of the corresponding application scenarios, proving the application usefulness. MobiStream tests were conducted using a HTC Touch Pro running Windows Mobile 6.1 and GPRS/EDGE/UMTS/Wi-Fi enabled. Table 5 presents the average frame periods over the available mobile networks. Figure 7 illustrates the typical latency values measured when streaming a sequence of photos over Wi-Fi.
Table 5. MobiStream’s average frame periods.
Wi-Fi UMTS EDGE GPRS Average frame period [ms] 764 1782 3753 4203
0 20 40 60 80 100 120 140 160 180 200 0 5 10 15 20 25 Number of Frames La te nc y [s ] HTTP Post Web Services
Fig. 7.Variation of the latency over a sequence of photos, using HTTP Post and Web Services, over Wi-Fi.
Concerning the average frame reception period, measured in the web server, the analysis of the results presented on Table 5 show that, as expected, MobiStream’s performance over Wi-Fi is much better than over the remaining networks, with the minimum achievable average frame period being substantially lower. The consequence is that when using Wi-Fi it is possible to work with higher frame rates. In the best case scenario, the frame rates provided are 3 images every 2 seconds over Wi-Fi, 2 images every 3 seconds over UMTS, 1 image every 3 seconds over EDGE and 1 image every 4 seconds over GPRS. These values prove the application usefulness, given their suitability to the scenario C characteristics, that is, for surveillance systems, for example, as described in section 3. Still regarding the average frame period, HTTP Post and Web Services provide similar performances over the available mobile networks.
As far as the latency is concerned, an analysis of the results presented in Figure 7 show that the delay for transmitting the initial photos of the sequence is higher for Web Services, due to an initial set up and connection details caching done by the Windows Mobile OS. As HTTP Post does not require this initial set up, its initial delays are lower, around 3s or 4s, and they remain approximately constant over time and equal to the Web Services delay values after this initial behavior. Therefore, it is possible to conclude that HTTP Post provides a better real-time experience, since the latency of the initial photos of the sequence has a major influence in the time delay observed between the capture of a photo and the moment of its display by the client application. Given the characteristics of the application scenario supported by MobiStream, scenario C, which aims for a real-time experience, HTTP Post should be the data transfer protocol implemented.
The network conditions may have a severe impact in the tests performed for both implemented applications. Given that Wi-Fi and specially UMTS, EDGE and GPRS are public networks, accessible to everyone, the data rates available are affected by the network traffic load, that is, by the number of users and the amount of traffic produced by them, among several factors. Hence, the randomness associated to it cannot ensure that the network conditions do not influence the results. To minimize this influence, several rounds of tests were conducted, allowing to reach a more representative average value.
7 Conclusion
The multimedia sharing solution presented in this paper implements five of the application scenarios defined in section 2: MobiShare implements scenarios A, E, F and G; and MobiStream implements scenario C. The multimedia modalities supporting these scenarios are: video uploading, photo uploading and audio uploading, in MobiShare, and photo streaming, in MobiStream. The mobile usage environment analysis, by taking into account the application scenarios and monitoring the network conditions, aims to provide the end user a maximized QoE, by recommending the user to switch among modalities, whenever appropriate, or by changing the streaming frame rate according to the network conditions.
Additionally, a goal of this work is to share guidelines and code samples useful for the development of mobile multimedia sharing applications – some material is available at
http://www.img.lx.it.pt/~plc/temp/guidelines.pdf.
Future work directions include the development of applications supporting video streaming. This requires solving some of the existing API constraints related to video camera access and to implementation of streaming protocols in the mobile devices.
Acknowledgement
The authors acknowledge the support of Fundação para a Ciência e Tecnologia (FCT).
References
1. J. Akkanen, O. Karonen, and J. Porio, "Peer-to-Peer Video Streaming on Mobile Phones," in 5th IEEE Consumer Communications and Networking Conference, Las Vegas, USA, 2008, pp. 1253-1254.
2. S. Xie, B. Li and G. Keung, “The Peer-to-Peer Live Video Streaming for Handheld Devices”, 5thIEEE Consumer Communications and Networking Conference, Las Vegas, USA, 2008, pp. 1265 – 1266
3. J. Faichney and R. Gonzalez, "Video Coding for Mobile Handheld Conferencing," Multimedia Tools and Applications, vol. 13, no. 2, p. 165–176, Feb. 2001.
4. K. Yu, J. Lv, J. Li, and S. Li, "Practical real-time video codec for mobile devices," Proceedings of the International Conference on Multimedia and Expo, vol. 3, pp. 509-512, Jul. 2003.
5. M. Lundan and I. Curcio, "3GPP streaming over GPRS Rel '97," in International Conference on Computer Communications and Networks, Tampere, Finland, 2003, pp. 101-106.
6. M. Lundan and I. Curcio, "Mobile Streaming Services in WCDMA Networks," in IEEE Symposium on Computers and Communications, Cartagena, Spain, 2005, pp. 231-236.
7. I. Haratcherev, J. Taal, K. Langendoen, R. Lagendijk, and H. Sips, "Fast 802.11 link adaptation for real-time video streaming by cross-layer signaling," in IEEE International Symposium on Circuits and Systems, ISCAS, Kobe, Japan, 2005, pp. 3523-3526.
8. A. Basso, B. J. Kim, and Z. Jiang, "Performance evaluation of MPEG-4 video over realistic EDGE wireless networks," in The 5th International Symposium on Wireless Personal Multimedia Communications, Honolulu, Hawaii, USA, 2002, pp. 1118-1122.
9. R. Weber, M. Guerra, S. Sawhney, L. Golovanevsky, and M. Kang, "Measurement and Analysis of Video Streaming Performance in Live UMTS Networks," in 9th International Symposium on Wireless Personal Multimedia Communications, San Diego, USA, 2006.
10. A. Kyriakidou, N. Karelos, and A. Delis, "Video-streaming for Fast Moving Users in 3G Mobile Networks," in
International Workshop on Data Engineering for Wireless and Mobile Access, Baltimore, USA, 2005, pp. 65-72. 11. F. Pereira, "Multimedia Content Adaptation: May one fit all?," Computational Imaging and Vision, vol. 32, pp. 337-342, Mar. 2006.
12. T. C. Thang, "Modality Conversion for QoS Management in Universal Multimedia Access" IEE Proceedings -Vision, Image and Signal Processing, vol. 152, no. 3, Jun. 2005.
13. A. Vetro, "MPEG-21 Digital Item Adaptation: Enabling Universal Multimedia Access" IEEE Multimedia, vol. 11, no. 1, pp. 84-87, 2004.
14. Rysavy Research, "EDGE, HSPA and LTE: The Mobile Broadband Advantage" 3G Americas, pp. 34-35, Sept. 2007.
15. RIM, “Development Guide – BlackBerry Java Development Environment”, Version 4.6.1, Dec. 2008. 16. A. Wigley, “Microsoft Mobile Development Handbook”, Microsoft Press, 2007.
