Available at http://www.Jofcis.com
Research and Improvement on A Lightweight VOD Cloud
Storage Engine
Chang SHU
∗
,
Xingming ZHANG
School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Abstract
Cloud storage is a new concept which has expanded and developed from cloud computing. Despite being a relatively new concept, it already has a wide range of applications. In recent years, researchers in the field of cloud storage have focused on cloud computing’s storage capabilities and the structure of its levels of improvement. However, comparatively less attention has been given to the specific application environment of cloud storage needs, especially in VOD (Video-On-Demand), resource storage, and business applications; the cloud computing needs of cloud storage solutions have not been addressed effectively as well. Therefore, the present study provides an in-depth and detailed analysis of the current application of common cloud storage design ideas and the implementation method, including its performance advantages and disadvantages, focusing on its resource sharing capability and rapid low-cost building capacity. A lightweight cloud storage engine design suitable particularly for the storage of multimedia resources is proposed. This proposed design is based on the current mainstream open source cloud storage system, focusing on cloud storage engine architecture, communication protocols, synchronization mechanisms, and the positioning mechanism of block storage files. Finally, a simulation environment for the comparison test shows that the system is stable and highly available.
Keywords: Cloud Computing; Lightweight Engine; Cloud Storage; Vod; Load Balancing
1
Introduction
In recent years, major companies have introduced a variety of cloud storage techniques [1-4], and related products online offer cloud storage services; however, hardware costs and the complexity of large software architectures bring a certain degree of difficulty in the field of cloud storage ap-plication. Given current technical conditions, the spread of cloud storage application must rely on clustering technology [5-6], storage virtualization technology [7-8], distributed file systems [9-10], and other key technology solutions. Researchers on storage cloud computing are often more con-cerned with cloud computing storage capabilities and the structure of the level of improvements. Hao and Chuohao made a film based on the automatic detection of modified data to trigger the sync, a strategy to improve the efficiency of synchronization [11]. Nepal, Chan, and Moreland
∗Corresponding author.
Email address: [email protected] (Chang SHU).
1553–9105 / Copyright© 2012 Binary Information Press January 2012
proposed the adoption of virtual storage architecture in the design of a simple XML-based allo-cation of storage interface [12] to achieve scalable distributed storage service purposes. Likun, Yongwei, and Guangwen designed a storage architecture based on Zetta Data, proven by exper-iments to increase the throughput and reduce the resource consumption of the effective storage cluster [13]. Talattinis and Hadoop conducted commercial data processing of a large number of experiments [14], which was followed by An and Dandan who also focused on the cloud-based storage solutions Hadoop studied [15]. However, these researchers are less concerned with the specific application environment of cloud storage needs, especially in VOD, resource storage, and business applications, and particularly the cloud computing needs of cloud storage solutions which have not been addressed effectively to date. Compared with general public storage requirements on the Internet, these applications have a greater need for resource sharing capability and rapid low-cost building capacity.
Therefore, the present work provides an in-depth and detailed analysis of the current application of common cloud storage design ideas and the implementation method, including its performance advantages and disadvantages, focusing on its resource sharing capability and rapid low-cost building capacity. A lightweight cloud storage engine design (LiDFS) particularly suitable for the storage of multimedia resources is proposed. The proposed design is based on the current mainstream open source cloud storage system, focusing on cloud storage engine architecture, com-munication protocols, synchronization mechanisms, and positioning mechanism of block storage files. Finally, a simulation environment for the comparison test shows that the system is stable and highly available.
2
A Lightweight Cloud Storage Engine
2.1
Design and system architecture
The cloud storage system abstracts business computing and stores resource physical mapping. It can allocate storage based on business resources dynamically. It is highly scalable, highly reliable, low-level transparent, and secure; it has a common underlying architecture of monitoring and service interface. Typically, a standard cloud storage system at least includes the following levels: access layer, application interface layer, basic management and storage layer, and different systems with different complexity levels for different cloud storage service targets contained in all levels. The present article draws on common cloud storage architecture, and basing on the relaxation coupled asymmetric structure framework, it proposes a special class of multimedia resources for the video store, which is a lightweight cloud storage engine framework called “LiDFS.” The system logical framework is shown in Figure 1.
In the above figure, the controller (clusters) is responsible for scheduling work and has a load-balancing effect for access connections. The storage node in the cluster group and status in-formation will be recorded in the memory, which is a hub connecting the client to the storage node. All servers are equal; no Master-Slave relationship is required to avoid a single point of failure effectively. The storage cluster is assembled as group based; the same groups of files on the storage nodes are identical (RAID 1). All storage nodes files and file attributes (meta data) are saved to their own nodes. Under the policy of storage, the active node status information is reported regularly to the controller; different groups of storage servers do not communicate with one another. The simple working mechanism of the proposed lightweight system reduces the load
Fig. 1: System logical framework
substantially, thus improving access efficiency effectively.
2.2
Design of communication protocols and access procedures
For any storage system, reading the data storage device of the system bandwidth is an important factor in effective throughput. Developing a large-scale video network service system design and improving the system performance and memory bandwidth remain a bottleneck in terms of resources; appropriate optimization strategies must be adopted to improve the rate of data retrieval and transmission and thereby achieve the objective of improving the overall system throughput. Therefore, GoogleFS [16], MongoDB [17], and other general designs of a distributed storage system for the engine are utilized, focusing on inter-module communication protocol from the file access protocol and design improvements.
(1) Protocol Format
The communication protocol is based on TCP/IP, with the protocol package consisting of two parts: the header and the body. The header is 10 bytes in size, and the format is as follows:
Table 1: Communication protocol format
Header(10 bytes) Body(N) body length (8 bytes) command(1 byte) status(1 byte) · · ·
The body in the packet format depends on the command, and it can be empty. (2) File Storage and Reading Process
Fig 2: File storage process Fig 3: File storage process
• The client asks thecontroller to u-pload to the storage node.
• The controller returns an available
storage node.
• The client and the storage node
communicate directly and complete the file upload; the storage node returns the file ID.
• The client has the priority to ob-tain the download path directly from the file ID (priority to download from the initial group, the source n-ode; if the node fails or refuses, the
controller has to be asked for other effective nodes.
• The client asks the controller for a storage node which can download the specified file; the argument is the
file ID (group name and file name).
• The controller returns an available
storage node.
• The client and storage node com-municate directly to complete the file download.
2.3
Improved storage mechanism based on peer grouping
Typically, for a large-scale VOD application, resource files are often stored on an NAS/SAN-based multi-level storage system. In recent years, with the development of video-on-demand application technology, some researchers made improvements on the storage structure and scheduling algo-rithm. For instance, Sanchez developed the MAPFS-Grid architecture [18]. Yang et al. proposed a P2P-based metadata storage cluster [19]. However, although these studies improved effectively the efficiency of resource access, their contributions are too hardware dependent, thus reducing business expansion significantly. For example, when faced with a large-scale video-on-demand which reads requests rather than writes them, and a single file size is very large (a 100-minute 720 p standard definition video is about 1.5G), the storage requirement would be unimaginable. Although HadoopDFS [20], GoogleFS [16] and other such distributed file systems can provide good scalability and compatibility, their complex structure and file block storage overhead caused by additional indexing are not suitable for video resource storage. Therefore, for such
application-s, the expansion capacity of storage systemapplication-s, the cost, and the compatibility of factors involved need to be considered.
(1) Storage Strategy
According to the design of the engine, the destination node of the uploaded file needs to be selected by the controller. Uploaded files generate a file ID, which makes downloading the same file easier for the client; the client can have priority access to the download path based directly on the file ID (priority to download from the initial group, the source node). If the node fails or refuses, the controller has to be asked for another effective node. To improve the hit rate in reading and spreading the connection of each node request as much as possible (because the time it takes to read a file is often too long; scattering reading services to different nodes can improve substantially the storage quality of the service), requesting files many times should be avoided in the same disk storage.
Video files tend to use the traditional block storage strategy. Although it offers increased flexibility from the perspective of file scheduling and content distribution, the cost of the system resources of the server is indeed improved substantially. Further, the complexity of the algorithm also makes the storage system difficult to extend and construct rapidly. Therefore, the lightweight storage engine does not offer file block storage compared with simpler and more efficient engines. It can also meet the actual needs of the vast majority of Internet applications.
Taking all these factors, the various nodes are defined within the same group as the parameter index (because the upload Group ID is designated generally by business applications; different sets of parameters are not compared here):
• Node disk I/O maximum bandwidth W.
• The node has access to the file transfer bandwidth
• Node CPU utilization Pc.
• Node Memory utilization Pm.
Then the current node real-time performance indicatorsσ =
Pn i=1Ai
W·P c·P m (1) are calculated, where
describes the value of the greater and better performance of the node. Therefore, the storage engine will be stored in two different business strategies to meet the types of services:
Strategy No. 1: Always upload file to a specific group of nodes with the best performance, according to indicators, that is ,t. The node’s CPU, memory, disk I/O, and other parameters are the best among all nodes in this group.
Strategy No. 2: Always upload file to a specific group of nodes whose performance indicators are the worst, that is . The node’s CPU, memory, disk I/O, and other parameters may not be the best among all nodes in the group, but they are the least busy.
Next, the file read service hit rate is calculated for these two strategies:
Assume the storage node to support I/O connections is the number N, and the file storage transfer time is T. Users request service randomly. Suppose the requested time of user i is ; the request before N allows users to obtain the service immediately. If before the N + 1 user request, the first user’s service is completed, that is , then the N + 1 requests can obtain a response immediately; otherwise, time interval of the request will be denied. After, the first user connection is released, and the system can now respond to new requests.
This way, the file read request can be considered approximately as many consecutive cycles; the length of each cycle t1+T, can be divided into the “acceptable” and “rejection” stages. The
length of the “acceptable” stage is tN, and the length of the “acceptable” stage is t1 +T −tN.
For the length of time in t, the average request isλ·t, and the average is. Therefore, for reading services, the rejection rate can be expressed as follows:
ρ1 =N umber Rejected/T otal N umber of Requests
=λ·Rejected T ime/λ·T otal Request T ime
= (t1+T − N λ)/( 1 λ +T) = (1 +λ·T −N)/(1 +λ·T) = 1−N)/(1 +λ·T) (2)
When T increases, ρ1 increases, and when T decreases, ρ1 decreases. Therefore, to reduce the
rejection rate of reading services,T must be decreased orN increased. The main factors affecting
T are the performance parameters of the node itself, the client connection times, file size, and bandwidth, among others. Figures 2 and 3 show the engine’s file access process. When the client uploads a file, the file ID is not specified by the client, but the storage nodes help generate and return it to the client. The file ID contains a group name, the file relative path, and the filename, as shown in Table 2. When the client needs to read the file, it can be downloaded directly from the file ID to obtain the path which navigates to the file (priority to download from the initial group, the source node). If the node fails or refuses, the client can try to ask for another effective node from the controller.
Table 2: Uploaded file name definition
File ID
group name disk name folder name file name
Therefore, pictures and documents for the class of small files usually take storage Strategy No. 1. This method can guarantee effectively the quality of service by high-performance nodes and can reduce the rejection rate of the file read service. For the storage file of video resources of large files, reading takes a long time because of the connection, and the connected state should be maintained; further, reducing effectively the value of T is difficult. Therefore, storage Strategy No. 2 should be adopted for such documents. According to the policy of increasing duplicate numbers, the N value should be increased as much as possible so that when a large number of file reads are requested, the service rejection rate will be reduced effectively.
(2) File Synchronization and Storage Expansion
The engine is used in grouping identical files on the storage server (RAID 1), so different groups of storage node do not communicate with one another. Based on a lightweight design, the controller’s configuration file does not appear in the storage node, and the storage node
configuration file will list all of the controllers. This rule indicates that the connection between the storage node and the controller will be initiated by the former. The storage node will start a thread connection and communicate with each controller.
Thecontroller will then store the classification in the memory andstorage nodes under various groups; the storage connects nodes through their own groups to a file. Therefore, the service can
be restarted directly by obtaining information from local disk stores. The storage node will be recorded in the memory of all servers in this group, and the server’s log information will be saved to the file. Thecontroller and the storage node will be synchronized with eachstorage node list. When adding a new controller, the storage node connected to the controller finds that the latter returns a list of storage nodes fewer than the local records. Then the storage node will be synchronized to the controller.
The same groups ofstorage nodes are equivalent; file upload, deletion, and other operations can be carried out in anystorage node. The file synchronization within the same group is only between
storage nodes; using the push mode, the source server is synchronized to the target server. In a file upload, a group may have three storage nodes A, B, and C. File F is uploaded to server B, and the fileF from the remaining two servers synchronize A and C. The operation that uploads file F to server B may be called source operation. File F in server B is the source data file. It is synchronized to the operation of servers A and C as a backup operation; fileF in A and C is backup data. The synchronization rules are summarized as follows:
• Synchronization occurs only in the group between storage nodes.
• Only the source data require synchronization; backup data do not require re-synchronization.
• The second rule has an exception: when adding a new storage node, one of the existing storage nodes will have all the data (including the source and backup data) synchronized to the new server.
The storage node is set in seven states:
• FDFS STORAGE STATUS INIT: It has not yet been synchronized with the data source server.
• FDFS STORAGE STATUS WAIT SYNC: The source of the existing data synchronization server is awaiting synchronization.
• FDFS STORAGE STATUS SYNCING: In synchronization
• FDFS STORAGE STATUS DELETED: The server is removed from the group.
• FDFS STORAGE STATUS OFFLINE: Offline.
• FDFS STORAGE STATUS ONLINE: Online; cannot provide services.
• FDFS STORAGE STATUS ACTIVE: Online; can provide services.
When the storage node status is FDFS STORAGE STATUS ONLINE, and the storage node initiates a heartbeat to the controller, the controller changes the status to FDFS STORAGE
STATUS ACTIVE.
When adding a new storage node, the system will complete automatically the existing data synchronization processing logic as follows:
Fig.2: Process of node add-ins
• Storage node A connects to the controller, and the controller sets storage node A’s status to FDFS STORAGE STATUS INIT. Storage node A asks the source server for additional synchronization time. If the group only has storage node A or the group has uploaded 0 files successfully, the data do not need to be synchronized, and storage node A can provide online services. At this time, the controller sets its status to FDFS STORAGE STATUS ONLINE; otherwise the controller will set its state FDFS STORAGE STATUS WAIT SYNC into the second step of the treatment.
• If the controller assigned to storage node A synchronizes existing data from source storage node B, other storage nodes in the same group will communicate with the controller; they will know that storage node A is added and will start the synchronization thread. They will ask the controller for the source data server and the synchronized deadline of storage node A. Storage node B will synchronize all data to storage node A before the synchro-nization deadline. The rest of the storage node will start normal synchrosynchro-nization from the synchronized deadline; only the source of the data synchronization will be synchronized with storage node A. After the deadline for storage node B to synchronize data to storage node A, additional synchronization will be switched to normal, and only the source data will be synchronized.
• After storage node B synchronized all of the data to storage node A, no more data need to be synchronized. Storage node B can request the controller to set the state of storage node A to FDFS STORAGE STATUS ONLINE;
• When storage node A launches a heartbeat to the controller, the controller will set the status to FDFS STORAGE STATUS ACTIVE;
Based on the above design of synchronization, the expansion mechanism can achieve flexibly the storage engine capacity increase. When a group’s storage server access pressure is large, the storage server in this group can expand service capabilities (vertical expansion). When the system capacity is insufficient, the group can expand storage capacity (horizontal expansion). This condition will not only ensure the stability of the storage service but will also improve substantially the system scalability.
File synchronization is asynchronous, so some cases can cause synchronized delays. To avoid the situation wherein a newly uploaded file cannot be read in a storage node that has not yet been synchronized, the controller turns need to be set to storage node selection (round-robin). When the client asks the controller storage node for the file to be downloaded, the controller returns the following four conditions to satisfy one of the storage nodes:
• The file is uploaded to the source storage node; files are directly uploaded to the server.
• File creation time stamp < the storage node is synchronized to the file timestamp. This means that the current file has been synchronized.
• File creation time stamp = The storage node is synchronized to the file timestamp, (Current timeFile creation time stamp) >A file synchronization requires the maximum time (e.g., 5 minutes);
• (Current time File creation time stamp)>File synchronization delay threshold such as the set threshold value of 1 day; the file synchronization must be completed in one day.
This approach avoids not only file synchronization delay but also timing problems in updating the storage node in more than one file.
3
Experiment and Analysis
3.1
Test model
In testing, different storage engines were used based on Gigabit Ethernet (default configuration includes three storage nodes). Using Load Runner as the load generator, a large number of pre-written scripts were employed to test the performance of the client. Access to a large number of connections is needed to test streaming server performance. These connections to access the file content according to distribution and the distribution of service request time intervals should be defined in different spatial models (access content model) and time models (access time of interval model).
• Content access model
To verify the generality of the storage system and to simulate the client’s access behavior, the test access to the target file in the distribution followed uniform distribution, that is, all simulated client visits were distributed evenly in all of the documents. Meanwhile, to explain better the storage engine in terms of the effect of different application types of support, the document test was classified as follows:
Table 3: Classification of test file
File Size Number of files Group1 0-200KB 100000 Group2 200KB-10MB 2000 Group3 10MB-500MB 100
The testing and storage engine was connected in a short manner, and the file upload was repeated for a certain type of file (such as type 0C200 KB). Uploaded files that reached a specific number (e.g. 100,000) were no longer uploaded.
• Time access model
What time of the terminal server access, will affect the server performance. The start of the session in the timeline according to the distribution of the interval was designed to be uniform, following the Poisson distribution model.
In uniform distribution, the number of clients is at a fixed time interval, request to the server one by one. This is an ideal model; a relatively realistic model is the Poisson distribution, which has statistics and probability similar to the discrete probability distribution. It is expressed as follows:
P{ξ=k}= λ
k
k!e
−λ, k = 0,1,· · · (3)
Where λ is the request rate, and K is the number of simulated clients.
3.2
Aggregate throughputs
To test the performance of file reading and writing based on different protocol architectures, the present study conducted simulation experiments comparing GoogleFS, Hadoop, mongoDB, and LiDFS classification of the various data files. The results are shown in Table 4.
The analysis is shown in the image above. In the design requirements of GoogleFS, all data blocks are stored in 64 MB. Although it is effective in reducing the size of the metadata, the im-provement in file access speed was relatively small. Group1CGroup3 files’ access speed was ideal. However, the file was still used due to the block storage mode, and the Master node was used to control and manage the various data chunk nodes and to read obvious bottlenecks. Therefore, in reading and writing large files, especially in high concurrency, the read and write rates decreased obviously. The MongoDB data used a lightweight TCP/IP writing protocol, which facilitates a simple packaging above the BSON data layer. This makes the written data command include a 20-byte message header; moreover, it used a larger system level virtual memory mechanism, improving substantially the read and write performance for small files. However, it also increased severely the memory overhead of the operating system, particularly in large file access perfor-mance, resulting in a serious decline. The proposed LiDFS not only further simplified the access protocol (10 bytes) and allowed the client to read directly the file from the source storage node to some extent. It also offered effective access pressure sharing of large files to the whole system. Therefore, its large file reading performance was the best.
Table 4: File read and write performance in different protocol architecture
Write Rate Test (Gigabit LAN) Read Rate Test (Gigabit LAN)
Group1
Group2
Group3
4
Conclusions
The present study made an in-depth and detailed analysis of the current application of common cloud storage design ideas, including the implementation method and its performance advantages and disadvantages. It also focused on resource sharing capability and rapid low-cost building ca-pacity, and correspondingly proposed a lightweight cloud storage engine design (LiDFS) which is particularly suitable for the storage of multimedia resources. The design was based on the current mainstream open source cloud storage system, particularly on cloud storage engine architecture, communication protocols, synchronization mechanism, and positioning mechanism of block stor-age files which enhance the design. Finally, a simulation environment for the comparison test showed the characteristics of the engine and its suitable application scenarios.
However, the experiment on the storage system was based mainly on test results under normal circumstances. The following issues also need to be considered: engine performance in a produc-tion environment, reliability and excepproduc-tional cases in the storage node, and the performance of the storage engine. The proposed storage mechanism and other groups in the present two strategies have their advantages and disadvantages. The application is selected automatically based on file size and the rules of the second strategy. Our future work will focus on the adaptive strategy of
the present mechanism.
References
[1] http://aws.amazon.com/ 2011. 4.
[2] EMC Corporation. The VMWare software paekage [EB/OL]. http://www. vmware.com. 2011. 3. [3] Cambridge Univ.Xen [EB/OL]. http://www.cam.ac.uk/research/ SRG/netos/xen. 2011. 2. [4] RedHat Cloud Computing [EB/OL]. http://www.redhat.com/ solutions/cloud/. 2011. 4.
[5] Diane Barrett, Gregory Kipper. Visions of the Future: Virtualization and Cloud Computing. Virtualization and Forensics, 2010, pages 211 – 220.
[6] P. Padala, X. Zhu, Z. Wang, S. Singhal, and K. G. Shin, “Performance evaluation of virtualization technologies for server consolidation”, HP Techinal Report HPL-2007-59R1, 2008.
[7] Peter Zornig. Statistical simulation and the distribution of distances between identical elements in a random sequence. Computational Statistics & Data Analysis, Volume 54, Issue 10, 1 October 2010, pages 2317 – 2327.
[8] Chen Xiaojun, Zhang Jing, Li Junhuai, Li Xiang. Resource reconstruction for optimal utilization in virtual computing systems. Journal of Computational Information Systems, Vol. 7, p1364 – 1372, 2011. 4.
[9] Virtualization Technology 2010, http://www.kernelthread.com/publications/virtualization/ (ac-cessed on 05. 05. 2010).
[10] Thomas Olzak, Jason Boomer, Robert M. Keefer, James Sabovik. Integrating application and presentation virtualization (Terminal Services). Microsoft Virtualization, 2010, pages 339 – 375. [11] Hao Zhang, Chuohao Yeo, Kannan Ramchandran. Rate efficient remote video file synchronization,
2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 19 April 2009 to 24 April 2009.
[12] Shiping Chen, Surya Nepal, Jonathan Chan, David Moreland, John Zic. Virtual Storage Ser-vices for Dynamic Collaborations. 16th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE 2007), 18 June 2007 to 20 June 2007. [13] Likun Liu, Yongwei Wu, Guangwen Yang, Weimin Zheng. ZettaDS: A Light-weight Distributed
Storage System for Cluster, The Third ChinaGrid Annual Conference (chinagrid 2008), 20 August 2008 to 22 August 2008.
[14] Kyriacos Talattinis, Aikaterini Sidiropoulou, Konstantinos Chalkias, George Stephanides. Paral-lel Collection of Live Data Using Hadoop, 2010 14th Panhellenic Conference on Informatics, 10 September 2010 to 12 September 2010.
[15] An Qin, Dandan Tu, Chengchun Shu, Chang Gao. XConveryer: Guarantee Hadoop Through-put via Lightweight OS-Level Virtualization, 2009 Eighth International Conference on Grid and Cooperative Computing, 27 August 2009 to 29 August 2009.
[16] The Google File System. http://labs.google.com/papers/gfs.html.
[17] MongoDB. http://www.mongodb.org/display/DOCS/Mongo+Wire+Protocol.
[18] Alberto Sanchez, Maria S. Perez. A high performance suite of data services for grids. Future Generation Computer Systems, 26 (2010), 622 – 632.
[19] YANG Jun, ZHAO Guo-hua, WANG Ke-jia, SONG Jun-de. A modern service-oriented distributed storage solution. The Journal of China Universities of Posts and Telecommunications. 2009. 9. [20] Hadoop. http://hadoop.apache.org/.
