Experiences with Cassandra
Cloudy is based on Cassandra and uses its storage as well as its messaging system.
The development process of Cassandra was in alpha state when Cloudy was started.
As the documentation was extremly sparse a lot of the initial work was to read and
adjust parts of the code. Especially the storage system including logging and data movement was very buggy and a significant part of the thesis was to fix several bugs of this multithreaded and distributed system.
Cassandra was shortly moved from Google Code to Apache and a lot of changes and bug fixes were added. A next step is to integrate the added features of Cloudy to the new code base of Cassandra to get an overall much more stable system.
Stability
Some problems with the TCP thread pool of Cloudy limit the systems scalability.
Nodes crash if the load exceeds their capacity and the message system blocks which leads to corrupted nodes. It is also possible that the whole JVM crashes.
Cloud Bursting
At the moment Cloudy runs either local without the cloud bursting feature or on EC2 where this feature could be enabled. To also use it in a local environment, the API needs to be adjusted. This means, that it has to be defined how to start a local node given a token.
A Graphics
Boxplot
Boxplots are often used in this thesis to represent response times as the lowest value as well as the median and some kind of a confidence interval is shown for every dis-tribution. The lowest value can be viewed as a approximation for the theoretical value and the median as an approximation for the expectation.
●
●
●●
●
group1 group2
−4
−2 0 2 4 6
median 3rd quartile
1st quartile upper whisker (CI)
lower whisker (CI) outlier
Figure A.1: Simple representation of two normal distributed variables. Group1 has a N (1, 22) and group 2 a N (0, 0.72) distribution. Shown are the median and the 1st and 3rd quartile as well as the upper and lower whisker. The upper whisker is the highest data value smaller than 1.5 · IQR + 3rd quartile, with IQR the interquartile range.
B Miscellaneous
80-20 rule
The 80-20 rule used in this thesis is a definition of load distribution, which is as fol-lows.
20% produce 80% of the system’s load, where 20% of the 20% produce 80% of the 80% of the system’s load and so on. It ist a recursive definition of load distribution.
Figure B.1 show the distribution function of the 80-20 rule for 10000 different keys.
Note: the 80-20 rule confirms a pareto distribution.
0 2000 4000 6000 8000 10000
1e−05 1e−04 1e−03 1e−02 1e−01
Figure B.1: 80-20 distribution function. The y-axis is in logirthmic scale.
Bibliography
[1] “Amazon web service.” http://aws.amazon.com/.
[2] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber, “Bigtable: a distributed storage system for structured data,” in OSDI ’06: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, (Berkeley, CA, USA), pp. 15–
15, USENIX Association, 2006.
[3] B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni, “Pnuts: Yahoo!’s hosted data serving platform,” Proc. VLDB Endow., vol. 1, no. 2, pp. 1277–1288, 2008.
[4] “Cassandra: A structured storage system on a p2p network.” http://code.
google.com/p/the-cassandra-project/.
[5] R. Buyya, C. Yeo, S. Venugopal, M. Ltd, and A. Melbourne, “Market-oriented cloud computing: Vision, hype, and reality for delivering it services as computing utilities,” in Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications (HPCC-08, IEEE CS Press, Los Alamitos, CA, USA), 2008.
[6] “Twenty experts define cloud computing.” http://cloudcomputing.
sys-con.com/read/612375_p.htm.
[7] “Google docs.” docs.google.com.
[8] “Google app engine.” http://code.google.com/appengine/.
[9] “Facebook.” www.facebook.com.
[10] “Azure service platform.” www.microsoft.com/azure.
[11] “Gogrid.” http://www.gogrid.com.
[12] “Typica, java api for amazon’s ec2.” http://code.google.com/p/
typica/.
[13] S. Ghemawat, H. Gobioff, and S. Leung, “The Google file system,” ACM SIGOPS Operating Systems Review, vol. 37, no. 5, pp. 29–43, 2003.
[14] D. Karger, E. Lehman, T. Leighton, M. Levine, D. Lewin, and R. Panigrahy,
“Consistent hashing and random trees: Distributed caching protocols for reliev-ing hot spots on the World Wide Web,” in ANNUAL ACM SYMPOSIUM ON THEORY OF COMPUTING, vol. 29, pp. 654–663, THE ASSOCATION FOR COMPUTING, 1997.
[15] D. Eastlake and P. Jones, “US secure hash algorithm 1 (SHA1),” 2001.
[16] D. Karger and M. Ruhl, “Simple efficient load-balancing algorithms for peer-to-peer systems,” Theory of Computing Systems, vol. 39, no. 6, pp. 787–804, 2006.
[17] A. Rao, K. Lakshminarayanan, S. Surana, R. Karp, and I. Stoica, “Load balancing in structured p2p systems,” Lecture notes in computer science, pp. 68–79, 2003.
[18] M. Burrows et al., “The Chubby lock service for loosely-coupled distributed sys-tems,” in Proc. of the 7th OSDI, 2006.
[19] M. Abdallah and H. Le, “Scalable range query processing for large-scale dis-tributed database applications,” -, -.
[20] I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan, “Chord:
A scalable peer-to-peer lookup service for internet applications,” in ACM SIG-COMM 2001 Conference, Aug. 2001.
[21] G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels, “Dynamo: amazon’s highly available key-value store,” in SOSP ’07: Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles, (New York, NY, USA), pp. 205–220, ACM, 2007.
[22] A. Hermann, “Cloudy, a cloud storage system with adaptive transaction guaran-tees,” Master’s thesis, ETH Zurich, Feb. 2009.
[23] B. Bloom, “Space/time trade-offs in hash coding with allowable errors,”
[24] N. Hayashibara, X. Defago, R. Yared, and T. Katayama, “The/spl phi/accrual failure detector,” in Reliable Distributed Systems, 2004. Proceedings of the 23rd IEEE International Symposium on, pp. 66–78, 2004.
[25] M. Welsh, D. Culler, and E. Brewer, “SEDA: An architecture for well-conditioned, scalable internet services,” ACM SIGOPS Operating Systems Re-view, vol. 35, no. 5, pp. 230–243, 2001.