C.1 In-laboratory
Within the performed experiments data replicas located in remote sites are con-nected by a Local Area Network. Each server holds a copy of the replicated database and an instance of the middleware.
Figure C.1: Laboratory environment for Theta replication implementation
Clients applications are run on SunFire V490, 4 x UltraSPARC-IV+ 1500MHz, 16GB RAM, Solaris 10. The clients are modeled as the separate processes located in both sites and connect to the nearest replicas. Approximately the overall load is balanced equally between sites, since the number of clients processes and runs is the same in both sites during all of the experiments.
The Network specification: All servers are connected via an Ethernet-LAN running at 1000Mbps.
The relational database systems used in the laboratory experiments: Oracle ver. 9 - 11 and PostgreSQL 8.1.4.
The replication core software is compiled with the Sun C 5.8 compiler for servers provided by Sun, and gcc 3.4.5 is used for Linux servers.
Oracle database communication is maintained with the usage of Oracle Instant Client 10.2.
Since test servers and network are normally used for work during workdays, most of the experiments are conducted only by night hours and at weekends to avoid the influence of the users applications.
Laboratory environment configuration for Theta replication method imple-mentation with 2 replication nodes is presented in fig. C.1. A particular amount of clients, which in fact are dedicated processes simulating clients, is run on two clients servers. Clients submit theirs transactions to middleware and after transactions are processed in database, an answer is sent back to client. The transaction time, which is defined as difference between transaction start and end, is measured within the research tests. Transaction start means a moment just before client sends transaction to middleware, transaction end is a moment after client receives answer from the middleware.
The specifications of the servers used in particular scenarios with 1, 2, 5 and 10 replicas is presented in table C.1.
C.2 IBIS case
Within the tests there are performed numbers of experiments exploring system properties as well as analysis of chosen, particular system use cases. Experiments
Number of replicas Hardware configuration
1 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5
2
2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5
5
2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5
10
2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5 2 x Intel Xeon 2.6GHz, 4GB RAM, RedHat EL 5
NServer ME420 G4, 2 x Intel Xeon 2.00GHz, 4GB RAM, RedHat EL 4 NServer ME420 G4, 2 x Intel Xeon 2.00GHz, 4GB RAM, RedHat EL 4 NServer ME420 G4, 2 x Intel Xeon 2.00GHz, 4GB RAM, RedHat EL 4 Sun Fire 480R, 2 x UltraSparc III+ 1050MHz, 4GB RAM, Solaris 10 Sun Fire 480R, 2 x UltraSparc III+ 1050MHz, 4GB RAM, Solaris 10
Table C.1: Hardware configuration – in-laboratory tests
are conducted on several machines working concurrently.
A series of Cluster Supervisor Agents are run in every machine to realize particular tasks related to strategy management. Each machine running Cluster Supervisor Agents is treated as an autonomous site and is connected to a sep-arate configuration database (replica). For the test purpose it is run up to 10 parallel processes of Cluster Supervisor Agent to simulate higher load in the test environment with limited amount of machines.
To operate on data related to task management Dispatcher Agents also work on data stored in replicas. Each Dispatcher Agent connects and operates on dedicated Configuration Database replica.
In the most of research tests Dispatcher Agents and Cluster Supervisor Agents are connected to a dedicated, local replicas. This allow to verify system behavior with larger number of replicas. However, Dispatcher Agents and Cluster
Super-Figure C.2: Test environment for IBIS system with data replication
visor Agents that work in the same cluster can share local replicas. Sharing of replicas by agents in local cluster allows to gain better performance thanks to replication between clusters, and to lower costs of machines required in a single cluster.
A series of Working Agents are run in every machine to realize particular tasks. In the test conducted on the purpose of this research it is assumed that Working Agents share common, centralized Working database.
Transaction times for IBIS system are measured exactly in the same way as within the laboratory tests.
Sample configuration presenting test environment for IBIS system with data replication is presented in fig. C.2. Each replica has the number of IBIS agents attached and the middleware manage transactions and executes them in data replicas on behalf of those agents.
Details of hardware configuration for IBIS test environment are presented in table C.2.
Name Description
Number of servers 1 - 20
Operating system Windows XP SP3
Virtual machine configuration
VMware virtual machine with 2CPU and 2GB RAM running on Intel Core2 Quad CPU 4GB RAM, Windows 7
1 Virtual Machine per server
Database PostgreSQL 8.1.4
Table C.2: Hardware configuration for IBIS replication tests
[1] A. Adya, B. Liskov, and P. O’Neil. Generalized isolation level defini-tions. pages 67–78. ICDE, 2000.
[2] M. K. Aguilera, C. Delporte-Gallet, H. Fauconnier, and S. Toueg. Thrifty generic broadcast. Springer, 2000.
[3] J. E. Armendariz, H. Decker, F. D. Munoz-Escoi, L. Irun-Briz, and R. de Juan-Marin. A middleware architecture for supporting adapt-able replication of enterprise application data. pages 29–43. Springer, 2005.
[4] R. Azoulay-Schwartz and S. Kraus. Stable strategies for sharing information among agents. Morgan Kaufmann Publishers Inc., 2001.
[5] P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency control and recovery in database systems. Addison-Wesley Longman Pub-lishing Co. Inc., 1987.
[6] A. B. Bondi. Characteristics of scalability and their impact on perfor-mance. Proceedings of the 2nd international workshop on Software and performance, pages 195–203, 2000.
[7] A. Bougettaya, Z. Malik, A. Rezgui, and L. Korff. A Scalable Middleware for Web Databases. Codd & Date, Inc., 2006.
[8] D. K. Burleson, J. Garmany, and S. Karam. A Practical Guide to DB2 Udb Data Replication V8. Rampant, 2007.
[9] M. J. Carey and M. Livny. Conflict detection tradeoffs for replicated data. Number 4, pages 703–746. ACM Trans. Database Syst., 1991.
[10] W. J. Chen, M. Otsuki, P. Descovich, S. Arumuggharaj, T. Kubo, and Y. J. Bi. High Availability and Disaster Recovery Op-tions for DB2 on Linux, UNIX, and Windows. IBM Redbooks, 2009.
[11] G. Chockler, I. Keidar, and R. Vitenberg. Group communication specifications: a comprehensive study. ACM Computing Surveys, 2001.
[12] P. K. Chrysanthis and G. Santhanakrishnan. Towards universal mobile caching. In Proc. of the 4th ACM Intl Workshop on Data Engineer-ing for Wireless and Mobile Access, 2005.
[13] E. F. Codd, S. B. Codd, and C. T. Salley. Providing OLAP (On-line Analytical Processing) to User-Analysts: An IT Mandate. Codd & Date, Inc., 1993.
[14] T.H. Corman, C.E. Leiserson, and R. L. Rivest. Introduction to Algorithms. AMIT Press, 1990.
[15] C. Coulon, E. Pacitti, and P. Valduriez. Consistency management for partial replication in a high performance database cluster. pages 809–
815. ICPADS, 2005.
[16] D. Daniels, L. B. Doo, A. Downing, C. Elsbernd, G. Hallmark, S. Jain, B. Jenkins, P. Lim, G. Smith, B. Souder, and J. Stamos.
Oracle’s symmetric replication technology and implications for application design. (2), 1994.
[17] Khuzaima Daudjee and Kenneth Salem. Lazy database replication with ordering guarantees. page 424. IEEE Computer Society, 2004.
[18] X. Defago, A. Schiper, and P. Urban. Total order broadcast and multicast algorithms: Taxonomy and survey. ACM, 2004.
[19] J. Delacroix. Towards a stable earliest deadline scheduling algorithm.
pages 263–291. Springer, 1996.
[20] A. J. Demers, K. Petersen, M. J. Spreitzer, D. B. Terry, M. M. Theimer, and B. B. Welch. Application-specific conflict res-olution for weakly consistent replicated dstabases. United States Patent US005603026A, 1997.
[21] G. Dobrowolski, J. Kozlak, and E. Nawarecki. Agent-based mod-eling of supply chains in critical situations. Springer-Verlag, 2007.
[22] S. Elnikety, F. Pedone, and W. Zwaenepoel. Generalized snap-shot isolation and a prefix-consistent implementation. In Technical Report, EPFL, 2004.
[23] EMC. Emc srdf - zero data loss solutions for extended distance replication.
EMC Technical Note P/N 300-006-714, 2009.
[24] R. Fan and N. Lynch. Efficient replication of large data objects. pages 75–91. In Proceedings of the 17th International Symposium on Distributed Computing, Seoul, 2003.
[25] J. Ferber. Multi-agent systems: An introduction to distributed artificial intelligence. Addison-Wesley, 1999.
[26] Apache Software Foundation. http://logging.apache.org/. Apache Software Foundation, 2009.
[27] J. Gable. Enterprise application integration. Information Management Journal, 2002.
[28] S. Ghosh. Distributed systems an algorithmic approach. Chapman and Hall/CRC, 2007.
[29] J. Gray. The transaction concept: Virtues and limitations. pages 144–154.
Tandem Computers, 1981.
[30] J. Gray, P. Helland, P. O’Neil, and D. Shasha. The dangers of replication and a solution. ACM SIGMOD, 1996.
[31] J. Gray, P. Homan, H. Korth, and R. Obermarck. A strawman analysis of the probability of deadlock. IBM RJ 2131, IBM Research, San Jose, CA., 1981.
[32] Z. Guessoum, J.-P. Briot, N. Faci, and O. Marin. Towards reliable multi-agent systems - an adaptive replication mechanism. Multiagent and Grid Systems, 2007.
[33] F. Haas, P. Reisner, and L. Ellenberg. The drbd user’s guide.
LINBIT Information Technologies GmbH, 2009.
[34] A. A. Helal, A. A. Heddaya, and B. K. Bhargava. Replication Techniques in Distributed Systems. Kluwer Academic Pub, 1996.
[35] J. Holliday. Replicated database recovery using multicast communica-tion. In NCA, pages 104–107. IEEE Computer Society, 2001.
[36] J. Holliday, D. Agrawal, and A. E. Abbadi. Using multicast com-munication to reduce deadlock in replicated databases. In SRDS, pages 196–205. IEEE Computer Society, 2000.
[37] J. Holliday, R. C. Steinke, D. Agrawal, and A. E. Abbadi. Epi-demic algorithms for replicated databases. IEEE Transactions on Knowl-edge and Data Engineering, 2003.
[38] B. Horling, B. Benyo, and V. Lesser. An approach for providing mobile agent fault tolerance. pages 529–536. ACM Press, 2001.
[39] C. S. Horstmann and G. Cornell. Core Java(TM) 2, Volume II – Advanced Features. Prentice Hall, 2004.
[40] iBATIS. http://ibatis.apache.org/. Apache Software Foundation, 2009.
[41] IBM. A Practical Guide to DB2 Udb Data Replication V8. IBM Redbooks, 2002.
[42] IBM. Ibm db2 universal database. replication guide and reference. IBM Technical Report SC27-1121-00, 2008.
[43] L. Irun-Briz, H. Decker, F. Castro-Company, J. E. Armendariz-Inigo, and F. D. Munoz-Esco. A slim middleware for database repli-cation. Springer, 2005.
[44] Ch. Jin-oh, S. Young Sang, and H. Bonghee. Update propagation of replicated data in distributed spatial databases. pages 952–963. Springer-Verlag, 1999.
[45] Apache JMeter. http://jakarta.apache.org/jmeter/. Apache Software Foundation, 2009.
[46] A. C. Jnior, A. Sousa, E. Cecchet, F. Pedone, J. Pereira, L. Ro-drigues, N. M. Carvalho, R. Vilaa, R. Oliveira, S. Bouchenak, and V. Zuikeviciute. Gorda open replication of databases. Universidade do Minho, 2004.
[47] C. Kaiser and S. Saad-Bouzefrane. How to manage replicated real-time databases in an overloaded distributed system? 2008.
[48] M. Kasper. Features and analysis of various approaches for replication in database systems. InterTech, 2008.
[49] M. Kasper. Locks, isolation levels and concurrency control in increasing efficiency and scalability of database systems. Number 1, pages 55–66.
UWND Automatyka AGH, 2008.
[50] M. Kasper. Middleware based replication for database systems. Num-ber 12, pages 349–358. UWND Automatyka AGH, 2008.
[51] M. Kasper and G. Dobrowolski. Proposal of highly scalable data replication theta method for distributed transactional systems. In proceed-ings of 19th International Symposium on Application of Systems Theory, In print.
[52] M. Kasper and G. Dobrowolski. Replication-based information shar-ing in multi-agent system for monitorshar-ing the internet. In proceedshar-ings of 1st International Workshop on Intelligent Computing In Large-Scale Systems, In print.
[53] B. Kemme and G. Alonso. Don’t be lazy, be consistent: Postgres-R, a new way to implement database replication. VLDB, 2000.
[54] B. Kemme and G. Alonso. A new approach to developing and imple-menting eager database replication protocols. Number 3, pages 333–379.
ACM, 2000.
[55] B. Kemme, F. Pedone, G. Alonso, and A. Schiper. Using optimistic atomic broadcast in transaction processing systems. pages 1018–1032. IEEE Computer Society, 2003.
[56] W. Khadzynov and M. Maksymiuk. Realizacja replikacji w systemach heterogenicznych baz danych. pages 164–174. Wydawnictwa Komunikacji i Lacznosci, 2006.
[57] G. Koren and D. Shasha. Dover; an optimal on-line scheduling algo-rithm foroverloaded real-time systems. pages 290–299. Real-Time Systems Symposium, 1992.
[58] V. R. Lesser and L. Gasser. Proceedings of the first international conference on multiagent systems. MIT Press, 1995.
[59] A. Ligeza. Logical foundations for rule-based systems. Wydawnictwo AGH, 2005.
[60] A. Ligeza, R. Klimek, and T. Szmuc. Analysis, classification and evaluation of selected problems and solutions for knowledge representation and verification. pages 261–266. Cracow Centre for Advanced Training in Information Engineering, 1999.
[61] Y. Lin, B. Kemme, M. Pati´no-Mart´ınez, and R. Jim´enez-Peris.
Middleware based data replication providing snapshot isolation. pages 419–
430. ACM Press, 2005.
[62] Q. Lu and M. Satyanarayanan. Improving data consistency in mobile computing using isolation-only transactions. In Proceedings of the Fifth Workshop on Hot Topics in Operating Systems, Orcas Island, Washington, 1995.
[63] Microsoft. Microsoft sql server 2000. Microsoft, 2005.
[64] MySQL. http://dev.mysql.com/. MySQL, 2009.
[65] J. Nowak, W. Zegarowski, P. Gosztyla, E. Sobon, and K. Pisiak. Dokumentacja. system ibis wersja 1.0. Katedra Informatyki AGH, 2008.
[66] Oracle. An introduction to oracle partitioning. Oracle White Paper, 2005.
[67] Oracle. Oracle database advanced replication. Oracle Technical Report B14226-02, 2007.
[68] Oracle. Oracle streams replication administrator’s guide. Oracle Techni-cal Report B14228-04, 2008.
[69] Oracle. http://www.oracle.com/technetwork/database/features/oci/in dex.html. Technical report, 2009.
[70] Oracle. Oracle data guard with oracle database 11g release 2. Oracle Technical White Paper, 2009.
[71] Oracle. http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136424.html. Oracle, 2010.
[72] T. Pankowski. Podstawy baz danych. Wydawnictwo Naukowe PWN Warszawa, 1992.
[73] T. Pankowski. Reasoning about data in xml data integration. pages 2506–2513. Information Processing and Management of Uncertainty in Knowledge-based Systems, 2006.
[74] J. F. Paris. Voting with witnesses: A consistency scheme for replicated files. pages 606–612. Proc. Sixth International Conference on Distributed Computing Systems, 1986.
[75] HTML Parser. http://htmlparser.sourceforge.net/. Sourceforge.net, 2006.
[76] M. Pati´no-Mart´ınez, R. Jim´enez-Peris, B. Kemme, and G. Alonso. Middle-R: Consistent database replication at the middleware level. Number 4, pages 375–423. ACM, 2005.
[77] F. Pedone, S. Elnikety, and W. Zwaenepoel. Database replication using generalized snapshot isolation. pages 73–84. IEEE Computer Society, 2005.
[78] F. Pedone and A. Schiper. Generic broadcast. Springer, 1999.
[79] D. Peleg. Distributed computing: A locality-sensitive approach. SIAM, 2000.
[80] K. Petersen, M. Spreitzer, D. B. Terry, M. Theimer, and A. J.
Demers. Flexible update propagation for weakly consistent replication. In SOSP, pages 288–301, 1997.
[81] F. Piedad. High availability: Design, techniques, and processes. Prentice Hall, 2000.
[82] E. Pitoura and B. Bhargava. Data consistency in intermittently con-nected distributed systems. IEEE Transactions on Knowledge and Data Engineering, 11(6):896–915, 1999.
[83] E. Pitoura and P. K. Chrysanthis. Caching and replication in mobile data management. IEEE Data Eng. Bull.
[84] Ch. Plattner and G. Alonso. Ganymed: scalable replication for trans-actional web applications. pages 155–174. Springer-Verlag, 2004.
[85] PostgreSQL. http://jdbc.postgresql.org/. PostgreSQL, 2009.
[86] PostgreSQL. http://www.postgresql.org/docs/8.1/static/libpq.html.
Technical report, 2009.
[87] Z. Qiansheng, H. Quanyi, G. Jiming, W. Renqiang, and Z. Shaobo. A model of spatial data integration interoperability on or-acle spatial. Springer Berlin Heidelberg, 2007.
[88] M. Rabinovich and E. D. Lazowska. An efficient and highly available read-one write-all protocol for replicated data management. University of Washington Department of Computer Science and Engineering, 1992.
[89] M. Rabinovich and E. D. Lazowska. Efficient support for partial write operations in replicated databases. 1994.
[90] W. Rakoczy. System ibis - raport z dnia 2007.04.20. Katedra Informatyki AGH, 2007.
[91] U. Ramachandran, M. Solomon, and M. Vernon. Hardware sup-port for interprocess communication. Proceedings of the 14th annual inter-national symposium on Computer architecture, 1987.
[92] R. Van Renesse and A. S. Tanenbaum. Voting with ghosts. pages 456–
462. Proc. 8th International Conference on Distributed Computing Systems, 1988.
[93] Y. Saito and M. Shapiro. Optimistic replication. ACM Computing Surveys, 37(1):42–81, 2005.
[94] M. Satyanarayanan. The evolution of coda. ACM Trans. Comput. Syst., 20(2):85–124, 2002.
[95] D. A. Shakib, S. Norin, and M. L. Benson. System and method for distributed conflict resolution between data objects replicated across a computer network. United States Patent US005787262A, 1999.
[96] S. Sheikh and R. Ganesan. Replication of multimedia data using master-slave architecture. Computer Software and Applications Conference, pages 66–70, 1997.
[97] F. De Assis Silva and R. Popescu-Zeletin. An approach for providing mobile agent fault tolerance. pages 14–25. LNCS, 1998.
[98] D. Skeen and M. Stonebraker. A formal model of crash recovery in a distributed system. IEEE Transactions on Software Engineering, 9(3):219–
228, 1983.
[99] Slony. http://www.slony.info/. Slony Development Group, 2009.
[100] Spring Source. http://www.springsource.org/. Spring Source Comunity, 2010.
[101] W. R. Stanek. Microsoft SQL Server 2008 Administrator’s Pocket Con-sultant. Microsoft, 2008.
[102] W. R. Stevens. UNIX Network Programming. Prentice-Hall, 1990.
[103] M. Stonebraker. Concurrency control and consistency of multiple copies of data in distributed ingres. IEEE Trans. Softw. Eng., 1979.
[104] P. Sujoy. Pro SQL Server 2008 Replication. Apress, 2008.
[105] Sybase. Replication strategies: Data migration, distribution and synchro-nization. Sybase, 2007.
[106] Symantec. Veritas volume replicator option by symantec: A guide to understanding volume replicator. Symantec, 2006.
[107] ObjectWeb Team. http://c-jdbc.ow2.org. ObjectWeb Open Source Mid-dleware, 2008.
[108] FAQ. http://www.tech-faq.com/database-replication.shtml. Tech-FAQ, 2009.
[109] D. Terry, A. Demers, K. Petersen, M. Spreitzer, M. Theimer, and B.Welch. Session guarantees forweakly consistent replicated data.
In Proceedings of the International Conference on Parallel and Distributed Information Systems, pages 140–149, 1994.
[110] D. B. Terry, M. M. Theimer, K. Petersen, A. J. Demers, M. J.
Spreitzer, and C. H. Hauser. Managing update conflicts in bayou, a weakly connected replicated storage system. In SOSP ’95: Proceedings of the fifteenth ACM symposium on Operating systems principles. ACM, 1995.
[111] R. H. Thomas. Epidemic algorithms for replicated databases. A majority consensus approach to concurrency control for multiple copy databases, 1979.
[112] Tika. http://tika.apache.org/. Apache Software Foundation, 2006.
[113] TPC. Tpc-c benchmark. Transaction Processing Performance Council, 2007.
[114] R. Vandewalln. Database Replication Prototype. PhD thesis, University of Groningen.
[115] P. Vicente and L. Rodrigues. An indulgent uniform total order algo-rithm with optimistic delivery. IEEE Computer Society, 2002.
[116] H. Vogler, T. Kunkelmann, and M. L. Moschgaht. An approach for mobile agent security and fault tolerance using distributed transactions.
Darmstad University, 1997.
[117] M. Wiesmann. Group communications and database replication: tech-niques, issues and performance. PhD thesis, Ecole Polytechnique Federale De Lausanne, 2002.
[118] M. Wiesmann, F. Pedone, A. Schiper, B. Kemme, and G. Alonso.
Database replication techniques: a three parameter classification. IEEE Computer Society, 2000.
[119] Inc. Wikimedia Foundation. http://en.wikipedia.org/wiki/. Wikime-dia, 2011.
[120] A. Wolski. Database replication for the mobile era. ICDE, 2002.
[121] M. Xiong, K. Ramamritham, J. Haritsa, and J. A. Stankovic.
Mirror: A state-conscious concurrency control protocol for replicated real-time databases. page 100, 1999.
[122] T. Yi-Cheng, Y. Jingfeng, S. Gang, and P. Sunil. Multiquality data replication in multimedia databases. IEEE Transactions on knowledge and data engineering, 19(5), 2007.