Theta replication system is implemented for both C and Java programming lan-guages, and for different hardware platforms and databases. The wide-ranging evaluation of the implemented method has been performed and built replication software can be used for the purposes of data sharing based on Theta replication method. It is planed to package the software and to prepare universal installer to enable easier deployment of the replication software as well as its configuration.
Attached materials can be used to set up such replication environment as it is used within the performed evaluation of Theta method.
CD content
Description of the elements are placed on the attached CD is presented in figA.1.
Binaries of the IBIS agents equipped with services for communication with middleware replication subsystem as well as libraries providing database services used by IBIS agents are not included on the CD since the author of this disser-tation is not the owner of the IBIS system.
Content of replication system Directories
• lib - libraries providing database services used by IBIS agents.
• logs - logs of the middleware components.
• tlogs - middleware transactional logs.
Middleware components for in-laboratory implementation
• cman - connection manager,
• qman - queue manager,
• crm - conflict resolution manager,
• spex - stored procedure executor,
• gidg - global identifier generator.
• lib ora - support for Oracle database,
• lib func - additional functions.
Middleware components for IBIS implementation
• cman.class - connection manager,
• qman.class - queue manager,
• crm.class - conflict resolution manager,
• spex.class - stored procedure executor,
• gidg.class - global identifier generator,
• rcvman.class - recovery manager.
Communication classes
• CmanConnectionHandler.class - implementation of the listener for cman,
• GidgConnectionHandler.class - implementation of the listener for gidg,
• ReceiveMessageInterface.class - RMI communication interface.
Configuration files
• theta.properties - definition of the main parameters for each of the middle-ware components (database connection strings, IP, port numbers, etc.),
• log4j.cman.properties - log4j configuration for cman,
• log4j.spex.properties - log4j configuration for spex,
• log4j.recover.properties - log4j configuration for rcvman.//
Setting up environment
Example directory structure for IBIS replication implementation is presented in fig. A.2.
Figure A.2: Middleware directory structure for IBIS implementation
presented in fig. A.3.
Figure A.3: Running middleware processes for replication in IBIS
Implementation details
B.1 In-laboratory
The platform compatibility of the implementation of Theta replication method is fulfilled by the development of its code in C language, which compiles and executes as expected on many different hardware and software environments.
Every Theta replication middleware component is implemented in C language.
The data replication core software is compiled with the usage of:
• Sun C 5.8 compiler for servers provided by Oracle Sun,
• Gcc 3.4.5 for linux based servers.
Communication between the middleware components is based on the usage of Inter-Process Communication (IPC) mechanisms [91, 102]. IPC is a set of tech-niques for the exchange of data among multiple threads in one or more processes.
Processes may be running on one or more computers connected by a network.
IPC techniques are divided into methods for message passing, synchronization, shared memory, and remote procedure calls (RPC). The method of IPC used may vary based on the bandwidth and latency of communication between the threads, and the type of data being communicated.
CMAN middleware process acts as a server listening on a port and waiting for incoming requests from clients. Each of the client’s connections is managed by a dedicated thread to allow clients to establish multiple, concurrent connections.
Implementation of the connection handling is based on TCP sockets [102] usage.
A TCP socket is defined as an endpoint for communication, and consists of the pair (IP Address,Port). A TCP connection consists of a pair of sockets, which are distinguished by client and server sockets.
The communication between connection manager and global identifier gener-ator is realized with the usage of TCP sockets. There is only one GIDG process for each of replicas in the laboratory implementation of the data replication. This process runs on a dedicated, separate machine, however, it is possible to run it on the same server machine where the other middleware processes are run.
Theta replication method is implemented and tested with both commercial and non-commercial open source relational database systems. The implemen-tation and tests are realized for Oracle 10.2.0.4 and PotgreSQL 8.1.4 relational database systems.
Oracle database connections is maintained with the usage of Oracle Call In-terface (OCI) [69] implemented within Oracle Instant Client 10.2. According to Oracle documentation, OCI is the most comprehensive, high performance, native C language based interface to the Oracle Database that exposes the full power of the Oracle Database. OCI provides the foundation on which various language specific interfaces such as Oracle JDBC-OCI, ODP.Net, Oracle Precompilers, Or-acle ODBC and OrOr-acle C++ Call Interface (OCCI) drivers are built. OCI is also used by leading Open Source interfaces such as the PHP OCI8 extension, ruby-oci8, Perl DBD::oracle and Phython cx oracle. OCI being the native interface to the Oracle Database also powers queries, statements and calls issued by the Oracle Database kernel internally. Various Oracle tools such as SQL*Plus, Real Application Testing (RAT), SQL*Loader and Data-Pump are also OCI based.
In addition, Oracle In-Memory Database Cache (IMDB Cache a.k.a. TimesTen) supports OCI and Oracle Pro*C/C++ Pre-compilers.
PostgreSQL database communication is based on the usage of C Library libpq [86]. Libpq is the C application programmer’s interface to PostgreSQL. Libpq is a set of library functions that allow client programs to pass queries to the
PostgreSQL back-end server and to receive the results of these queries. Libpq is also the underlying engine for several other PostgreSQL application interfaces, including those written for C++, Perl, Python, Tcl and ECPG.
B.2 IBIS case
To ensure compatibility with the IBIS system based on Java platform, data repli-cation implementation is realized in Java programming language. The Internet monitoring system with data replication implementation is build with the usage of Java compiler in the following version:
Java version ”1.6.0 20”
Java(TM) SE Runtime Environment (build 1.6.0 20-b02)
Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing)
Communication between the middleware components for IBIS system data replication implementation is based on the usage of Remote Method Invocation (RMI) [39, 71]. Java RMI enables the programmer to create distributed Java technology-based to Java technology-based applications, in which the methods of remote Java objects can be invoked from other Java virtual machines, possibly on different hosts. RMI uses object serialization to marshal and unmarshal parame-ters and does not truncate types, supporting true object-oriented polymorphism.
Java RMI is included with Java SE and is available as a separate download for Java ME.
The communication between connection manager and global identifier gener-ator is based on TCP sockets [102] usage. In the implementation for IBIS system there is only one GIDG process dedicated for servers in every site which runs within its own Java Virtual Machine (JAVA VM) and is located in the same server machine where the other middleware processes run. However, it is possible to exclude GIDG from the common server, and invoke within Java VM that runs on the dedicated server.
PostgreSQL 8.1.4 relational database system is used to store data in the IBIS system. The communication with the centralized database used for storing tem-porary agents data is realized with iBATIS framework [40]. IBATIS is a data mapper framework that made it easier to use a relational database with object-oriented applications. Data flow in iBATIS based environment is presented in fig 5.14. There are both Java and .Net implementations of iBATIS. To implement database interface relying on the iBATIS data mapper application objects, XML, and SQL are used.
Theta replication method is implemented for replication-based information sharing in multi-agent IBIS system. Data in the system is also stored in Post-greSQL 8.1.4 relational database system. The middleware components communi-cate with data replicas by stored procedures calls, which are realized through the JDBC driver for PostgreSQL database packaged into postgresql-8.4-701.jdbc3.jar [85]. PostgreSQL JDBC driver allows Java programs to connect PostgreSQL database using standard, database independent Java code. It is a Java Type IV implementation. The PostgreSQL JDBC driver provides a complete imple-mentation of the JDBC 3 specification in addition to some PostgreSQL specific extensions.
For compatibility with the IBIS system logging within data replication is re-alized on the basis of the Log4j technology. Log4j for Java (log4j-1.2.14.jar) is a product included in the Apache Logging Services Project frameworks. The Apache Logging Services Project creates and maintains open-source software re-lated to the logging of application behavior and released at no charge to the public [26].