Abstract—The execution of a client/server application involving database access requires a sequence of database transaction events (or, T-events), called a transaction sequence (or, T-sequence). A client/server database application may have nondeterministic behavior in that multiple executions thereof with the same input may produce different T-sequences. In this paper, we present a framework for testing all possible T-sequences of a client/server database application. We first show how to define a T-sequence in order to provide sufficient information to detect race conditions between T-events. Second, we design algorithms to change the outcomes of race conditions in order to derive race variants, which are prefixes of other T-sequences. Third, we develop a prefix-based replay technique for race variants derived from T-sequences. We prove that our framework can derive all the possible T-sequences in cases where every execution of the application terminates. A formal proof and an analysis of the proposed framework are given. We describe a prototype implementation of the framework and present experimental results obtained from it.
Index Terms—Concurrent programming, reachability testing, client/server, database management system.
æ
1
I
NTRODUCTIONG
ENERALLYspeaking, a client/server program is a type of distributed program (or concurrent program). Let P be a distributed program. Multiple executions of P with the same input may produce different results—this is called nondeterministic behavior [1], [2]. Because of this, when testing P with input X which is a sequence of inputs for processes in P, a single execution is insufficient to determine the correctness of P with X. Even if P with input X has been executed successfully many times, it is possible that a future execution of P with X will produce incorrect results. An execution of a distributed program exercises a sequence of synchronization events called asynchronization sequence (or SYN-sequence). Examples of process synchronization include P and V primitives applied to a shared semaphore, monitor-entry procedures, send/receive message primitives, and general sharing of memory [3], [4]. The distributed program (or concurrent program) exhibits nondeterministic behavior because different executions of P with the same input X may produce distinct SYN-sequences. The meanings of some terms are not standard. In this paper, a test of P with input X is to execute P with input X once to obtain a SYN-sequence and check the result of execution. Duplicat-ing test means that different tests of P with input X produce the same SYN-sequence. Exhaustive testing of P
with input X is to conduct a lot of tests which exercise all feasible SYN-sequences of P with input X.
In this paper, we focus on the problem of testing client/ server SQL database applications that exhibit nondetermi-nistic behavior. The client interacts with the servers using the SQL [5], which is a powerful set-oriented language consisting of a few commands; it was created as a language for databases that adhere to the relational model [6]. Consider a client/server SQL application P with input X. The X contains input for each client as well as the initial state of each SQL database. During the execution of P, clients send SQL transactions to database servers concur-rently. Whenever there are two or more SQL transactions, sharing the same portion of data in the same database server, a race condition may occur. Executing these SQL transactions in different orders, i.e., with different race outcomes, may produce different results. Thus, our target programs are distributed programs in which multiple clients race for data in relational databases.
We present a framework using the reachability testing scheme to test SQL client/server applications. Our scheme provides a way of overcoming the nondeterministic behavior. If every execution of P with input X terminates, reachability testing of P with input X can accomplish exhaustive testing of P with input X and, thus, can determine the correctness of P with X. However, existing methods for reachability testing can only be applied to the testing of shared-memory [7] and asynchronous-message-passing [8], [9], [10] distributed programs.
First, we design the data structure of the SYN-sequence for race analysis of SQL transactions. It is the transaction sequence (T-sequence) that consists of many transaction . G.-H. Hwang is with the Department of Information and Computer
Education, National Taiwan Normal University, 162, Hoping E RD. Sec. 1, Taipei, Taiwan 106. E-mail: [email protected].
. S.-J. Chang is with Telecommunications Laboratories, Chunghwa Telecom Co., Ltd., No. 12, Lane 551, Min-Tsu Road Sec. 5 Yang-Mei, Taoyuan, Taiwan 326. E-mail: [email protected].
Since the SQL transaction actually has the most complicated data-access models, the T-sequence can also handle other types of synchronization model, including shared-memory, semaphore, monitoring, and message-passing models. Second, we design new algorithms to derive race variants from a T-sequence. Because the data-access models of shared memory and message passing are far simpler than the SQL database, the algorithms for deriving race variants presented in [7], [8], [9], [10] can no longer be applied. We present two algorithms in this paper. The first one is a modified version of the old algorithm presented in [7]. It generates a race-variant diagram (RV diagram) to derive race variants. The second is a novel algorithm that analyzes the order of T-events in a T-sequence to generate race variants. Formal proofs and complexity analyses are given for both algorithms. The experimental results show that the second algorithm is much more efficient than the first one, and we consider it a significant breakthrough in reach-ability-testing technology.
Third, we develop a scheme for performing prefix-based replay for an SQL client/server database program. We present two approaches: the first one is based on the intrinsic synchronization mechanism of an SQL database server, and the other is based on thread synchronization. We have implemented the entire architecture and have performed many experiments on it. The experimental results show that the scheme we propose in this paper is a practical method for testing client/server database applications. In addition, the codes we implemented for the experiments can be obtained via the Internet (see Section 7).
This paper is organized as follows: Section 2 surveys previous work on database testing and several approaches to the testing of distributed programs. Section 3 discusses how to performa race analysis for SQL transactions. In Section 4, we present two approaches for deriving race variants from a T-sequence. Section 5 presents two approaches for prefix-based replay. Section 6 presents the results of experiments and Section 7 concludes this paper.
2
P
REVIOUSW
ORK ANDT
ESTINGA
PPROACHES FORD
ISTRIBUTEDP
ROGRAMSNo techniques targeted specifically toward the nondetermi-nistic behavior of database applications have been de-scribed in the software-testing research literature. However, some studies have investigated techniques for generating automated and semiautomated tests for database systems and applications. The work of Davies et al. [12] on data-constraint testing investigated the feasibility of automati-cally producing test data to a database for load testing and establishing the integrity of the stored data. They proposed a scheme to check if the database has the ability to store, retrieve, update, and delete records correctly. The work in [13] proposed a testing technique to help assure that database programs meet a user’s specification. Some other work has assessed the performance of database manage-ment systems rather than tested applications for correctness [14], [15], [16], [17].
The only related work is that by Chu and Dobson [18], which involved a client/server database program that exhibited nondeterministic behavior. But, they did not propose any solution. However, there has been a lot of work on testing technologies for ordinary nondeterministic concurrent programs that do not involve database access. The remainder of this section provides a brief overview of this work. Nondeterministic testing (also called multiple-execution testing) of a concurrent program P involves the following steps: 1) Select a set of inputs of P, and 2) for each selected input X, execute P with Xmanytimes and examine the result of each execution. Nondeterministic testing of P with input X has two major problems; one is that some feasible SYN-sequences of P with input X may be executed many times, and the other is that some feasible sequences-may never be executed [7], [19].
Deterministic testing of a concurrent program P involves the following steps: 1) Select a set of tests, each of the form (X,S), where X and S are an input and a SYN-sequence of P, respectively, and 2) for each selected test (X,S), perform a deterministic execution of P with input X according to S. The forced execution determines whether S is feasible for P with input X. Deterministic testing of P allows the use of SYN-sequences selected according to the implementation and specification of P. However, deterministic testing has additional problems to solve. One major problem is deciding which pairs of inputs and SYN-sequences to select for a concurrent program. A number of methods for solving this problem have been proposed [20], [21], [22].
Fig. 1 illustrates the concept of reachability testing. Assume that S is the SYN-sequence of an execution of P with input X. Reachability testing of P with input X and SYN-sequence S involves the following steps:
1. Use S to derive a set of prefixes of other feasible SYN-sequences of P with input X. Such prefixes (race variants of S) are derived by changing the outcome of race conditions in S. An execution which follows a race variant of S will always exercise a SYN-sequence which is different from S.
2. For each new race variant derived in Step 1, perform a prefix-based replayof P with input X and the race variant to execute and collect an additional SYN-sequence for P with input X. The prefix-based replay of P with a race variant R includes two phases:
record the subsequent synchronization events to obtain a SYN-sequence of an execution of P.
3
R
ACEA
NALYSIS FORSQL D
ATABASET
RANSACTIONSThe concurrent queries of several clients to the same database may access the same portion of data. The majority of modern SQL servers are transaction servers; the client invokes remote procedures that reside on the server with an SQL database engine. The remote procedures on the server execute a group of SQL statements called a transaction, and the SQL statements contained within this group either all succeed or fail as a unit. Also, a transaction will lock the accessed data automatically. The special case of a transac-tion is one with only a single SQL statement.
All process synchronizations in a parallel program were modeled in [11] as operations on shared data (or shared memory). It showed that this characterization of process interactions is not restrictive, since many communication and synchronization primitives can be reduced to opera-tions on shared data. In particular, message passing can be modeled as communication through a shared port or mailbox. The work in [7] showed how to perform reach-ability testing of concurrent programs using read and write operations to shared memory. Let P be a concurrent program using read and write operations. Each shared variable in P is assigned a version number, which is initialized to zero and increased by one after each write operation on it. An execution of P involves two types of synchronization events: read and write. A read event is denoted as R(U,V), which refers to a read operation on variable U with the version number being V, and a write event is denoted as W(U,V), which refers to a write operation on variable U with the resulting version number being V. Thus, if W(A,1) and R(A,1) are issued by different processes in the concurrent program, we can determine that W(A,1) and R(A,1) have a race condition and that W(A,1) happened before R(A,1). In [9], Bechini and Tai presented algorithms for vector timestamps developed to determine the “happen before” relations between events of an execution of a message-passing program. The algorithms can be applied to perform race analysis. In [10], Lei and Tai improved the work in [8]. It showed how to derive all race variants for asynchronous message-passing programs for efficient searching and insertion without using prefix-based testing.
the shared-memory model, in which each read or write operation only accesses a single shared variable). Third, the shared-memory model has a fixed number of shared variables which are declared and initialized before program execution. However, with theINSERT andDELETE opera-tions in SQL, the number of rows in any table always varies depending on the SQL transaction operations of clients. We can treat read/write access to shared memory as a special case in the data access model of SQL transaction in which each row represents a shared variable.
Consider the essential data structure of an SQL database. Data are stored as rows in tables. A primary key is a field whose value must be unique for each row. It seems that the most precise way to identify the accessed rows in an SQL transaction is to store the table names as well as the primary keys of the accessed rows. However, we will show that there are some situations in which storing the primary key of each access row of an SQL transaction does not help to detect the race condition. We provide three examples to illustrate this point. The first example considers the case where two concurrent database transactions are the SQL statementsINSERTandSELECT, respectively. Assume the table name iscarand that columnidis the primary key of this table (see Table 1).
The two SQL transactions issued by two concurrent clients areT1 andT2, whereT1 is
BEGIN_TRANSACTION
INSERT INTO car VALUES (5, LS430, Lexus, 3,500,000) END_TRANSACTION
andT2is
BEGIN_TRANSACTION
SELECT * FROM car WHERE car.car_price > 600,000 END_TRANSACTION
Transaction T1 inserts a row (5, LS430, Lexus,
3,500,000) into tablecar, and transactionT2selects rows
fromcarfor whichcar_price> 600,000. Assume that these transactions are issued by two concurrent clients. IfT1 is
executed prior toT2, then the primary keys of accessed rows
of the SQL transaction issued byT1andT2are 5 and 1, 2, and
5, respectively. By observing the primary-key information in the two T-events, it is obvious that the two clients race because they access the same row, i.e., the row with primary key 5. However, ifT2is executed prior toT1, then the primary keys
The second example is where the two concurrent database transactions are SELECT and UPDATE. The two database transactions issued by two concurrent clients are
T3 andT4, whereT3 is
BEGIN_TRANSACTION
SELECT * FROM car WHERE car.car_price > 600,000 END_TRANSACTION
andT4 is
BEGIN_TRANSACTION
UPDATE car SET car_price = 550,000 WHERE car_name = "Altis"
END_TRANSACTION
If T3 is executed prior to T4, then the primary keys of
accessed rows of the SQL transaction issued byT3 andT4
are 1 and 2, and 2, respectively. By observing the primary-key information in the two T-events, it is obvious that the two clients race because they access the same row. However, if T4 is executed prior to T3, then the primary
keys of accessed rows of the SQL transaction issued byT4
andT3are 2 and 1, respectively. In this case, the
primary-key information in the T-events does not exhibit the race condition.
The above two examples show that it is inappropriate to use the primary keys of accessed rows in database transactions to decide whether two SQL transactions exhibit a race condition. This is because the accessed rows are determined by the Boolean predicates which are not related to the primary key in the WHERE clause of the SQL statement. Consider the following example where the Boolean predicates are bound to the primary key. The two database transactions issued by two concurrent clients are
T5 andT6, whereT5 is
BEGIN_TRANSACTION
SELECT * FROM car WHERE car.id in (1,3) END_TRANSACTION
andT6 is
BEGIN_TRANSACTION
UPDATE car SET car_price = 71,532 WHERE car.id=2 END_TRANSACTION
Irrespective of whether T5 executes prior to T6 or T6
executes prior toT5, the primary keys of the accessed rows
of the SQL transaction issued byT5andT6are 1 and 3, and
2, respectively. Sincef1;3g [ f2g ¼ , there is no data race between the two transactions.
Using the above three examples, we can design the data structures of the T-event and T-sequence so that they contain enough information to perform a race analysis. Let the client/server application P contain clientsCL1;CL2;. . .;CLn,
and the SQL database compriseDB1;DB2;. . .;DBmamong
all the database transaction servers, wheren >1andm >0
(note that there may be more than one database in a database server). An execution of P exercises a sequence of SQL transactions (the T-sequence). More specifically, the T-sequence of P is denoted as (T1;. . .;Ti;. . .;Tx), where
Ti;1ix, is a T-event. We define the T-event as the
following data structure:
Ta= {Database_name, Database_transaction_order, Client_name, Client_transaction_order, [Table_name1, Operation1, Primary_Key_List1], . . . [Table_namei, Operationi, Primary_Key_Listi], . . . [Table_names, Operations, Primary_Key_Lists] }
Tais a T-event. The header of a T-event records the name
of the database and the name of the requested client, which here areDatabase_nameandClient_name, respectively. Each database is associated with a transaction number, which is initialized to one and increased by one after each transaction is performed on this database. Database_
transaction_orderrecords the order of this transaction
relative to all transactions executed on the database by different clients. The Client_transaction_order in-dicates the order of this transaction relative to all transac-tions executed by the client.
After the header, there is a series ofaccessed table records. We have mentioned that a single SQL statement may access multiple rows in multiple tables. For the race analysis, we translate each SQL statement into multiple accessed table records. Each accessed table record is represented by Table namei,Operationi, andPrimary Key Listi,1is.
Table namei is the table name. We summarize the row
operations as havingINSERT,READ,UPDATE, andDELETE
instructions. Note that these four instructions are operations to rows rather than SQL statements. The combination of these four operations can represent any SQL data-manip-ulation statements, including static and dynamic SQL [5]. Primary Key Listi has one of following two forms:
1. The {*} represents that the primary keys of the accessed rows depend on the status of the database and the WHERE clause in the SQL statement (e.g., SQL transactions T2,1 T3, and T4 shown above). It
means that an operation is possible to act on any row in a table.
2. If the primary keys of the access rows do not depend on the status of the database and theWHEREclause in the SQL statement, Primary Key Listi is a set of
primary keys of the formfk1;k2;. . .gwhich records the primary keys of accessed rows (e.g., SQL transactionsT1,T5, andT6shown above).
Example 1 (Fig. 2) shows an SQL transaction with five SQL statements and its corresponding T-event.
In the following, we define a race relation of two T-events (see Fig. 3). Two T-events are said to exhibit a
transaction raceif they can access the same portion of data in the same database, and if the results of their execution may differ with their order of execution.
1. Note that transactionsT1andT6are different from the transactions in Example 1.
Referring to Definition 1, conditions 1-3 mean that Tu
and Tv are two SQL transactions which act on the same
table in a database, but are issued by different clients. Condition 4 means that the operations are not both READ, and condition 5 determines if the two operations issued by two clients may act on the same row. Note that {*} means that an operation is able to act on any row in a table.
In the above discussion, we have provided examples which illustrate that the race may occur between INSERT
and READ or between READ and UPDATE; two INSERT
operations may also race. Consider the case where two clients C1 and C2 are trying to insert a row with the same primary key into the same table in a database. If Client C1 executes prior to Client C2, then the SQL transaction of C2 Fig. 2. Example 1.
will fail; conversely, C1 will fail is C2 executes first. The execution order of the twoINSERToperations changes the execution results. Another case is when the two operations
are DELETE. Consider the case where two clients are
attempting to delete the same row from the same table. The first client will delete the row successfully, after which the second client will receive a reply stating that it cannot successfully delete the row from the database server. The client may perform another transaction if it cannot success-ful delete the row from the table. Thus, it is obvious that the order of execution of transactions issued by the two clients affects the results. This is also a type of race between two transactions.
4
D
ERIVER
ACEV
ARIANTS FROM AT
RANSACTIONS
EQUENCESection 3 defined the format of a T-sequence and showed how to detect a transaction race between two T-events in a T-sequence. This section presents algorithms for deriving race variants from a T-sequence.
Let the client/server application P contain clients CL1,
CL2;. . .;CLn, and the SQL database comprise DB1,
DB2;. . .;DBm among all the database transaction servers,
wheren >1andm >0. Let TS be a feasible T-sequence of P with input X. X contains input for each client and the initial state of each database. For convenience, we define the following notation:TSðCLiÞis the ordered set of the T-events
of ClientCLi, andTSðCLi;jÞis thejthT-event of ClientCLi.
4.1 The Race Graph of a T-Sequence
Before we describe how to derive race variants for a T-sequence, we show how to construct a directed partial-ordered graph of a T-sequence (see Fig. 4). We call this the
race graph of a T-sequence. The race graph is used to determine if there exists a race condition whose outcome can be changed.
The vertices in a race graph represent T-events. The added edge between two T-eventse1ande2represents that
e1 must happen before e2. We use the following example
to illustrate Algorithm 1 (Fig. 4). Assume there are two databases, DB1 and DB2: TA1 and TA2 are tables of DB1; TA3 is a table of DB2; there are three clients C1, C2, and C3; and TS includes the following T-events:
TS(C1,1) = {DB1,1,C1,1,[TA1,read,*], [TA2, read,*]} TS(C1,2) = {DB2,1,C1,2,[TA3,insert,4]}, TS(C2,1) = {DB1,2,C2,1,[TA1,insert,5]}, TS(C3,1) = {DB2,2,C3,1,[TA4,insert,5]}, TS(C2,2) = {DB2,3,C2,2, [TA3,read,*], [TA4,read,*]}, TS(C2,3) = {DB1,3,C2,3,[TA1,delete,*], [TA2,read,*]}, TS(C1,3) = {DB1,4,C1,3,[TA1,read,*], [TA2,read,*]},
We summarize the race relation between the T-events in TS in Table 2.
After Steps 1 and 2 of Algorithm 1, we have the graph shown in Fig. 5a. After Step 3, we obtain the final race graph, as shown in Fig. 5b.
Given a T-sequence TS, we denote the race graph of TS as
Race-graph(TS). In the following, we give definitions which are related to the race graph, which will be referred to in the algorithm for deriving race variants (see Figs. 7 and 8).
Note thate1happened beforee2means not only thate1
occurred beforee2, but also that there is a causality relation
betweene1ande2[23].
Fig. 4. Algorithm 1.
TABLE 2
Race Relationships between T-Events in T-Sequence TS
If f is a feasible prefix of a T-sequence TS, then f
corresponds to a feasible intermediate execution state which can reach the execution state of TS. We use Fig. 6 to illustrate Definition 3. P1 is a feasible prefix of TS, but P2 is not a feasible prefix of T because TS(C2,1) happened before TS(C1,3). Since TS(C1,3) is in P2, TS(C2,1) should also be included in P2 to make P2 a feasible prefix of TS.
4.2 Generate an RV Diagram to Derive
Race Variants
In this paper, we propose two schemes for deriving race variants from a T-sequence TS in Sections 4.2 and 4.3, respectively. In the first scheme, we construct the RV diagramfor TS, which is a tree with the path from the root to the node representing a feasible prefix or race variant of TS. The nodes in the RV diagram for TS are generated by considering all the possible interleavings of T-events. It is an
n-ary tree if the number of concurrent clients is n. For a node in the RV diagram for TS, the path from the root node of the diagram to this node is a totally ordered sequence of T-events. The RV diagram scheme was first proposed in [7], which uses the version number of variable access to determine if there exists a different race outcome in the nodes of the RV diagram. However, the transaction order is insufficient for race analysis in a T-sequence. As shown in the example of Section 4.1 (Fig. 5b), two T-events TS(C1,2) = {DB2,1, C1, 2, [TA3, insert, 4]} and TS(C3,1) = {DB2,2, C3, 1, [TA4, insert, 5]} access the same database with different transaction orders, but do not race. The reordering of these two T-events cannot change the execution result. Thus, the
algorithm for generating race variants from the read-write sequence presented in [7] cannot be applied to the T-sequence. Since the version number is insufficient for determining whether race conditions are changed, the RV diagram defined in [7] is insufficient. Algorithm 2 (see Fig. 9) is our new algorithm for generating an RV diagram based on the race graph. Each node N in the RV diagram for TS contains aclient-transaction-order vectorand arace graph:
Client-transaction-order vector: (I1;I2;. . .;In), where Ij;
1jn, is the order number of the last T-event in the
jthclient that is executed for the generated node N.
Race graph: This represents a race variant if N is a race-variant node; otherwise, it represents a feasible prefix of the T-sequence if N is a prefix node.
The following algorithm (Fig. 9) shows how to drive race variants from a T-sequence.
Algorithm 2 generates an RV diagram to simulate the execution of the issuing of T-events by clients. Each generating of a child node represents an execution of a T-event by a client. A prefix node corresponds to a feasible prefix of TS which does not have different race outcomes. A race-variant node contains a sequence of T-events which have different race outcomes. In case a node is marked, it is either a race-variant node or a prefix node which does not need to generate child nodes anymore.
We use the T-sequence TS in the running example of Section 4.1 (Fig. 5b) to illustrate Algorithm 2. Fig. 10 is part of the RV diagram generated from TS. Note that TS(Ci,j) is abbreviated to Ti,j in the figure. There are two race-variant Fig. 5. An example of the race graph.
Fig. 6. An example illustrating the definition of feasible prefix.
nodes. Consider the prefix node with client transaction order (1,1,0). In Step 3 of Algorithm 2, observe the following:
j=1: This generates a prefix node whose client transaction order is (2,1,0). Since there is already a prefix node with client transaction order (2,1,0), this prefix node is labeled “marked.”
j=2: This generates a node whose client transaction order is (1,2,0), which adds TS(C2,2) to the race graph of its own. In the target T-sequence TS, TS(C3,1) happened before TS(C2,2), but TS(C3,1) is not in its race graph. Thus, it is a race-variant node.
j=3: This generates a prefix node whose client transaction order is (1,1,1).
Fig. 11 shows all the race variants derived by Algorithm 2 with the T-sequence TS in the running example of Section 4.1 (Fig. 5b) as input. There is a total of 10 race variants derived:
RV2-1 to RV2-10. We can easily find out that some race variants are feasible prefixes of others. For example,RV2-3is a feasible prefix of both RV2-6 and RV2-10; RV2-10 is a feasible prefix ofRV2-6. It will make the prefix-based replay of the three race variantsRV2-3,RV2-6, andRV2-10able to generate the same T-sequence. This means that during the testing process it will be possible to duplicate tests. We discuss this issue in more detail in Section 4.5.
4.3 Analyzing Edges in a Race Graph to Derive Race Variants
In this section, we propose another algorithm for deriving race variants. Instead of generating an RV diagram, it only has to analyze the edges in the race graph of a T-sequence. The algorithm tries to reverse the direction of the edges so as to derive race variants of a T-sequence. The analysis of the edges is based on thein-edgesof each T-event in the race Fig. 9. Algorithm 2.
graph (see Definition 4 (Fig. 13) for the definitions of the in-edge and out-in-edge).
We use Fig. 12 to illustrate Definition 4. The in-edges of T-event TS(2,2) are (TS(1,1), TS(2,2)) and (TS(1,2), TS(2,2)). The out-edges of T-event TS(2,2) are (TS(2,2), TS(3,1)) and (TS(2,2), TS(3,3)). With Definition 4, we now present Algorithm 3 (see Fig. 14).
For each T-event, if it is with k in-edges, the algorithm generates 2k1 subsets of in-edges and tries to reverse
them so as to generate race variants. Also, after reversing some edges, a cycle means that the sequence is infeasible, i.e.,e1happened beforee2ande2happened beforee1is not
possible. We use Fig. 15 to illustrate Algorithm 3. In Step 4 of Algorithm 3, assume E is the second event of Client 2. There are two in-edges of E. We show the three cases for
i¼1,i¼2, and i¼3. Note that iis the index variable in Step 4.
Fig. 11 shows all the race variants derived by Algorithm 3 with the T-sequence TS in Section 4.1 (Fig. 5b) as input. Note that TS(Ci,j) is abbreviated to Ti,j. There is a total of six race variants derived:RV3-1toRV3-6. However, Algorithm 2 derives 10 race variants from the same T-sequence. This motivates us to study the correctness of the two algorithms and the differences between them. In Section 4.4, we prove that reachability testing with the two algorithms to derive race variants can execute all the possible T-sequences if the client/server program always terminates. In Section 4.5, we compare the two algorithms. We focus on the running time
analysis and investigating the duplication of the same tests by the two algorithms.
4.4 The Correctness of Algorithms 2 and 3
In this section, we investigate whether the two algorithms can perform the exhaustive testing for a client/server program. We first need two additional definitions (see Figs. 17 and 18).
In Step 3 of Algorithm 4 (see Fig. 19), it first adds a T-event e and its edges to GMCF P ife is a
not-happened-before node in bothRG1andRG2. Then, it removesefrom
RG1 andRG2. This process continues until there does not
exist any T-event which is a not-happened-before node in both RG1 and RG2. The algorithm shows one method of
deriving the MCFP from two T-sequences, from which we can easily know that the MCFP of two T-sequences is unique. Fig. 16 shows an example for the definition of the MCFP. In the example, the MCFP has three T-events. In the extreme case, the MCFP of two T-sequences has no T-event. Now, we show why all the feasible T-sequences can be derived if we use Algorithms 2 and 3 to derive race variants in reachability testing (see Theorem 1 and Theorem 2).
Theorem 1. Assume that every execution of a client server program P with input X terminates. According to Algorithm 2, reachability testing of P with input X derives and executes all feasible T-sequences of P with input X.
Proof.Refer to Fig. 1. Assume we have a T-sequence S of P with input X. Since every execution of P with input X terminates, we assume that the number of T-events in S is n, where n is a positive integer, which we denote in the following asjSj ¼n. Assume a feasible T-sequenceS0of P with input X. We prove that conducting the process shown in Fig. 1 by applying Algorithm 2 to derive race variants can always perform a testing of P with X andS0. LetTMbe the MCFP of S andS0, and letjTMj ¼m, m is
a positive integer. It is trivial thatm jSjandm jS0j. Fig. 11. Race graphs of the race variants derived by Algorithms 2 and 3.
Reachability testing will first invoke Algorithm 2 with S as its input (see Fig. 1). As the nodes in the RV diagram are generated by considering all the possible interleav-ings of T-events of S, there must exist at least one prefix node whose race graph is the same as the race graph of
TM in the generated RV diagram. This is because TM is
also a feasible prefix of S. We denote this nodeNM. In the
following, we prove that there must be at least one child node ofNM which is a feasible prefix ofS0.
LetTSMbe the set of T-events in S but not inTM, and
let TS0M be the set of T-events in S0 but not in TM.
Without loss of generality, let e be a not-happened-before event in Race-graph (TS0M). First, we have thatTMþeis
a feasible prefix ofS0. It is trivial that e must not be a not-happened-before event in Race-graph (TSM). If e is a
not-happened-before event in both Race-graph (TSM)
and Race-graph (TS0M), then it must be that e2TM.
According to Step 3 of Algorithm 2, this will generate a child node thathas Race-graph (TMþe) as its race graph.
Since there must exist an event thathas happened before e in Race-graph(S), this child node is a race-variant node of the generated RV diagram.
Then, the reachability testing will conduct a prefix-based replay of TMþe. Assume that this produces a
T-sequenceS1. It is trivial thatTMþeis a feasible prefix of
S1andS0. LetTMþ1be the MCFP ofS1andS0. We easily see
that jTMþ1j jTMj þ1¼mþ1. Again, applying
Algo-rithm 2 toS1will derive a race variant of sizejTMþ1j þ1.
Repeating the above process, we will eventually derive a T-sequence that is equal toS0in the prefix-based replay.tu Fig. 14. Algorithm 3.
Theorem 2. Assume that every execution of a client/server program P with input X terminates. According to Algorithm 3, reachability testing of P with input X derives and executes all feasible T-sequences of P with input X.
Proof.Refer to Fig. 1. Assume that we have a T-sequence S of P with input X. Since every execution of P with input X terminates, we assume that jSj is n, n is a positive integer. Assume a feasible T-sequenceS0of P with input X. We prove that conducting the process in Fig. 1 by applying Algorithm 3 to derive race variants can always perform a testing ofS0.
Let TM be the MCFP of S and S0, and let jTMj ¼m,
where m is a positive integer. It is trivial that m jSj and m jS0j. The reachability testing will first invoke Algorithm 3 with S as its input (see Fig. 1). The same way as the proof of Theorem 1, let TSM be the set of
T-events in S but not in M, and TS0M be the set of
T-events in S0 but not in M. Without loss of generality, let e be a not-happened-before event in Race-graph (TS0M). It is trivial that e must not be a
not-happened-variants, Step 4 in Algorithm 3 will change the direction of the in-edges of an event of S in any combination, i.e., 2jnumber of in-edgesj1. The changing of
in-edge directions of e in S to derive TMþe must fall
into one of the 2jnumber of in-edgesj1 cases (see Fig. 20).
Then, the reachability testing will conduct a prefix-based replay of TMþe. Assume that this produces a
T-sequenceS1. It is trivial thatTMþeis a feasible prefix
of S1 and S0. Let TMþ1 be the MCFP of S1 and S0. We
easily see thatjTMþ1j jTMj þ1¼mþ1. Again,
apply-ing Algorithm 3 toS1 will derive a race variant of size
jTMþ1j þ1. Repeating the above process, we will
even-tually derive a T-sequence that is equal toS0in the
prefix-based replay. tu
Fig. 18. Definition 6.
4.5 Analysis of Algorithms 2 and 3
The analyses of the algorithms are divided into two parts. One is the complexity analysis of Algorithms 1, 2, and 3 (Complexity Analyses 1-3). The other is the analysis of duplicating tests during reachability testing (Theorems 3-9).
Complexity Analysis 1: The running time of the algorithm for deciding if Tu and Tv exhibit a transaction
race in Definition 1 is OðmnMP KL2 Þ in which m, n, and MP KL are the number of accessed table records in Tu
and Tv, and maximal number of elements in the
Primary_ Key_List, respectively.
Proof.For each pair of the
½Table nameui;Operationui;Primary Key Listui
inTu and
½Table namevj;Operationvj;Primary Key Listvj
in Tv, 1im, 1jn, it needs the running time
OðM2
PKLÞto check if
Primary Key Listui[Primary Key Listvj
is the empty set. Since there aremnpairs of accessed table records to be checked, the running time is
OðmnM2
PKLÞ. tu
Complexity Analysis 2:Assume that the T-sequenceTSis from the execution ofPwithnclients. Let|TS|,K,MAT R, and
MP KLbe the number of T-events inTS, the number of edges
between different clients in the race graph ofTS, the maximal number of accessed table records in T-events ofTS, and the maximal number of elements in Primary_Key_List, respectively. The running time for Algorithm 1 to generate the race graph ofTSisOðM2
AT RMP KL2 jT Sj
2
þKÞ. The running time for Algorithm 2 to derive race variants fromTS
isOðjT Sj2KþnjT SjþM2
AT RMP KL2 jT Sj
2
Þ.
Proof. For Algorithm 1, it first needs OðM2
AT RMP KL2 jT Sj ðjT Sj 1Þ=2Þ ¼OðM2
AT RMP KL2 jT Sj
2
Þto com-pute all the transaction race relationships between jT Sj T-sequences. Steps 1, 2, and 3 of Algorithm 1 are bound on
OðjT SjÞ, OðjT SjÞ, and OðKÞ, respectively. Thus, the running time of Algorithm 1 isOðM2
AT RMP KL2 jT Sj
2
þ jT Sj þ jT Sj þKÞ ¼OðMAT R2 MP KL2 jT Sj2þKÞ.
For Algorithm 2, the running time for identifying the happened-before relations between jT SjT-events is
OðjT Sj2KÞ. The RV diagram for T S is an n-ary tree with a maximal depth of jT Sj. It is because each expanding of a child node from a prefix node simulates an execution of a T-event and the total number of T-events is jT Sj. The number of nodes in an n-ary tree with depth jT Sj is ðnjT Sjþ11Þ=ðn1Þ. We
have OððnjT Sjþ11Þ=ðn1ÞÞ ¼OðnjT SjÞ. Algorithm 2 is
bound on the running time to generate the race graph and the number of nodes in the generated RV diagram. We have the running time of Algorithm 2 is OðjT Sj2KþnjT SjþM2 AT RMP KL2 jT Sj 2 þKÞ ¼OðjT Sj2 KþnjT SjþM2 AT RMP KL2 jT Sj 2 Þ. tu
Complexity Analysis 3: The running time for Algorithm 3 to derive race variants fromTSis OðjT Sj2 2KþM2
AT RMP KL2 jT Sj
2
þKÞÞ. Note thatjT Sj,K,MAT R,
andMP KLare defined in Complexity Analysis 2.
Proof.Assume that thejT SjT-events areV1; V2;. . ., andVjT Sj.
Let ki be the number of in-edges of Vi. Then, we have
K¼k1þk2þ. . .þkjT Sj. In addition to generating the
race graph for TS, the running time of Algorithm 3 is bound on the time for executing Step 4 for each T-event, i.e.,
OðjT Sj 2ki1ÞforV
i. Note thatOðjT SjÞis to check if there
is a loop inG0. Thus, the running time of Step 4 isjT Sj 2k11þ jT Sj 2k21þ. . .þ jT Sj 2kjT Sj1<jT Sj 2Kþ jT Sj 2Kþ. . .þ jT Sj 2K¼ jT Sj2 2K. The running time isOðjT Sj22KþM2 AT RMP KL2 jT Sj 2 þKÞÞ. tu According to Fig. 11, we can easily determine that the number of race variants generated by Algorithms 2 and 3 may be different. That is, given the same T-sequence, Algorithms 2 and 3 may derive different sets of race variants. We can divide the race variants into six groups. For example, in the race variants derived by Algorithm 2,
RV2-3 is a feasible prefix of both RV2-6 and RV2-10 and
RV2-10is a feasible prefix of RV2-6in group 5. A similar situation also holds in groups 1 and 6.
Theorem 3. Assume that RV1 and RV2 are two race variants derived by Algorithm 2 from a T-sequence. It is possible that RV1 is a feasible prefix of RV2.
Proof. This comes directly from the example shown in Fig. 11, where RV2-3 is a feasible prefix of both RV2-6
and RV2-10. tu
Theorem 4.Assume two racevariants RV1 and RV2. If RV1 is a feasible prefix of RV2, then the prefix-based replay of RV1 and RV2 with the same input may produce the same T-sequence. Proof. Assume that the prefix-based replays of RV1 and
RV2 produce two feasible T-sequences T1 and T2, respectively. T1 is prefixed with RV1. The T-events of T1 after RV1 are obtained in the monitor phase of the prefix-based replay. Since the execution of T-events in the monitor phase is not controlled, it is possible to produce any feasible sequences of T-events. Since T2 is also prefixed with RV1 (because T2 is prefixed with RV2 and RV2 is prefixed with RV1), it is possible that T1 is
equal to T2. tu
Theorem 3 and Theorem 4 show that it is possible to duplicate the same test if we employ Algorithm 2 to derive the race variant. However, it is impossible for Algorithm 3 to duplicate the same test in this situation (see Theorem 5).
Theorem 5.Assume a T-sequence TS, and that RV1 and RV2 are two race variants derived by applying Algorithm 3. It is impossible that RV1 is a feasible prefix of RV2.
Proof. According to the operation of Theorem 3, we consider the following two cases: 1) We assume that RV1 and RV2 are generated by altering the in-edges of the same node in step 4 of Theorem 3. If the node is e, then e must be in a T-event in RV1 and RV2. However, according to Step 4 of Theorem 3, the in-edges of e of RV1 and e of RV2 must be different. Thus, it is
impossible for RV1 to be equivalent to RV2 in this case. 2) We assume that RV1 and RV2 are generated by altering the in-edges of the different nodes in Step 4 of Theorem 3. Assume that RV1 is generated by altering the in-edges of node e. If RV1 is a feasible prefix of RV2, e must also be a T-event of RV2. However, since we apply Theorem 3 to change the in-edges of e to derive RV1 and RV2 are derived by altering the edges of another node, the in-edges of e must be different in RV1 and RV2, so it is impossible for RV1 to be a feasible prefix of RV2. It is clear that each case leads to a contradiction. tu We have shown in Theorem 5 that it is impossible for Algorithm 3 to derive two racevariants from the same T-sequence, for which one is the other’s feasible prefix. Also, in the example shown in Fig. 11, none of the race variants derived by Algorithm 3 is another’s feasible prefix. However, this does not mean that performing reachability testing with Algorithm 3 to derive race variants will not duplicate the same test. Consider the race graph G of a T-sequence shown in Fig. 21. By changing the in-edges of two T-events E1 and E2 in G, we can derive at least two race variants RV1 and RV2. Fig. 21 also shows that it is possible for two T-sequences obtained by applying prefix-based replay of RV1 and RV2 to be identical. Since each prefix-based replay of a racevariant conducts a test, it is obvious that Algorithm 3 cannot also avoid duplication of the same test. The example shown in Fig. 21 motivates Theorem 6. In addition to Theorem 4, Theorem 6 shows another situation in which the prefix-based replay of two race variants may produce the same T-sequence.
Theorem 6.Assume two race variants RV1 and RV2. Let M be the MCFP of RV1 and RV2. If (set of nodes in Race-graph (RV1-M)\set of nodes in Race-graph (RV2-M))2is the empty set, then the prefix-based replays of RV1 and RV2 may produce the same T-sequence.
program without any control and recording the order of the executed synchronization events. Consider the prefix-based replaying of RV1. Because we do not control the execution of the concurrent program in Step 2, it is possible to produce a T-sequence thathas RV2 as its
feasible prefix. tu
Following Theorem 6, Theorem 7 discusses another situation in which it is impossible for the prefix-based replays of two race variants to produce the same T-sequence.
Theorem 7.Assume two race variants RV1 and RV2. Let M be the MCFP of RV1 and RV2. If (set of nodes in Race-graph (RV1-M) \set of nodes in Race-graph (RV2-M)) is not the empty set, then the prefix-based replays of RV1 and RV2 produce different T-sequences.
Proof. We first assume e2 (set of nodes in Race-graph (RV1-M) \ set of nodes in Race-graph (RV2-M)). Note that e is a T-event. Let N1 and N2 be the sets of not-happened-before nodes in Race-graph (RV1-M) and Race-graph (RV2-M), respectively. According to the definition of MCFP, we haveN1\N2¼ . We consider the following two cases: 1) Assume that node e is in one of N1 and N2. Without loss of generality, we assume e is in N1. Since N1\N2¼ , we have that e is in Race-graph (RV2-M-N2). It is obvious that the replay of RV1 and RV2 will not produce the same T-sequence because one of the nodes in N2 must occur before e in RV2. 2) Assume that e is not in N1 and N2 (i.e., e is in Race-graph (RV1-M-N1) and Race-Race-graph (RV2-M-N2)). It is impossible for the replaying of RV1 and RV2 to produce the same T-sequence because there must exist a node
x2N1 that has occurred before e in RV1 plus a node
y2N2 that has occurred before e in RV2. Since
N1\N2¼ , we havex6¼y. tu Theorem 8.Assume a T-sequence TS, and that RV1 and RV2
Race-graph (RV2-3-M) \ set of nodes in Race-graph(RV2-10-M)) is the empty set. tu Theorem 9. Assume a T-sequence TS, and that RV1 and RV2 are two race variants derived by applying Algorithm 3. It is possible that (set of nodes in Race-graph (RV1-M) \ set of nodes in Race-graph (RV2-M)) is the empty set.
Proof. This comes directly from the example shown in Fig. 21. Set of nodes in Race-graph (RV1-M) is E1 and set of nodes in Race-graph (RV2-M) is E2. E1 \ E2 is the
empty set. tu
With Theorems 6 and 9, we can conclude that it is also possible to duplicate the same test if we employ Algorithm 3 to derive the race variants during reachability test. From the above discussion, we conclude that both Algorithms 2 and 3 may conduct the same test. However, the probability of Algorithm 3 duplicating the same test seems lower. Fig. 11 is an illustrative example. The experimental results in Section 6 also demonstrate this.
5
P
REFIX-B
ASEDR
EPLAY FORD
ATABASET
RANSACTIONSIn addition to the implementation of Algorithms 2 and 3, we have also implemented a prefix-based replay technique for race variants based on T-sequences. The prefix-based replay of a racevariant includes two steps: 1) controlling the execution of the program by following the execution order specified in the racevariant, and then 2) executing the program without any control and recording the order of the executed T-events. Note that the two steps are the replay and monitor phases we mentioned in Section 2. After the two steps, the program has been executed once. This procedure conducts a test and produces a T-sequence. For example, see Fig. 21. RV1 is the race variant to be replayed in the first step and we obtain six T-events in the second step. It produces a T-sequence with nine T-events.
A scheme for performing monitoring and replaying of concurrent shared-memory programs has been presented previously [11]. The scheme cannot perform prefix-based replay of a racevariant because it cannot switch between the monitoring and replay phases during execution. It must first replay the execution order according to the target race variant and then monitor the execution of the concurrent program to conduct the two steps of prefix-based replay. A method has been proposed [8] for performing prefix-based replay of a racevariant, but it can only be applied to concurrent software which uses shared memory or sema-phores to perform process synchronization. In this paper, we propose a scheme that performs prefix-based replay for an SQL client/server program.
Fig. 22 shows how to modify the original client/server program to perform prefix-based replay of a race variant. First of all, there must be some variables added to the modified program, which are used to control the replaying and monitoring of the program. They can be divided into two classes: 1) those stored in the tables of databases and 2) local variables in each client process. These variables should be initialized before performing the prefix-based replay. For each T-event E in the original program, anentry
and exit protocol must be inserted before and after E, respectively.
In this paper, we present two methods for performing prefix-based replay based on the data structure of the T-sequence defined in this paper. For the replay phase in Step 1 and monitor phase in Step 2 of the prefix-based replay, we need a mechanism to synchronize the execution of concurrent clients. In this paper, we present the entry and exit protocols based on two types of synchronization mechanism: one is based on the built-in synchronization functions in the SQL database system, and the other uses Java thread synchronization.
We first present the entry and exit protocols based on the built-in SQL database synchronization mechanism. Note that Client1 to Clientn are processes executed in different
machines. For clients executed in the same machines, we can consider it as special case of it. There are some variables added to the databases and client processes. The variables added to the databases are as follows:
. We add a table Transaction_order to each database. There is only one row in this table. The primary key isDTO, which represents the transaction order of the database in replaying and monitoring. The initial value ofDTOis 1.
. We add a new database called Un_replay_cli-ent, which has only one table, Un_finish_re-play_client. There is only one row in this table, and its primary key isUFRC. This variable represents the number of clients that have not finished the replay phase. The initial value of UFRC is the total number of clients in the application.
The local variables added to each client process are as follows:
. Mode: This variable represents whether the client is in Monitor or Replay mode. Its initial value is Replay.
. Number of replay events: This constant variable
represents the number of T-events which the client should replay. Assume that RV is the race variant Fig. 22. How to modify the source program to perform prefix-based replay.
that is going to be prefix-based replayed. It is initialized to the number of the T-events of the client inRV.
. Client transaction order: This variable
re-cords the order of T-events which the client is going to replay or monitor. Its initial value is 1.
The following local procedures should be invoked in the client process:
. LockDB(Name of Database): This is a function
to lock the use of the database Name of Data-base. While a database is locked by a client, all the database transactions issued by other clients are detained until the client callsUnLockDB(Name
of Database). Almost all implementations of
database servers support a way to lock the database, such as the setTransactionIsolationmember function of the Connection object in Java Database Connectivity [24].
. UnLockDB(Name of Database): This is a function
to unlock the databaseName of Database.
. My_next_DB_transaction_order(i): This
function returns the data transaction order according to the client transaction orderi.
Figs. 22 and 23 show the pseudocodes of entry and exit protocols, respectively, for the prefix-based replay of a race variant.
In the second case, we haveClient1toClientn as threads
in the same machine, which means that they can use their shared memory to synchronize themselves. This is based on
Lock_DB(Un_replay_Client); Num_UFRC = Select UFRC From
Un_replay_client. Un_finish_replay_ client; Update Un_replay_ client.
Un_finish_replay_ client Set UFRC = Num_UFRC - 1;
UnLock_DB(Un_replay_Client);
which is replaced with the following Java subroutine:
synchronized private void UFRC()
{
Un_finish_replay_client = Un_finish_replay_client - 1;
}.
Also, the Lock_DB(DB) and UnLock_DB(DB) functions are replaced withwait(DB.sem)andsignal(DB.sem), which are Java subroutines that implement the binary semaphore operations defined in [4]. The DB.sem sub-routine implements a semaphore with an integer value of either 0 or 1. Figs. 24 and 25 are the entry and exit protocols based on Java thread synchronization.
6
E
XPERIMENTSThis section presents the results from the experimental testing of three programs: Program 1 is a client/server program with three clients, where each client issues three SQL transactions.Program 2is a client/server program with three clients, where each client issues five SQL transactions. Fig. 23. Theentryprotocol based on built-in database synchronization.
generate the race variants, with Figs. 22 and 23 as the entry and exit protocols for prefix-based replay of a race variant.
RT_2 is the reachability testing using Algorithm 2 to generate the race variants, with Figs. 24 and 25 as the entry and exit protocols for prefix-based replay of a race variant.
RT_3 is the reachability testing using Algorithm 3 to generate the race variants, with Figs. 22 and 23 as the entry and exit protocols for prefix-based replay of a race variant.
RT_4 is the reachability testing using Algorithm 3 to generate the race variants, with Figs. 24 and 25 as the entry and exit protocols for prefix-based replay of a race variant. All the client processes and database servers are executed on PCs with 800-MHz Intel Pentium III processors and the MS Windows 2000 operating system. Client
programs and database servers are located in the same network segment (with a bandwidth of 100Mbps). The database system is Microsoft SQL Server 7.0 [26]. The client application programs and code for implementing of reach-ability testing are compiled and executed in Java Runtime Environment, Standard Edition (build 1.3.0_01) [27].
Fig. 27a shows the experimental results froma multiple-execution test of Program 1. It is obvious that this program duplicates a lot of tests since, from 1,000 tests, an average of only 5.2 different T-sequences are derived. We repeat the testing 10 times. Fig. 27b shows the experimental results from using schemes RT_1, RT_2, RT_3, and RT_4 to test Program 1. Each test is repeated three times. We discover Fig. 24. Theexitprotocol based on built-in database synchronization.
Fig. 25. Theentryprotocol based on Java thread synchronization. For ease of understanding, we still show the pseudocode rather than real Java souce code.
that using Algorithm 2 to derive race variants tends to duplicate more tests than using Algorithm 3.
Fig. 27c shows the time required to perform tests ME, RT_1, RT_2, RT_3, and RT_4 on Program 1. This figure shows that using Java thread synchronization (Figs. 24 and 25) to perform prefix-based replay is much more efficient than synchronizing clients with the database synchroniza-tion mechanism (Figs. 22 and 23). Consider the programs which use the SQL database synchronization mechanism, i.e, ME, RT_1, and RT_3. Although the multiple-execution testing scheme required the least time to perform a test (i.e., 8.8 seconds), it could only derive an average of six different T-sequences from 1,000 tests. Note that the time to perform a test includes initializing the tables in the database and the execution of all clients. Fig. 27d focuses on two points. The first one is the required time to generate race variants from a T-sequence, i.e., the execution times of Algorithms 2 and
3. The second one is the number of duplicate T-sequences during reachability testing. The figure shows that Algo-rithm 2 is slightly faster than AlgoAlgo-rithm 3. However, considering the number of duplicated tests, Algorithm 2 is much worse than Algorithm 3. In this example, Algorithm 3 (RT_3 and RT_4) duplicates less than five tests among 24 T-sequences, whereas Algorithm 2 (RT_1 and RT_2) duplicates more than 80 tests.
The experimental results of Programs 2 and 3 are shown in Figs. 28 and 29, respectively.
From these experimental results, we draw the following conclusions. First, the multiple-execution testing scheme is not a practical way of testing SQL client/server programs since a tremendous number of the tests are duplicated. Also, most T-sequences are never exercised. Second, although Algorithm 2 can always exhaustively execute all possible tests; this is associated with the duplication of Fig. 26. Theexitprotocol based on Java thread synchronization.
many tests. Algorithm 3 duplicates fewer tests than Algorithm 2, with the result that, although Algorithm 3 is slightly slower than Algorithm 2, reachability testing using Algorithm 3 to generate race variants is more efficient than using Algorithm 2. Third, using the intrinsic synchroniza-tion mechanism in an SQL server to perform prefix-based replay is much slower than using Java thread synchroniza-tion. Finally, RT_4 provided the best performance in all of our experiments.
7
C
ONCLUSIONIn this paper, we have provided a framework for exhaustive testing of a client/server database application that exhibits nondeterministic behavior. We prove theoretically that the proposed scheme can derive all the possible T-sequences of a database application if each execution of application terminates, and our experimental results support this.
In the key technology of reachability testing (i.e., for deriving race variants from a SYN-sequence), instead of generating an RV diagram (Algorithm 2) we have devel-oped a new algorithm (Algorithm 3) that is based on the edge analysis of the race graph of the target SYN-sequence. Our theoretical analysis suggests that Algorithm 3 dupli-cates fewer tests, which is confirmed by the results of our experiments. This algorithm can be applied to the data-access model of database applications, which is the most
general type of model and, hence, the algorithm can be applied to any data-access model for reachability testing. The Java byte code of it can be obtained at http:// bashful.ice.ntnu.edu.tw/~ghhwang/RV_gen.zip. We con-sider the algorithm to be a significant breakthrough in reachability-testing technology.
To test a concurrent program having a huge or infinite number of feasible T-sequences, we can combine reachability testing with some strategies for selecting T-sequences. There is room for further investigation.
A
CKNOWLEDGMENTSThe authors would like to thank the anonymous referees for a number of useful suggestions to improve the paper. G.-H. Hwang and S.-J. Chang’s work was supported in part by the Republic of China National Science Council under grant 89-2218-E-260-016 and ROC MOE/NSC program for promoting academic excellence of universi-ties under grant 89-E-FA04-1-4.
R
EFERENCES[1] C.E. Mcdowell and D.P. Helmold, “Debugging Concurrent Programs,”ACM Computing Surveys,vol. 21, no. 4, Dec. 1989.
[2] K.C. Tai and R.H. Carver, “Testing of Distributed Programs,” Parallel and Distributed Computing Handbook, A.Y. Zomaya, ed., chapter 33, McGraw-Hill, 1996.
Fig. 29. Experimental result of Program 3. (a) Using ME to test Program 3. (b) Reachability testing of Program 3. (c) Time required to perfrom reachability testing of Program 3. (d) Time required to generate the race variants for Program 3.
Fig. 28. Experimental result of Program 2. (a) Using ME to test Program 2. (b) Reachability testing of Program 2. (c) Time required to perfrom reachability testing of Program 2. (d) Time required to generate the race variants for Program 2.
[7] G.-H. Hwang, K.C. Tai, and T.L. Huang, “Reachability Testing: An Approach To Testing Concurrent Software,”Int’l J. Software Eng. and Knowledge Eng.,vol. 5, no. 4, pp. 493-510, Dec. 1995.
[8] K.-C. Tai, “Reachability Testing of Asynchronous Message-Passing Programs,”Proc. Second Int’l Workshop Software Eng. for Parallel and Distributed Systems,1997.
[9] A. Bechini and K.-C. Tai, “Timestamps for Programs Using Messages and Shared Variables,” Proc. Int’l Conf. Distributed Computing Systems,pp. 266-273, May 1998.
[10] Y. Lei and K.-C. Tai, “Efficient Reachability Testing of Asynchro-nous Message-Passing Programs,” Proc. Eighth IEEE Int’l Conf. Eng. for Complex Computer Systems,pp. 35-44, Dec. 2002.
[11] T.J. LeBlanc and J.M. Mellor-Crummey, “Debugging Parallel Programs with Instant Replay,”IEEE Trans. Computers, vol. 36, no. 4, pp. 471-482, Apr. 1987.
[12] R.A. Davies, R.J.A. Beynon, and B.F. Jones, “Automating the Testing of Databases,”Proc. First Int’l Workshop Automated Program Analysis, Testing and Verification,June 2000.
[13] D. Chays, S. Dan, P.G. Frankl, F.I. Vokolos, and E.J. Weyuker, “A Framework for Testing Database Applications,”Proc. ACM Int’l Symp. Software Testing and Analysis,2000.
[14] M.J. Carey, D.J. DeWitt, and J.F. Naughton, “The 007 Benchmark,” Proc. 1993 ACM SIGMOD Int’l Conf. Management of Data,pp. 12-21, May 1993.
[15] J. Gray, P. Sundaresan, S. Englert, K. Baclawski, and P.J. Weinberger, “Quickly Generating Billion-Record Synthetic Data-bases,” SIGMOD Record (ACM Special Interest Group on Manage-ment of Data),vol. 23, no. 2, pp. 243-252, June 1994.
[16] D. Slutz, “Massive Stochastic Testing of SQL,” Proc. Conf. Very Large Databases,pp. 618-622, Aug. 1998.
[17] Trans. Processing Performance Council, TPC-Benchmark C, 1998.
[18] H. Chu and J. Dobson, “Towards Quality Programming in the Automated Testing of Client/Server Application,” Proc. Pacific Northwest Software Quality Conf. ’98 and Proc. Int’l Conf. Software Quality ’98,Oct. 1998.
[19] D. Helmbold and D. Luckham, “Debugging ADA Tasking Programs,”IEEE Software,vol. 2, no. 2, pp. 66-74, 1985.
[20] R.N. Taylor, “A General-Purpose Algorithm for Analyzing Concurrent Programs,”Comm. ACM,vol. 21, no. 7, July 1978.
[21] M. Young and R.N. Taylor, “Combining Static Concurrency Analysis with Symbolic Execution,” IEEE Trans. Software Eng., vol. 14, no. 10, Oct. 1988.
[22] C.E. McDowell, “A Practical Algorithm for Static Analysis of Parallel Programs,” J. Parallel and Distributed Computing, vol. 6, pp. 515-536, 1989.
[23] L. Lamport, “Time, Clocks, and the Ordering of Events in a Distributed System,”Comm. ACM,vol. 21, no. 7, pp. 558-565, July 1978.
[24] M. Gruber, “Mastering SQL,”Sybex,Jan. 2000.
[25] G. Steele, J. Gosling, and G. Bracha,Java(TM) Language Specifica-tion,second ed. B. Joy, ed., Addison-Wesley, June 2000.
[26] S. Bjeletich, G. Mable, and D.W. Solomon, “Microsoft SQL Server 7.0 Unleashed,”Sams,first ed., May 1999.
[27] Sun Microsystem, The Source for Java(TM) Technology, http:// java.sun.com, 2002.
testing, parallelizing compilers, Internet security, thin-client technolo-gies, and groupware.
Sheng-Jen Chang received the BBA degree while in the Department of Management In-formation Systems at National Cheng-Chi Uni-versity in 2000 and the MBA degree while in the Department of Information Management at Na-tional Chi-Nan University in 2002, respectively. He has been an associate researcher in the Telecommunication Laboratories of Chunghwa Telecom Co., Ltd., Taiwan, since 2002. His research interests include concurrent program testing and database system testing.
Huey-Der Chureceived the PhD degree (1998) from the Centre for Software Reliability at the University of Newcastle upon Tyne, England, funded by the National Science Council in Taiwan. He is an associate professor at Takming College in Taiwan, as well as the executive director of the Chinese Software Quality Asso-ciation. His current research interests are in software process improvement, knowledge management, and quality management.
. For more information on this or any other computing topic, please visit our Digital Library athttp://computer.org/publications/dlib.
