A CLUSTERING-BASED HYBRID REPLICA CONTROL PROTOCOL FOR HIGH AVAILABILITY IN GRID ENVIRONMENT

In recent years, with the emergence of grid computing system, the number of distributed sites has become very large. When thousands of sites involved in a grid computing system, data replication can improve data availability, communication cost and provide fault-tolerance in the system. In the literature, many replica control protocols have been proposed for managing replicated data. However, in large scale distributed system, most of these protocols still require a bigger number of replicas for maintaining consistency, thus degrade the performance of the protocols. Therefore, in this study, we proposed a new replica control protocol named Clustering-Based Hybrid (CBH) protocol. CBH protocol employs a hybrid replication strategy by combining the advantages of two common replica control protocols into one to improve the performance of the existing protocols. We analyzed the communication cost and availability of the operations and compare CBH protocol with recently proposed replica control protocol named Dynamic Hybrid (DH) protocol. A simulation model was developed using Java to evaluate CBH protocol. Our results show that the proposed protocol provides lower communication cost and higher data availability than DH protocol.


INTRODUCTION
A grid is a very large scale distributed network computing system that is designed to provide reliable access to data and computational resources over a wide area network and across organizational domains (Foster et al., 2001;Krauter et al., 2002). In a largescale distributed system and dynamic network such as grid, data replication has been implemented to provide high availability, fault tolerance and increase the performance of the system (Lamehamedi et al., 2002;Latip et al., 2008;Mabni and Latip, 2011). Replication is a technique of creating multiple copies of data items at distributed sites. However, some issues arise in dealing with replication of data. A major issue with replication is maintaining consistency of the replicas (Belalem and Slimani, 2006;Yang et al., 2007;Radi et al., 2008). When data are replicated at several sites, a consistency control protocol must be enforced to ensure the consistent view of the data. Other issue is communication cost where, in general, as the system size grows, the cost of executing operations in replicated databases increases since many replicas may need to be accessed for maintaining the consistency of the replicas Science Publications JCS (Choi and Youn, 2012). The availability of a data on a large network is another issue (Lamehamedi et al., 2002;Lamehamedi et al., 2003;Latip et al., 2009) because geographically it is distributed and has different database management to share across the grid network. In a distributed system, copies or replicas of an object may be stored at several sites in the network. These multiple copies of an object must appear as a single logical object to the transaction which is known as one-copy equivalence (Bernstein and Goodman, 1984). Two operations which are read and write are allowed on replicated data. A Read Quorum (RQ) or Write Quorum (WQ) is defined as a set of copies that is sufficient to execute the read or write operation. The selection of a quorum must satisfy the quorum intersection property to ensure one-copy equivalence among the replicas and maintain their consistent state. The property stated that for any two operations o[x] and o'[x] on an object x, where at least one of them is a write, the quorums must have a nonempty intersection (Gifford, 1979). Thus, the basic property employed in maintaining consistency of replicated data is to require conflicting operations to lock at least one common copy and this is enforced by the replica control protocol.
In recent years, with the emergence of grid computing system, the number of distributed nodes has become very large. However, in large scale systems, most of the earlier replica control protocols still require a bigger number of replicas for read and write operations, thus degrade the performance of the protocols. The degradation of performance is due to the consistency enforcement where a large number of replicas must be accessed through the wide area network. In this study, we propose a new replica control protocol called Clustering-Based Hybrid (CBH) protocol for grid environment. The proposed protocol employs a hybrid replication strategy by combining the advantages of two common replica control protocols into one to improve the performance of earlier protocols. The proposed protocol groups nodes into clusters and organizes these clusters into a tree structure which enables the protocol to minimize the number of replicas for read or write operations as well as reduce the communication cost and provide high availability.

RELATED WORKS
This section describes the related work in replica control protocols. They differ from each other mainly in the number of replicas involved in performing the read and write operations. These protocols have satisfied the quorum intersection property to ensure one-copy equivalence among the replicas and maintain their consistent state.

Primary Copy Protocol
Primary Copy (PC) protocol is a simple algorithm that designates one copy of a data object as primary copy (Stonebraker, 1979;Ahamad et al., 1992;Zhou and Holmes, 1999). The node that maintains the primary copy of the data object is therefore responsible for maintaining consistency of the object. Any other node that maintains a non-primary copy is known as a slave copy. A read and write operations are executed only at the primary copy. For the write operation, once the primary copy is updated, it will be propagated out to all the nodes that maintain the slave copies. The communication cost for read and write operations are low because only one replica is accessed by the operations.
The advantage of this protocol is that it is easy to implement as compared to other protocols and it is one of the most widely implemented replication techniques. However, it has a limitation where, if the node that maintains a primary copy fails, then an update operation can no longer be performed until the node becomes available again. Agrawal and Abbadi (1990) proposed the Tree Quorum (TQ) protocol, which is based on a logical tree structure over a network (Agrawal and Abbadi, 1990). Figure 1 illustrates the diagram of a tree quorum structure with thirteen copies in a tree of height = 2 and degree of node D = 3, where every copy represents a replica. In this protocol, a read quorum consists of the root replica. If the root is inaccessible, then majority replicas of its children are added as members of this quorum. Furthermore, for every inaccessible replica, majority replicas of its children are added as members and so forth. The examples of valid read quorums of Fig. 1 are 1} and {2,3} when root replica is inaccessible and {5,6,8,9} when replica-2 and replica-3 are inaccessible. On the other hand, a write quorum is constructed by having as its members the root and any majority replicas of the root's children and any majority replicas of their children and so forth until the leaves are reached. Figure  1, the examples of valid write quorums are {1,2,3,5,6,8,9} and {1,3,4,9,10,11,12}. The strength of this protocol is that read operation may access only one replica. Thus, this protocol allows very low read cost in the best case. However, it has some drawbacks such as, as the level of the tree increases, the number of replicas grows rapidly, thus increases the communication cost. Fig. 1. A tree organization of 13 copies of data objects (Agrawal and Abbadi, 1990)

Dynamic Hybrid Protocol
A recently proposed hybrid replica control protocol is Dynamic Hybrid (DH) protocol (Choi and Youn, 2012). This protocol combines the grid and tree structure so that the overall topology can be flexibly adjusted using the tree height, number of descendants and grid depth. Figure 2 illustrates the network of Dynamic Hybrid protocol with 31 replicas in (3,3,2) topology, where the three arguments are representing the height h, number of descendants s and grid depth g respectively. In the tree structure of height h, the read operation reads the root replica or the s descendants of the root replica if the root is not available. The descendants of the root serve as the new root replica of the sub tree. The process is repeated until level h-1 is reached. Furthermore, in the grid network of depth g, read operation reads s replicas or go to the next level if one of the replicas is not available. Thus, the read cost is only 1, if the root replica is available. The examples of valid read quorums of Meanwhile, the write operation reads the root replica, one replica of the root's descendants, one replica of these previously selected replicas' descendants and so forth until the leaves are reached. Furthermore, in the grid network of depth g, write operation reads only one replica in each level down to the last level. Figure 2, the examples of valid write quorums are {R0, R1, R4, R13, R22} and {R0, R2, R7, R16, R27}.
The strength of this protocol is that it capitalizes the merits of tree and grid protocol to allow low operation cost and high availability. However, this protocol has drawback where, as the network size grows, large number of replicas still need to be accessed to maintain data consistency and therefore, degrade the performance of the system.

THE PROPOSED PROTOCOL
In this section, we present the system model and algorithm for the proposed protocol called Clustering-Based Hybrid (CBH) Protocol. In this study, the terms a number of nodes and a number of sites will be used interchangeably.

System Model
The system consists of N sites that communicate with each other by sending messages over a communication network. We assumed that sites fail independently and communication links do not fail to deliver messages. In CBH protocol, the N sites in the network are logically grouped into several nonintersecting groups. For the purposes of illustrations we shall divide the N sites into N disjoint groups with each group having approximately N sites. Each group is called a cluster. These clusters are logically organized as a tree of height h and descendantss. We defined the nodes in the tree to be a sequence of clusters C 0 , C 1 ,…C i , C i+1 , …C n . We assume that the nodes in each cluster are logically organized into two dimensional grid structures. For example, if CBH protocol consists of 81 nodes, it will be divided into 9 clusters with 9 nodes in each cluster. The nodes in each cluster will be logically organized in the form of 3×3 grid. Figure 3, an example of a ternary tree of height = 2 with 81 nodes is presented. Each cluster designates the middle node of the cluster as the cluster head which is colored in black in Fig. 3 and has the replica or primary copy of the data object. The center of the cluster is selected because it is the shortest path to get a copy of the data from most of the directions in the cluster.

Proposed Algorithm
Here, we describe the hybrid strategy of CBH protocol, where it combines the advantages of two common replica control protocols namely Tree Quorum (TQ) protocol and Primary Copy (PC) protocol. Figure  3, we show a system with 81 nodes for which we use TQ protocol on top of PC protocol as the replication strategy. In order to simplify the description, we assumed that every replica is assigned exactly one vote. Figure 3 shows a system with 81 nodes where the nine clusters named C 0 , C 1 , …,C 8 are "logical replicas" which are managed by using TQ protocol. The logical replica C 0 serves as the root cluster, whereas the logical replicas C 1 ,…,C 8 are its descendants. Each logical replica contains a cluster of physical nodes with one middle node called physical replica which has the replica or primary copy of the data object. The physical replica in the root cluster C 0 is called root replica. Thus, for a system with N nodes, there will be N clusters and N replicas. To illustrate the algorithm, the replication strategy involves two strategies: "Local Replication", where PC protocol is used for the replication strategy for managing the physical replica within a cluster and "Global Replication", where TQ protocol is used as the replication strategy for managing the logical replicas between clusters.

Read Operation
A read operation according to TQ protocol, which is used as the global replication, can be performed by reading the root replicaC 0 if C 0 is accessible or if the root replica is inaccessible then majority replicas of its children are added as members of this quorum. Furthermore, for every inaccessible replica, majority replicas of its children are added as members and so forth. In CBH protocol, a logical replica is available for a read operation if a read operation can be performed on the physical replica which it contains, using the applied local replication strategy which is PC protocol. This means that the precondition for reading a logical replica is a read quorum of RQ = 1 if it's contained physical replica is accessible for read operation. Thus, in Fig. 3, assuming that the root replica is accessible, by employing TQ Protocol, this leads to a minimal read cost of 1. However, if the root replica is not available, then the majority replicas of its children have to be accessed which results in the read cost of 2. The examples of valid read quorums of Fig. 3 are {C 0 } and {C 1 , C 2 } if the root replica is not available.

Write Operation
A write operation according to TQ protocol, which is used as the global replication, can be performed by reading the root replicaC 0 and any majority replicas of the root's children and any majority replicas of their children and so forth until the leaves are reached. In CBH protocol, a logical replica is available for a write operation if a write operation can be performed on the physical replica which it contains, using the applied local replication strategy which is PC protocol. This means that the precondition for writing a logical replica is a write quorum of WQ = 1 if it's contained physical replica is accessible for write operation. Thus, in Fig. 3, assuming that the root replica is accessible, a write operation performed on a logical replica requires WQ = 1 of its physical replica. Thus, by employing TQ protocol, we obtain a write cost of 7. An example of valid write quorum of Fig. 3 is {C 0 , C 1 , C 2, C 4 , C 5, C 7 , C 8 }.

Correctness of CBH Algorithm
Here, we demonstrate that the read and write quorums constructed by the CBH protocol will always have a non-empty intersection.

Theorem
The CBH protocol guarantees the intersection of read and write quorums.

Proof
= The proof is by induction on the height of the trees. If a read quorum consists of the root replica of the tree, it is guaranteed to have a nonempty intersection with any write quorum since a root replica must be accessed in any write quorum. On the other hand, if the read quorum consists of a majority of physical replicas for sub trees of height h, it is guaranteed to have at least one sub tree in common with any write quorum. Since the sub trees are of height h, the induction hypothesis guarantees that read and write quorums will have a non-empty intersection.

JCS
Hence, by induction, the CBH protocol guarantees non-empty intersection between read and write quorums.
Therefore, for the proposed CBH, the read/write conflict can be guaranteed to be detected because a read operation locks the root replica while a write operation locks the root replica and majority of physical replicas of the descendants. As for the conflict between two write operations, it can be guaranteed to be detected since any two write operations have to share the root replica. For example, in Fig. 3, valid RQ are {C 0 } and {C 1 , C 2 }, whereas, valid WQ is {C 0 , C 1 , C 2, C 4, C 5 , C 7 , C 8 }. Note that C 1 is included in both the valid RQ and WQ for detecting read/write conflict.

PERFORMANCE ANALYSIS AND COMPARISON
A simulation model was developed using Java to validate CBH protocol. In this section, we evaluate and compare the performance which are read and write communication costs and data availability of CBH and DH protocols.

Cost Analysis
The communication cost of an operation is computed based on the number of replicas involved in the read or write operation.
The read operation of DH protocol needs to access only the root replica which is very effective if the root replica is available. The minimum read cost C read is Equation 1: And the write cost C write that depends on the value of h and g is Equation 2: On the other hand, for CBH protocol, the communication cost is estimated based on the TQ protocol as given in (Chung, 1994) , where h denotes the height of the tree, D is the degree of logical replicas in the tree and M is the majority of D where Equation 3: Therefore, for a tree of height h, the maximum quorum size is M h and the communication cost for read operation C CBH , R is in the range of 1 ≤ C CBH , R ≤ M h . Meanwhile, the communication cost for write operation C CBH , W is given in Equation 4: where, i = 0,…,h.
For the read communication costs, both CBH and DH protocols have the same minimum read cost of 1, which is achieved by accessing only the root replica of the tree if the root replica is accessible. Table 1 and Fig. 4 illustrates the write communication cost of CBH and DH protocols for an example system with different total number of nodes, n = 81, 121, 225 and 289. Here, the protocols have the same number of descendants which is 3 but they differ in their heights based on the number of nodes. As an example, for 121 nodes in the system, DH protocol is based on height of 4 and grid depth of 3 which is configured in (4,3,3) topology and CBH is based on height of 2.
For the write operation as illustrated in Table 1 and Fig. 4, it is apparent that the overall write communication cost for CBH protocol is lower than that of DH protocol. It can be seen that CBH needs to access only 9 replicas on 225 copies for the write operation. On the other hand, for DH with 225 copies, the number of replicas that need to be accessed for performing write operation is 11. The result in Fig. 4 shows that the average write cost for CBH is 8.0 and DH is 9.5. Thus, CBH has reduced the average write cost by up to 15.8% compared to DH.

Availability Analysis
In this section, we analyzed the read and write availability of the protocols.
In DH protocol, the overall availability is obtained using the availability of tree and grid structure. The availability of read operation of the grid structure is as given in Equation 5: Thus, the overall availability of read operation for DH protocol is as given in Equation 6: The availability of write operation of the grid structure is as given in Equation 7: Thus, the overall availability of write operation for DH is as given in Equation 8: On the other hand, for CBH protocol, the overall availability is obtained using the combination of availability for PC protocol and TQ protocol. The availability of read operation of PC protocol is as given in Equation 9, where p is the probability of data file accessing between 0.1 to 0.9 and i is the increment of n.
Thus, the CBH overall availability of read operation for a tree of height h + 1 is as given in Equation 10: The availability of write operation of PC protocol is as given in Equation 11: Thus, the CBH overall availability of write operation for a tree of height h + 1 is as given in Equation 12:  Figure 5 and 6 show the read and write availability of CBH and DH protocols for 121 nodes respectively. We assume that all data copies have the same availabilities.   Figure 5, we can see that the read availability for CBH protocol is higher than DH protocol. This is due to CBH needs only to access small number of replicas for the read operations. The result shows that CBH protocol has average of 16.8% higher for read availability compared to DH protocol for all probabilities of data accessing. However, in case of write availability for 121 nodes, DH protocol is better than CBH protocol as illustrated in Fig. 6.

CONCLUSION
In this study, a new replica control protocol named Clustering-Based Hybrid (CBH) protocol has been proposed for the management of replicated data in large scale distributed system especially in grid environment. In the proposed protocol, the N sites in the network are logically grouped into several nonintersecting groups called clusters and organized in a tree structure. CBH protocol employs a hybrid replication strategy by combining the advantages of Primary Copy (PC) protocol and Tree Quorum (TQ) protocol to improve the performance of the existing protocols. In CBH, by grouping the nodes into clusters and having only one replica in each cluster has resulted in a small number of replicas involved in performing read and write operations. The analysis of CBH protocol was presented in terms of communication costs and data availability. Compared to DH protocol, CBH requires lower communication cost while providing higher read availability which is preferred for large scale distributed system.