What about the data?

We talk about services a lot. But what about the data they serve? Where do the Web pages reside to which a Web server provides access? People want to access their mail through POP, but where are their mail folders stored?

This is another very tricky part of clustering: sharing of data or common access to data. There are, as always, multiple solutions, depending primarily on the frequency of changes to your data, and the amount of data involved.

The different options for storing and accessing data in a cluster include:

• rsync

• Network File System

• Global File System

• Intermezzo

• Backend database 1.3.7.1 rsync

If your content is primarily static Web pages (contact information, for

example) or a reasonably small FTP site, you can store all the data locally on each of the actual servers. To keep the data synchronized, you can simply use a mirroring tool such as rsync that runs periodically; for example, twice an hour. Availability is good with this solution because all of the data is stored on each server individually. It does not matter if one server goes down for some reason, and you do not need to rely on a central storage server. Figure 12 shows an example of this solution:

Figure 12. Using rsync for local server data synchronization

This solution is not suitable if you have large amounts of data (more than a few gigabytes) changing more often (more than a few times in a week) and you must keep it synchronized. In that case, network or distributed file systems may provide a better solution.

1.3.7.2 Network File System

You can use Network File System (NFS), a widely used, commonly known, stable network file system. It is easy to use and requires a central NFS server that exports the shared data. This data is “mounted” across the network by the real servers. This approach looks like the one shown in Figure 13 on page 22.

real server 1

real server 2

real server 3

local data local data local data

Receive updates over the network from real server 1 twice an hour by running, for example, rsync using cron

rsync

Figure 13. Using NFS for central data storing

Although NFS is simple to implement and use (on Linux), it has two major drawbacks:

• Slow performance

• Single point of failure

Although the performance may be acceptable for a small cluster solution, you should always be aware that if the NFS server goes down, the real servers are no longer be able to access your data and cannot provide their service.

This might make you think about setting up a redundant, highly available NFS server, which is not easy to attempt. You must take care of the clients’ file handles and keep the clustered NFS servers synchronized. There is no “out of the box” solution here. Therefore, NFS is not a real solution for a cluster environment.

That is why there is a need for a different approach to the data in a cluster file system using cluster-capable file systems such as the Global File System (GFS) and Intermezzo.

1.3.7.3 Global File System

GFS implements the sharing of storage devices over a network. This includes shared SCSI, Fibre Channel (FC), and Network Block Device (NBD). The Global File system sitting on top of these storage devices appears as a local file system for each box (see Figure 14).

real server 1 (data mounted

via NFS)

local data (exported via

NFS)

NFS server

real server 3 (data mounted

via NFS) real server 2

(data mounted via NFS)

Figure 14. GFS

The Global File System is a 64-bit, shared disk file system focusing on:

• Availability: If one of the clients goes offline, the data can still be accessed by all the other GFS clients.

• Scalability: It doesn’t suffer from concepts based on a central file server, as NFS does.

Furthermore, GFS can pool separate storage devices into one large volume, and perform load balancing between the workload generated by all the clients.

To set up GFS, you need to decide on the transport medium to use:

• Shared SCSI: A pool of disks physically attached to more than one computer, which gives you a low cost, but not very scalable, solution using already deployed equipment.

• Fibre Channel: Provides you with a storage area network, that the individual devices are attached to, and is a more expensive, large scale setup.

• IP:Similar to using tunneling to attach to your client over a traditional network, but not yet a widely used option. IP is limited by the network bandwidth, but it allows you to attach any client without direct FC or SCSI connection to your storage pool.

GFS client

GFS client

GFS client

shared storage devices

Shared SCSI or Fibre Channel or IP

GFS implements the storage pooling, the file locking, and the real file system.

It is still under development, but should be usable.

1.3.7.4 Intermezzo

Opposite to GFS, which is a shared file system, Intermezzo is an implementation of a distributed file system. This means that there is no central storage pool, but each machine has its own kind of local storage.

Their storage is synchronized using traditional TCP/IP networks. Intermezzo still features a client-server model. The server holds the authoritative data, while the clients only have a locally cached version of the data, which is kept synchronized. Intermezzo supports disconnected operation and is able to reintegrate when connected again. Figure 15 shows a simple Intermezzo configuration.

Figure 15. Sample Intermezzo setup

Intermezzo uses a traditional file system, such as ext2, to hold its data and puts a software layer in between that is responsible for journaling updates and keeping the data synchronized. Please note that Intermezzo, such as GFS, is still under development, but is already usable. It requires a Linux kernel recompilation to implement it today.

1.3.7.5 Backend database

Another option for storing and accessing your data, and one that might already be in place, is a backend database, such as DB/2. This database can be highly available, but that is not part of our Linux clustering solution. Your real servers must be able to connect to this database and store data or extract data using, for example, remote SQL queries inside PHP featuring dynamic Web pages. This is a very convenient and widely used option. An

replication

example of such a configuration is shown in Figure 16. Consider the backend database as the existing enterprise database server running on S/390 or other UNIX platforms, for example.

Figure 16. A cluster of front-end servers in conjunction with a fault-tolerant backend database