Failures During c-treeACE Shutdown

8.3.1

Server Shuts Down Improperly

This section discusses ways in which a c-tree Server shutdown may fail to complete properly. A normal c-tree Server shutdown is indicated by the following messages in the server status log:

Fri Sep 26 14:30:08 2003

- User# 12 Server shutdown initiated Fri Sep 26 14:30:09 2003

- User# 12 Communications terminated Fri Sep 26 14:30:09 2003

- User# 12 Perform system checkpoint Fri Sep 26 14:30:09 2003

- User# 12 Server shutdown completed Fri Sep 26 14:30:09 2003

- User# 12 Maximum memory used was 116088930 bytes

A normal c-tree Server shutdown operation may not complete properly for the following reasons:  The server may terminate abnormally at shutdown due to a fatal exception.

 The server may complete its shutdown without successfully terminating all active client threads.

If the server shutdown terminates abnormally due to a fatal exception, the server does not write the “Server shutdown completed” message to the server status log. Follow the recovery

procedures described in the section “Recovering From Abnormal Server Termination” (page 118). If the server shutdown completes without successfully terminating all active client threads, the server avoids writing a final checkpoint to the transaction logs. This causes the server to perform automatic recovery on its next startup. The server notes this situation at shutdown by logging the following message to CTSTATUS.FCS:

Mon Sep 29 16:15:51 2003

- User# 13 Clients active: skipped system checkpoint. Auto-recovery on next start up

Treat this situation similar to an abnormal server termination, because it is possible that clients were modifying PREIMG and non-transaction files at the end of server shutdown which could mean that some updates did not get written to disk. Before restarting the c-tree Server, open

PREIMG and non-transaction files using a standalone utility to determine if they need to be

rebuilt. The server’s automatic recovery ensures that TRNLOG files are in a consistent transaction state on the next server startup.

8.3.2

Server Shutdown Hangs or Takes Excessive Time

The c-tree Server shutdown process may take a long time for the following reasons:  The server must write all updated data and index cache pages to disk before shutdown

completes. If the server is configured to use a large cache, there can be many updated cache pages that must be written to disk at shutdown, which increases the shutdown time.

 The server processes entries in the delete node queue at shutdown. The presence of many entries in the delete node queue can lead to increased server shutdown time.

 The server allows clients time to recognize that the server is shutting down and to disconnect from the server. The shutdown time can be long if many clients are connected or if the server is not able to terminate connected clients.

Monitoring c-tree Server Shutdown Progress

The progress of the c-tree Server shutdown can be monitored in the following ways:  Monitoring messages the c-tree Server writes to the server status log during shutdown.  Monitoring messages the c-tree Server writes to its console or standard output during shut  Monitoring system resource usage by the c-tree Server during shutdown.

The c-tree Server shutdown is normally accompanied by messages in the server status log such as the following:

- User# 13 Process delete node Q.... - User# 13 Clients still active... - User# 13 Clients shutting down....

These messages indicate the current operation the c-tree Server is performing, such as

processing delete node queue entries, terminating connected clients, and allowing clients time to shut down.

The server also writes shutdown messages to its console window or standard output. These messages provide more detailed information than the status log entries, including the remaining number of delete node queue entries, and the current number of active client threads:

Process delete node Q...<num_queue> entries. Clients still active...<num_active>

Clients shutting down...<num_active>

where <num_queue> is the current number of entries in the delete node queue and <num_active> is the current number of client threads that are still active.

System utilities can also provide insight into the c-tree Server’s progress during shutdown. Monitor disk activity on the c-tree data and index files to determine if the server is taking a long time to shut down because it is flushing data and index cache buffers. Monitor the number of active server threads to determine how many client threads are still active. See the “Monitoring c-tree Server Process State” section of this document for additional ways to monitor the state of the c-tree Server process.

Forcibly Terminating the c-tree Server During Shutdown

If the c-tree Server is taking a long time to shut down or if the server appears to hang during shutdown, the server process can be terminated using system utilities but be aware of the effect on c-tree data and index files. Forcibly terminating the c-tree Server process at shutdown effectively causes an abnormal server termination, which means that unwritten updates for

PREIMG and non-transaction files may be lost and that the server will perform automatic

recovery on its next startup in order to ensure TRNLOG files are in a consistent transaction state. For details on the state of c-tree data and index files in the event of an abnormal server

termination, see the topic “Server Terminates Abnormally” in the “Failures During c-tree Server Operation” section of this document.