ArcGIS
®9
Managing
Arc
SDE
®Copyright © 1999–2005 ESRI All rights reserved.
Printed in the United States of America.
The information contained in this document is the exclusive property of ESRI. This work is protected under United States copyright law and other international copyright treaties and conventions. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, except as expressly permitted in writing by ESRI. All requests should be sent to Attention: Contracts Manager, ESRI, 380 New York Street, Redlands, CA 92373-8100, USA.
The information contained in this document is subject to change without notice.
U.S. GOVERNMENT RESTRICTED/LIMITED RIGHTS
Any software, documentation, and/or data delivered hereunder is subject to the terms of the License Agreement. In no event shall the U.S. Government acquire greater than RESTRICTED/LIMITED RIGHTS. At a minimum, use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in FAR §52.227-14 Alternates I, II, and III (JUN 1987); FAR §52.227-19 (JUN 1987) and/or FAR §12.211/12.212 (Commercial Technical Data/Computer Software); and DFARS §252.227-7015 (NOV 1995) (Technical Data) and/or DFARS §227.7202 (Computer Software), as applicable. Contractor/Manufacturer is ESRI, 380 New York Street, Redlands, CA 92373-8100, USA.
ESRI, SDE, ArcView, MapObjects, the ESRI globe logo, ArcInfo, ArcSDE, ArcCatalog, ArcEditor, ArcMap, ArcGIS, ArcIMS, ArcStorm, Arc Objects, StreetMap, ArcInfo Librarian, GIS by ESRI, and www.esri.com are trademarks, registered trademarks, or service marks of ESRI in the United States, the European Community, or certain other jurisdictions.
Contents
Introducing the ArcSDE application server 1
The ArcSDE application server 2
Properties of an ArcSDE application server 4
Tips on learning about the ArcSDE application server 5
Creating ArcSDE application servers 7
The ArcSDE home directory 8 The DBMS database 9
The ArcSDE DBMS administration account 10 The ArcSDE service on UNIX and Windows 11 Windows installs 12
Accessing an ArcSDE application server through a firewall 14
Configuring ArcSDE application servers 15
The services.sde file 16
Operating system services files 18 The service name on Windows 19 ArcSDE DBMS environment variables 20 ArcSDE system environment variables 21 The dbinit.sde file format 23
Displaying ArcSDE system environment variables 24
Adjusting ArcSDE application server initialization parameters 25 Displaying the ArcSDE initialization parameters 38
Managing ArcSDE application servers 39
Before starting an ArcSDE application server 40
Starting a local ArcSDE application server on Windows 41 Starting a remote ArcSDE application server on Windows 42 Starting a local ArcSDE application server on UNIX 43 The sdemon command output 44
Starting a remote ArcSDE application server on UNIX 45
Pausing, resuming, and shutting down an ArcSDE application server 46
1
2
3
Shutting down an ArcSDE application server during the editing of a multiversioned table 49 Removing ArcSDE sessions (Windows and UNIX) 50
Monitoring ArcSDE application servers 53
Displaying ArcSDE application server status and lock table information 54 How ArcGIS uses ArcSDE locking 56
Displaying ArcSDE application server statistics 57 Displaying ArcSDE user session information 58
Troubleshooting the ArcSDE application server 59
What happens when you start an ArcSDE application server 60
What happens when an ArcSDE client connects to an application server 62 What happens when an ArcSDE client direct connects to the DBMS 64 Common ArcSDE startup problems on UNIX servers 65
Common ArcSDE startup problems on Windows servers 67 Setting the Windows SharedSection 69
The Windows Event Viewer 70 Examining the ArcSDE error logfiles 71 ArcSDE intercept 72
ArcSDE tracing 73
ArcSDE data dictionary 75
ArcSDE table definitions 97
ArcSDE application server command references 113
ArcSDE initialization parameters 125
5
6
A
B
C
D
Glossary 133
IN THIS CHAPTER
Introducing the ArcSDE application server
1
• The ArcSDE application server • Properties of an ArcSDE
applica-tion server
• Tips on learning about the ArcSDE application server
This chapter introduces the key components of the ESRI® ArcSDE® application
server and provides an overview of the underlying configuration. Subsequent chapters discuss the creation, configuration, and management of the ArcSDE application server. The appendices contain descriptions of the ArcSDE home directory, the data dictionary, table definitions, listings of the ArcSDE commands, and the ArcSDE initialization parameters.
An ArcSDE application server conveys spatial data between geographic information system (GIS) applications and a database. The database may be any one of the supported
database management systems (DBMSs) such as Oracle®, SQL
Server™, Informix®, or DB2®. The database could also be a
registered collection of ArcGIS® coverages or shapefiles. The
applications that can connect to and access spatial data from an ArcSDE application server include ArcGIS Desktop, ArcIMS®,
ArcView® 3, ArcInfo™ Workstation, MapObjects®, and ArcSDE
CAD Client, as well as custom-built applications created by either you or an ESRI business partner.
Two-tiered versus three-tiered architecture ArcSDE provides two alternative methods of connecting to the database instance—direct connection to the database instance in a two-tiered architecture or connecting through the ArcSDE application server in a three-tiered architecture. If it is your intention to adopt the two-tiered approach, then you should consult the Appendix titled ‘Making a Direct Connection’ in the
ArcSDE Configuration and Tuning Guide for <DBMS> PDF file, located in the documentation folder on the ArcSDE CD. This guide is primarily concerned with the administration of the ArcSDE application server. However, information concerning such things as the management of server configuration parameters (Chapter 3 and Appendix D) and troubleshooting (Chapter 6) also apply to direct connections.
Individual client connections to the ArcSDE application server. Dedicated gsrvr processes—
communicate with the DBMS on behalf of the client.
The sdemon command allows the administrator to manage and monitor the
ArcSDE service.
ArcSDE service—the giomgr process— manages client connections.
C1 Connection request C3 Connection request
C2 Connection request C4 Connection request
Verifies connection (username/password) Spawns one dedicated "gsrvr" process per
successful connection
ArcSDE Server Process—"giomgr"
Listens for connection requests
C1 gsrvr C4 gsrvr
Client 3 (for example, ArcIMS)
Client 4 (for example, CAD Client) Client 1
(for example, ArcView GIS)
Client 2 (for example, ArcInfo)
Managed by sdemon
The ArcSDE application server, a client/server configuration, has a number of defining properties; these are introduced in the following sections.
The home directory
Each ArcSDE application server has its own home directory,
which is defined by the system variable,SDEHOME. The home
directory contains the ArcSDE binary executable files, dynamic shared libraries, configuration files, and internationalization code pages.
The ArcSDE application server monitor—the giomgr process
Each ArcSDE application server has one giomgr process. This process listens for user application connection requests, spawns gsrvr processes, and cleans up disconnected user processes. The giomgr will not start if a valid server license has not been installed. Contact ESRI customer support if you have not already obtained a valid license.
The gsrvr process
The giomgr process spawns a gsrvr process for each application connected to the ArcSDE application server. The gsrvr process is dedicated to a single-user application connection. It
communicates with the database on behalf of the connected application. The gsrvr process responds to the application’s query and edit requests to the database.
The Transmission Control Protocol/Internet Protocol service name and port number
The ArcSDE application server, through the giomgr process, listens for application connection requests on a dedicated TCP/IP service name and port number. The gsrvr process, in turn, communicates with the application on the same TCP/IP service and port number. All communication between the applications and the ArcSDE application server take place using the TCP/IP service name and port number.
The configuration parameters
The ArcSDE application server can be configured to optimize the data flow between the connected applications and the database. The configuration parameters are stored in the
SDE.SERVER_CONFIG metadata table, making them accessible to multiple application servers and direct connections. The
parameters are managed by editing the configuration files and loading the values into the SDE.SERVER_CONFIG table. The default configuration file, giomgr.defs, is located in the etc folder of SDEHOME.
Tips on learning about the ArcSDE application server
About this book
This book is designed to help you create and maintain an ArcSDE application server. It discusses the ArcSDE application server— how to configure the application server, as well as how to monitor its usage. In addition to showing how to configure and manage your ArcSDE application, it also shows you how to monitor your service and cope with problems that may arise.
The focus of this book is not the creation or administration of a DBMS. This is a separate topic; for information on that topic,
see the ArcSDE Configuration and Tuning Guide for
<DBMS> PDF file, ArcSDE_Config_Gd_<DBMS>.pdf, located in the documentation folder on the ArcSDE CD.
Contacting ESRI
If you need to contact ESRI for technical support, refer to ‘ContactingTechnical Support’ in the ‘Getting more help’ section of the ArcGIS® DesktopHelp system.
You can also visit ESRI on the Web at www.esri.comfor more information on ArcSDE and ArcInfo.
ESRI education solutions
ESRI provides educational opportunities related to geographic information science, GIS applications, and technology. You can choose among instructor-led courses, Web-based courses, and self-study workbooks to find education solutions that fit your learning style. For more information, go to www.esri.com/ education.
IN THIS CHAPTER
2
• The ArcSDE home directory• The DBMS database
• The ArcSDE DBMS administration account
• The ArcSDE service on UNIX and Windows
• Windows installs
• Accessing an ArcSDE application server through a firewall
Creating ArcSDE application servers
Before you can create an ArcSDE application server, you must install the ArcSDE software. For ArcSDE installation instructions, refer to the ArcSDE Installation Guide for your DBMS.
When you install ArcSDE, the ArcSDE software is copied into a directory called the home directory, or SDEHOME.
The DBMS must be prepared by running the sdesetup utility for your DBMS. This utility creates the required ArcSDE and Geodatabase metadata tables under the SDE user’s schema.
Before you start an ArcSDE application server, check the following: • The ArcSDE software has been installed correctly, and the default
home directory (SDEHOME) has been established. On Windows®, any
installation errors will be reported to the <SDEHOME>\SDE90ORA.LOG file.
• A supported DBMS database has been configured for ArcSDE. • An ArcSDE user account exists in the DBMS, with enough free
space available to store the ArcSDE repository.
The ArcSDE home directory, also known by the system variable that stores its location (either %SDEHOME% on Windows or
$SDEHOME on UNIX®), contains all of the executable files and
dynamic libraries required to run an ArcSDE application server. This directory structure is automatically set up during the installation process. Each independent ArcSDE application server requires its own home directory.
The ArcSDE application server’s configuration files must then be updated to reflect the unique properties of the new application server. Many ArcSDE application servers can run on a single computer, with each application server requiring its own home directory, TCP/IP service name, and port number. The hardware platform must have sufficient resources available to support the processes of the ArcSDE software and the DBMS.
Refer to the ArcSDE Installation Guide for more information about creating additional ArcSDE application servers.
ArcSDE stores data in a DBMS database. This database must exist before the ArcSDE application server can be created. Consult your DBMS documentation for guidance on creating a DBMS database. You should also review the basic advice
provided in the ArcSDE Configuration and Tuning Guide for
<DBMS>, which is located in the documents directory of the CD–ROM installation media and in the ArcSDE home directory (SDEHOME).
Microsoft® SQL Server, IBM® Informix, and IBM DB2 supported
servers may contain one or more databases, although be aware that DBMS vendors vary in their definition of a database. An ArcSDE application server can only connect to one database server and to one database.
Configuration parameters that control the manner in which an ArcSDE application server connects to a database may be stored in the dbinit.sde file. This file is located in the %SDEHOME%\etc directory on Windows and in the $SDEHOME/etc directory on UNIX. This avoids the need to rely on environment variables set when the user logs in. The dbinit.sde file is discussed in detail in Chapter 3, ‘Configuring ArcSDE application servers’.
For optimum performance,the database and the ArcSDE
application server should coexist on the same host. However, it is possible for ArcSDE to connect to a database hosted on a remote machine. For more information on establishing a remote
connection to an ArcSDE service, refer to the ArcSDE Configuration and Tuning Guide for <DBMS>.
An ArcSDE application server requires the creation of an ArcSDE DBMS administration account. On Windows, the ArcSDE installation process gave you the opportunity to create an ArcSDE DBMS administration account and the necessary DBMS storage space to store the data dictionary tables owned by the ArcSDE administrator account. If this step was bypassed during the installation, it must be completed before the ArcSDE application server can be started up.
The procedure for creating the ArcSDE DBMS administration account differs for each ArcSDE product. For information on how to create the ArcSDE DBMS administration account, refer to
ArcSDE Installation Guide.
Under the ArcSDE DBMS administration account, ArcSDE stores its data dictionary tables. The tables are created and initially populated by the ArcSDE sdesetup command line utility. There is a separate utility for each DBMS.
Registering an ArcSDE service on UNIX
1. Edit the $SDEHOME/etc/ services.sde file.
2. Add the same service name and port number to the operating system /etc/services file.
The ArcSDE
service on UNIX
and Windows
Each ArcSDE application server listens for user connections on a dedicated TCP/IP service name and port number through the giomgr process.On UNIX systems, edit the $SDEHOME/etc/services.sde file and the /etc/services file to include the service name. On Windows systems, the ArcSDE application server is created during the ArcSDE software installation as a Windows service. The service name is stored in the registry under HKEY_LOCAL_MACHINE\ SOFTWARE\ESRI\ArcInfo\ ArcSDE\ArcSDE for <product>\<service name>.
1
2
## ESRI ArcSDE service name and port number #
esri_sde 5151/tcp
. . .
nfsd 2049/udp # NFS server daemon (clts) nfsd 2049/tcp # NFS server daemon (cots) lockd 4045/udp # NFS lock daemon/manager lockd 4045/tcp
esri_sde 5151/tcp # ArcSDE 9 service dtspc 6112/tcp # CDE subprocess control fs 7100/tcp # Font server
. .
$SDEHOME/etc/services.sde file
Operating system /etc/services file
Tip
Port number
Although it is not used at this stage, the port number in the SDEHOME/etc/services.sde file is included to remind the ArcSDE administrator that the service name is assigned to the 5151/TCP port in the operating system services file.
Windows installs
The installation program automatically enters the service name and port number for the default service name in both the registry and the Windows services file,C:\winnt\system32\drivers\ etc\services.
The installation program also creates the Windows service, setting it to automatic—that is, it will automatically start up when the server host is rebooted. Like all Windows services, the ArcSDE application server is started and stopped from the Windows services menu. Although it is more convenient to start the ArcSDE application server from the Windows service menu, it is sometimes necessary to use the sdemon command because error messages are printed directly to the MS–DOS command window instead of the event log. Using sdemon to start the service can be useful when trying to diagnose a startup problem. Unlike the Windows service, the sdemon command reads the service name from the
%SDEHOME%\etc\services.sde file. Edit this file to ensure that it contains the correct informa-tion before using sdemon.
Verifying an ArcSDE service on Windows 1. Open the Control Panel and
double-click Administrative Tools.
2. From the Administrative Tools menu, click Services. 3. From Services, make sure
the ArcSDE service has been started.
1
2
Monitoring the ArcSDE service
1. Use the sdemon command at the Windows MS–DOS command prompt to trap ArcSDE application server error messages.
1
Tip
Adding more ArcSDE services
If additional ArcSDE application servers are required, use the sdeservice command to define the new Windows service. For a complete description of the sdeservice command, see ‘Appendix C: ArcSDE application server command references’.
Accessing an ArcSDE application server through a firewall
To provide access to an ArcSDE application server inside a system security firewall, the host computer on which the ArcSDE application server is installed should be listed in your domain name server (DNS) database. The DNS must be registered with your Internet service provider (ISP) or directly with Network Solutions (formerly called InterNIC), the organization that registers Internet domain names.
Your DNS resolves the IP address of your computer to the name, or universal resource locator, you wish to make accessible to the Internet. In most cases, you will have more machines within your local network than you will have Internet IP addresses for. In this case, you would maintain your own set of internal IP addresses known only to your local area network (LAN). Your firewall, or proxy server software, will translate your internal IP addresses to Internet IP addresses when you access computers outside your LAN.
Since ArcSDE application servers listen for connections on a TCP/IP port number that corresponds to your service name, you must also add the TCP/IP port number to the computer’s host name when connecting to it.
For ArcSDE applications built with the ArcSDE C application programming interface (API), the host must always be specified using host name.
For ArcSDE Java applications you can specify an ArcSDE host name in two ways. You can either use the DNS name, if it is available, or you can connect to it directly using its Internet IP address.
For example, ESRI’s domain name—esri.com—has been
registered with Network Solutions, and we identified our DNS as IP address 198.102.62.1. Our DNS has the IP address for the
ArcSDE host Toshi in its DNS database. The internal IP address for Toshi is 46.1.2.324, which is translated to the IP address 198.102.62.5 when Toshi sends and receives information through the firewall. The ArcSDE application server running on Toshi is listening for connections on the service name esri_sde3, which corresponds to TCP/IP port number 5165. So, if you wish to connect to that particular ArcSDE application server, you must specify either the host name “toshi.esri.com:5165” or identify Toshi by its IP address “198.102.62.5:5165”. In both cases you must also include the service port number, 5165.
The giomgr process bequeaths the port number to the gsrvr process following a successful connection. Therefore, all communication to the ArcSDE application server occurs on the same TCP/IP port number.
If you cannot connect to an ArcSDE application server through a firewall, test the accessibility of the remote ArcSDE host with your Internet browser by specifying either the server name and TCP/IP port number or the IP address and TCP/IP port number as the URL.
The correct syntax is:
<server name>:<port number> <IP address>:<port number>
IN THIS CHAPTER
Configuring ArcSDE application servers
3
• The services.sde file
• Operating system services files • The service name on Windows • ArcSDE DBMS environment
variables
• ArcSDE system environment variables
• The dbinit.sde file format • Displaying ArcSDE system
environment variables
• Adjusting ArcSDE application server initialization parameters • Displaying the ArcSDE
initializa-tion parameters
ArcSDE conveys spatial data between a DBMS and applications. Configuring an ArcSDE application server focuses on maximizing data transfer between the server and the client with limited shared resources. The requirements of the application, the number of users, and the amount of data requested influence the configuration of the ArcSDE service.
The ArcSDE application server can be configured by modifying the
parameters in the configuration metadata tables stored in the ArcSDE user’s schema or the configuration files services.sde and dbinit.sde, located in the SDEHOME/etc directory.
The services.sde file
Each ArcSDE application server communicates with the database management system and the applications through a TCP/IP port. TCP/IP ports are defined by name in the operating system’s services file.
The SDEHOME\etc\services.sde file contains the service name and the unique TCP/IP port number on which the ArcSDE application server accepts connection requests. This port number is also assigned to each user or gsrvr process that the ArcSDE application server initiates. The port number listed in the
services.sde file does not designate operating system port use. It is included in the services.sde by convention as a reminder of the port number assigned to the service name in the operating system’s services file.
ESRI has registered the default esri_sde service name and 5151 TCP/IP port number with the Information Sciences Institute, Internet Assigned Numbers Authority.
The default services.sde file created during the installation process will contain the following:
#
# ESRI ArcSDE Remote Protocol #
esri_sde 5151/tcp
To change the default service name, simply edit the file and restart the service.
On UNIX systems the sdeservice.sde file is always used. On Windows systems, however, the services.sde file is used only when the service is started from the MS–DOS prompt using the sdemon command. If the service was started from the Windows service panel, ArcSDE uses the service name stored in the registry under
HKEY_LOCAL_MACHINE\HARDWARE\ESRI\ArcInfo\ArcSDE\ArcSDE for <dbms>\esri_sde.
When a match is found, ArcSDE starts the giomgr process. It listens for user connection requests on the TCP/IP port number assigned to the service name.
When the ArcSDE application server is started with the sdemon command, the service searches the system services file for a service name that matches the service name in the services.sde file.
If a match is not found, ArcSDE returns the following error on UNIX systems:
$ sdemon -o startsdemon -o startsdemon -o startsdemon -o startsdemon -o start
Please enter the SDE DBA password: *************************
This instance’s service name esri_sde not found in system services file.
To diagnose startup problems on a Windows platform, examine the event log. For more information, see Chapter 6,
‘Troubleshooting the ArcSDE application server’.
Mapping the SDEHOME/etc/services.sde entries to the system services file entries
Matches the service name in the operating system's services file with the service namefrom the $SDEHOME/etc/services.sde file and uses the corresponding port number for ALL
communication.
for example, 'esri_sde'
5151/tcp sdemon command
for example, 'esri_sde' 5151/tcp #ArcSDE service Reads the service name from $SDEHOME/etc/services.sde.
The services.sde file in each SDEHOME\etc directory should contain only one service name. If you add more than one service name to this file, ArcSDE will use the first one it encounters and ignores the rest.
The operating system services file contains many service names. At least one of them must match your service name, found in the services.sde file. Enter the service name into the operating system services file on the server and client machines.
Here is an example from an operating system services file that contains two ArcSDE services:
esri_sde 5151/tcp # ArcSDE default service esri_sde2 5161/tcp # another ArcSDE service
This defines two service names: esri_sde and esri_sde2. On UNIX systems, you can use the Network Information Service (NIS) services file if you are running NIS to avoid unnecessary duplication of effort when updating the client and server local system services files.
The dbinit.sde file of the new service must also be edited so it contains the connection information of the second DBMS. For more information on setting the DBMS connection variables in the dbinit.sde file, see ‘ArcSDE DBMS environment variables’ in this chapter.
One host computer supporting two ArcSDE application servers
Multiple ArcSDE application servers running on the same host require unique service names and port numbers. If you intend to create another ArcSDE application server on the same host, you must edit one of the services.sde files and amend the service name and port number accordingly. You must also update the system services file to include the new ArcSDE application server. Host Machine /etc/services file DBMS 1 DBMS 2 gsrvr processes giomgr processes
esri_sde 5151/tcp # ArcSDE default service giomgr port esri_sde2 5161/tcp # ArcSDE service # 2 giomgr port
ArcSDE Service 1 "esri_sde" port number 5151/tcp ArcSDE Service 2 "esri_sde2" port number 5161/tcp
Forcing a search of local services on HP–UX®
1. Copy the nsswitch.conf file from the /usr/newconfig/etc directory to the /etc directory. 2. Edit the file and change the
line ‘services: nis files’ to ‘services: files nis’.
Operating system
services files
The services file on aWindows platform is located under the winnt\system32\drivers\etc directory. Use an editor, such as Notepad, that does not embed format characters to edit the Windows services file.
The services file for UNIX systems is located at /etc/services. Some UNIX systems direct applications to search the NIS services file rather than the local services file. If you wish to have the operating system search the local services file instead, you must force it to do so.
Forcing a search of the local services on IBM AIX®
1. In the /etc directory, create the file netsvc.conf.
2. Add the line “services=local.nis”.
# /etc/nsswitch.conf:
#This file uses NIS (YP) in conjunction with # files.
# "hosts:" and "services:" in this file are used # only if the /etc/netconfig file has a "-" for # nametoaddr_libs of "inet" transports.
# the following two lines obviate the "+" entry # in /etc/passwd and /etc/group.
passwd: files nis group: files nis
# consult /etc “files” only if nis is down. hosts: files nis dns
networks: nis [NOTFOUND=return] files protocols: nis [NOTFOUND=return] files rpc: nis [NOTFOUND=return] files ethers: nis [NOTFOUND=return] files netmasks: nis [NOTFOUND=return] files bootparams: nis [NOTFOUND=return] files publickey: nis [NOTFOUND=return] files netgroup: nis
automount: files nis aliases: files nis
# for efficient getservbyname() avoid nis services: files nis
The service name
on Windows
On Windows systems, the services.sde file is only accessed when the ArcSDE application server is started with the sdemon command. When the service is started from the services menu, the service name is read from the system registry.
To change the service name in the Windows registry, use the sdeservice administration com-mand.
1. Shut down the ArcSDE application server from the Windows NT Services dialog box by clicking Stop.
2. From a DOS command prompt, remove the old service name using the sdeservice command with the delete option.
3. From a DOS command prompt, create the new service name with the sdeservice -o create option. 4. Restart the ArcSDE applica-tion server from the Services dialog box. Notice the service name has changed.
1
4
3
2
See Also
For more information about the sdeservice command, see ‘Appendix C: ArcSDE application server command references’.
The dbinit.sde file and the application server The ArcSDE application server reads system environment variables from the SDEHOME\etc\dbinit.sde file for such things as establishing the DBMS server to connect to and limiting the messages written to the sde error log (sde_<service_name>.log) file. System environment variables set within the dbinit.sde file override those set within the system environment of the user that starts the application server. For variables absent from both the dbinit.sde file and the user environment, default ArcSDE settings are used.
The system environment variables that control the ArcSDE application server’s connection to a DBMS depend on each individual DBMS.
For more details on setting environment variables, refer to the
ArcSDE Installation Guide for < DBMS>.
ArcSDE system environment variables
By default, SDEHOME determines on which ArcSDE application server the sdemon command will operate. This variable can be overridden by the -H option of the sdemon command.
setenv SDEHOME d:\arcexe\arcsde setenv SDEHOME d:\arcexe\arcsde setenv SDEHOME d:\arcexe\arcsde setenv SDEHOME d:\arcexe\arcsde setenv SDEHOME d:\arcexe\arcsde
SDESERVER determines the host of the ArcSDE application server for the connecting client application. This variable can be overridden by specifying the host in the application. If the host is not specified during connection and the SDESERVER variable is not set, the client application attempts to connect to an ArcSDE application server running on the local host.
setenv SDESERVER bruno setenv SDESERVER bruno setenv SDESERVER bruno setenv SDESERVER bruno setenv SDESERVER bruno
SDEINSTANCE is set in the environment of the client application and determines the ArcSDE service name to connect to.
Specifying the service in the application overrides this variable. If this variable is not set and the service name is not specified in the application, the service name defaults to esri_sde.
setenv SDEINSTANCE esri_sde setenv SDEINSTANCE esri_sde setenv SDEINSTANCE esri_sde setenv SDEINSTANCE esri_sde setenv SDEINSTANCE esri_sde
SDEDATABASE can be entered for the DBMS and has multiple database connection possibilities within a single application server. Specifying the database through the application overrides this variable. If the variable is not set and the database is not specified upon connection, the ArcSDE client connects to the default database.
setenv SDEDATABASE city_eng setenv SDEDATABASE city_eng setenv SDEDATABASE city_eng setenv SDEDATABASE city_eng setenv SDEDATABASE city_eng
SDEUSER specifies the username through which the ArcSDE client application will connect. Specifying the username in the application overrides this variable. If this variable is not set and the username is not specified in the application, an error is returned. A username must be specified.
setenv SDEUSER bob setenv SDEUSER bob setenv SDEUSER bob setenv SDEUSER bob setenv SDEUSER bob
SDEPASSWORD specifies the password for the username entered by the ArcSDE client application. Specifying the
password on the application connection tool overrides this variable. If the variable is not set and the password is not specified in the application, the application may prompt for the password. If the application does not prompt for the password, an error is returned.
setenv SDEPASSWORD fools.gold setenv SDEPASSWORD fools.gold setenv SDEPASSWORD fools.gold setenv SDEPASSWORD fools.gold setenv SDEPASSWORD fools.gold
SDETMP allows you to set the temp directory for the servers using this variable, but it will only be checked if the TEMP keyword is not set in the SDE.SERVER_CONFIG table.
setenv SDETMP c:\temp setenv SDETMP c:\temp setenv SDETMP c:\temp setenv SDETMP c:\temp setenv SDETMP c:\temp
SDEDBECHO echoes the contents of the dbinit.sde file during startup. For ArcSDE application servers started locally on a UNIX system, the output of SDEDBECHO is written to the screen. The SDEDBECHO output for an ArcSDE application server started on a remote UNIX ArcSDE application server is written to its sde.errlog file.
setenv SDEDBECHO TRUE setenv SDEDBECHO TRUE setenv SDEDBECHO TRUE setenv SDEDBECHO TRUE setenv SDEDBECHO TRUE
SDEVERBOSE reports internal messaging to the screen upon startup and writes gsrvr process startup and shutdown messages to sde.errlog.
setenv SDEVERBOSE TRUE setenv SDEVERBOSE TRUE setenv SDEVERBOSE TRUE setenv SDEVERBOSE TRUE setenv SDEVERBOSE TRUE
Set SDEATTEMPTS to the number of times you want the session to attempt to connect to an ArcSDE application server. Under normal conditions, only one attempt is required; however, should the ArcSDE application server become too busy satisfying the connection requests of many users at the same time, several attempts may be required before a session is able to establish a connection. Each time a session fails to establish a connection, it waits until the IP time out occurs. The IP time out is set by your network administrator but defaults to 75 seconds on most operating systems. By default, SDEATTEMPTS is set to 4. This should be adequate for most environments.
setenv SDEATTEMPTS 4 setenv SDEATTEMPTS 4 setenv SDEATTEMPTS 4 setenv SDEATTEMPTS 4 setenv SDEATTEMPTS 4
Set SDENOIPTEST to true if you do not want ArcSDE to test for the presence of the SERVER name in the HOSTS file. By default SDENOIPTEST is not set. Setting this variable can be useful if you have not set up the HOSTS file. The client will attempt to connect to the server for SDEATTEMPTS (4 times by default).
setenvSDENOIPTEST TRUE setenvSDENOIPTEST TRUEsetenvSDENOIPTEST TRUE setenvSDENOIPTEST TRUEsetenvSDENOIPTEST TRUE
Set GIOMGRLOGREFRESH to true if you wish to have the giomgr.log file overwritten each time the ArcSDE server is started. If you wish to accumulate the log messages (the default), do not set the variable.
setenv GIOMGRLOGREFRESH TRUE setenv GIOMGRLOGREFRESH TRUEsetenv GIOMGRLOGREFRESH TRUE setenv GIOMGRLOGREFRESH TRUEsetenv GIOMGRLOGREFRESH TRUE
Set SDELOGAPPEND to true if you want the sde error logfile to accumulate log messages and be truncated each time the server is restarted. Do not set the variable if you want the sde error logfile to be truncated upon startup.
setenv SDELOGAPPEND true setenv SDELOGAPPEND truesetenv SDELOGAPPEND true setenv SDELOGAPPEND truesetenv SDELOGAPPEND true
ESRI_US_STREETS_DIR specifies the folder that stores the
ArcGIS StreetMap™ USA reference data used by the ArcSDE
StreetMap USA locators. Set this environment variable to the folder containing the usa.edg file and other .edg StreetMap USA street data files. Note that the value of this environment variable should not contain a trailing slash (“/”) or backslash (“\”).
setenv ESRI_US_STREETS_DIR setenv ESRI_US_STREETS_DIRsetenv ESRI_US_STREETS_DIR setenv ESRI_US_STREETS_DIRsetenv ESRI_US_STREETS_DIR
d:\streetmap usa\usa\streets d:\streetmap usa\usa\streets d:\streetmap usa\usa\streets d:\streetmap usa\usa\streets d:\streetmap usa\usa\streets
LOCATION_ERRLOG defines the file to which ArcSDE location errors are logged. The value of this environment variable should contain the fully qualified name of the logfile. This file does not need to exist before setting the environment variable because the location framework creates it automatically. ArcSDE location errors will be logged to this file whenever the
LOCATION_VERBOSE environment variable is set to TRUE.
setenv LOCATION_ERRLOG setenv LOCATION_ERRLOGsetenv LOCATION_ERRLOG setenv LOCATION_ERRLOGsetenv LOCATION_ERRLOG
c:\temp\location.errlog c:\temp\location.errlog c:\temp\location.errlog c:\temp\location.errlog c:\temp\location.errlog
Set the LOCATION_VERBOSE environment variable to TRUE to log ArcSDE location errors. If the LOCATION_ERRLOG
environment variable is also set, location errors are logged to the specified file. If the LOCATION_ERRLOG environment variable is not set, or the specified file cannot be created, location errors are logged to SDEHOME\etc\location.errlog.
setenv LOCATION_VERBOSE TRUE setenv LOCATION_VERBOSE TRUEsetenv LOCATION_VERBOSE TRUE setenv LOCATION_VERBOSE TRUEsetenv LOCATION_VERBOSE TRUE
The dbinit.sde file format
The dbinit.sde file consists of comments and commands. Comments are any lines preceded by the pound sign (#). For example:
# This is the system environment for # the esri_sde ArcSDE application server
The commands in the dbinit.sde file accept two keywords: set and
unset. The set command enables the system variable and assigns it the value following the equal sign. The syntax of the set command is:
set <variable>=<value>
In this example, the SDEDBECHO variable is set to true, which echoes the variables set in the dbinit.sde file when the ArcSDE application server is started.
set SDEDBECHO=TRUE
The unset command disables the system variable. It is useful because it ensures that an undesired variable set in the login environment is not set when ArcSDE starts. The syntax of the unset command is:
unset <variable>
The following example of the unset command ensures that the Oracle TWO_TASK variable is not set:
Displaying ArcSDE system environment variables
To display the environment variables for an ArcSDE application server that is currently running, use:
$ sdemon -o info -I varssdemon -o info -I varssdemon -o info -I varssdemon -o info -I varssdemon -o info -I vars
A list of environment variables is returned in the format:
<variable>=<value>
The variables in the list are a combination of those found in the login system environment file and the dbinit.sde. The variables set in the dbinit.sde file always override the system environment file settings.
On Windows systems the application server is started as a Windows service; therefore, it may not have the same
environment setting as the logged in user’s environment. Thus, there will probably not be agreement with the MS–DOS set command for the logged in user.
The ArcSDE application server initialization parameters stored in the SDE.SERVER_CONFIG table control the configuration of the application server. The parameters are read when the ArcSDE application server starts.
The default values satisfy the needs of most ArcSDE installations. With the exception of adjusting the
MINBUFFSIZE and MAXBUFFSIZE parameters to improve data loading performance, the remainder of the parameters should not be altered unless you are certain you need to do so. ArcSDE applications making direct connects to a DBMS also read the parameters from the SDE.SERVER_CONFIG table. Not all initialization parameters that apply to the ArcSDE application server apply to ArcSDE applications making a direct connection. See ‘Appendix D: ArcSDE initialization parameters’ for a full list of initialization parameters. The following sections discuss in more detail some of the individual parameters and their impact on the ArcSDE application server.
READONLY parameter
The READONLY parameter allows you to start the ArcSDE application server in READONLY mode. Set this parameter to TRUE if you want to disable all writes by ArcSDE clients. When the READONLY parameter is set to FALSE, the default value, the ArcSDE application will allow users to edit the data of the ArcSDE feature classes and tables that they have write permissions for.
Session parameters
The session parameters are CONNECTIONS, TEMP, and TCPKEEPALIVE.
The CONNECTIONS parameter restricts the number of concurrent connections allowed by an ArcSDE application server.
Adjusting ArcSDE application server initialization parameters
The CONNECTIONS parameter does not restrict the number of direct connections. The default is 64 on UNIX and 48 on Windows.
The TEMP parameter specifies the full path to the directory that the ArcSDE application server uses for temporary things, such as the shared memory file on UNIX systems and temporary storage for the attribute binary large objects (BLOBs), that are larger than specified by the BLOBMEM parameter. For more information on this parameter, please refer to the section ‘Managing BLOB data’ later in this chapter. A directory with at least 5 MB of available space should be specified. More space may be required for ArcSDE application servers that support applications allowing users to concurrently access binary large objects stored in attribute BLOB columns.
Note that some files created by the ArcSDE application server are always stored in the /tmp directory. For instance, the
s.sde<service>.iomgr file is a UNIX protocol socket used for connections to the ArcSDE application server. It must always be created in a known location so that local clients, unaware of the SDEHOME location and the setting of TEMP, can find it. Thus it is always created in /tmp.
Sometimes the computer on which an application is running crashes or a user unexpectedly terminates an application. Unless the TCPKEEPALIVE parameter is set to TRUE, the ArcSDE application server process does not detect the absence of the client process and does not disconnect from the DBMS server. When this happens, the ArcSDE application server process remains, and the administrator must manually terminate it. The TCP/IP KEEPALIVE interval is a systemwide parameter that affects all application server processes running in the TCP/IP environment.
Client/Server transport buffers . ArcSDE Client (for example, ArcMap) Connection/Query Client Transport Buffer Result Set ArcSDE Server ServerTransport Buffer By setting the ArcSDE TCPKEEPALIVE parameter to TRUE, the
system TCP/IP KEEPALIVE settings are used. The default test interval is two hours of idle time—that is, the system checks the connected sessions every two hours.
When this happens, any ArcSDE connections whose client processes have been terminated are disconnected. If the network environment in which the ArcSDE application server operates is reliable, TCPKEEPALIVE may be set to TRUE. However, be aware that a disconnection may be triggered by short-term network outages (~10 minutes) when TCPKEEPALIVE is set to TRUE. By default, TCPKEEPALIVE is set to FALSE.
The TCPKEEPALIVE parameter does not affect direct connections.
Transport buffer parameters
After a user connects to an ArcSDE application server from an application, such as ArcMap™ or ArcCatalog™, the feature classes and tables stored in the database can be accessed. Each time any of these items are accessed, an ArcSDE stream is created. An ArcSDE stream is a mechanism that transports data between the database the ArcSDE application server is currently connected to and an application such as ArcMap. For further information on streams and the parameters available to control their operation, see ‘Streams’ later in this chapter.
When an ArcSDE stream is created, the ArcSDE application server process allocates transport buffers on both the client and the server. Transport buffers reduce I/O and improve performance by accumulating records and sending them across the network in batches rather than as individual records.
The records are collected in the ArcSDE application server process’ transport buffer and sent to the ArcSDE client transport buffer when the application is querying the database.
Alternatively, the records are collected in the client’s transport
buffer and sent to the ArcSDE application server process transport buffer when the application is writing data to the database.
Three parameters control the transport buffers.
MAXBUFSIZE The total amount of memory allocated to each transport buffer
MINBUFSIZE The minimum threshold of each transport buffer
MINBUFOBJECTS The minimum number of records per transport buffer
The MAXBUFSIZE represents the total amount of memory that is allocated to each transport buffer. The transport buffer stops accumulating records once MAXBUFSIZE is reached and waits for the request to send the records to the other buffer.
The MINBUFSIZE and MINBUFOBJECTS parameters are lower thresholds that prevent the records from being sent until one of them is attained. For example, if the application requests data, the request is deferred until the server transport buffer reaches either MINBUFSIZE or MINBUFOBJECTS.
The data transfer process from an ArcSDE application server transfer buffer to an ArcSDE client
When creating a geodatabase, you should raise the parameters to increase the transmission speed of data loading. Once loading is complete, reduce the transport buffers.
Before raising the MAXBUFSIZE parameter too high, consider the maximum overall impact to the server’s memory budget. The default MAXBUFSIZE adds 64 kilobytes per stream per
ArcSDE user connection. For example, if there are 100 users using an application that displays seven feature classes, ArcSDE allocates a total of 44,800 kilobytes (100 * 7 * 64) of memory for the server’s transport buffers. Doubling the MAXBUFSIZE in this example would result in 89,600 kilobytes of memory allocated to the server’s transport buffers. Excessive paging may result on the server if MAXBUFSIZE is set too high and
physical memory is not available to satisfy it. Set MINBUFSIZE to no more than one-half of the MAXBUFSIZE.
Setting MINBUFSIZE too high increases wait time. If
MAXBUFSIZE is 64K and MINBUFSIZE is 56K, the client will wait until the 56K threshold is reached before sending the transport buffer.
Reducing the MINBUFSIZE parameter improves the cooperative processing between client and server.
MINBUFOBJECTS depends on the size (bytes) of a row of data. MINBUFOBJECTS is examined first, then MINBUFSIZE. If the threshold of either parameter is breached, the contents of the transport buffer are sent.
ArcSDE application developers can override the giomgr.defs transport buffer parameters with the C API function
SE_connection_set_stream_spec. 1. Transfer buffer empty. MAXBUFSIZE MINBUFSIZE 2. < MINBUFSIZE Client sends query to server. Buffer loading begins no transfer to client.
MAXBUFSIZE
MINBUFSIZE
6. = MAXBUFSIZE If client is not waiting for transfer and buffer is full, buffer loading STOPS.
MAXBUFSIZE
MINBUFSIZE 5. > MINBUFSIZE
Buffer loading continues until client is waiting again.
MAXBUFSIZE
MINBUFSIZE 3. = MINBUFSIZE If client is waiting, data transfer begins.
MAXBUFSIZE
MINBUFSIZE
to Client
4. > MINBUFSIZE If client is not waiting, buffer loading continues.
MAXBUFSIZE
MINBUFSIZE
Array buffer parameters
For each ArcSDE stream that is created, ArcSDE allocates an array buffer.
Whenever possible, groups of records are transferred between ArcSDE and the DBMS server. For DBMSs that have
implemented array inserts, ArcSDE inserts records into the DBMS server array. All supported DBMSs, except Microsoft Access, have implemented array fetch mechanisms, so whenever ArcSDE issues a select statement on behalf of the user, it fetches or gets the records in an array.
Array buffers reduce the amount of I/O occurring between ArcSDE and the DBMS server by fetching and inserting data in larger chunks. Less I/O results in better performance. However, excessive paging can result if the array buffers are set larger than necessary. As a general rule, approximately 100 records is the optimum number that most applications can handle in the array process.
The array buffer parameters determine the size of the array buffers. Parameters that affect query performance are
SHAPEPTSBUFSIZE, ATTRBUFSIZE, MAXARRAYSIZE, and MAXARRAYBYTES.
MAXARRAYSIZE 100 #Max.array fetch size MAXARRAYBYTES 550000 #Max.array bytes
allocated per stream SHAPEPTSBUFSIZE 400000 #Shape POINTS array
buffer size ATTRBUFSIZE 50000 #Attribute array
buffer size
The array buffer parameters define the number of records transferred together for insert and query operations. During an array insert, ArcSDE fills the array buffer with records from the server transport buffer and transfers the entire array to the
DBMS. ArcSDE queries data in an array by using array buffers to receive the data.
Estimating SHAPEPTSBUFSIZE
Array inserts of feature geometry are divided into two parts: feature metadata and point data. Feature metadata, including the feature ID, number of points, and entity type, requires the same amount of space for each feature. The space required to store the point data, on the other hand, varies according to the number of points in the feature.
SHAPEPTSBUFSIZE determines the buffer size for the point data. The default setting is based on an array size of 100. Tuning SHAPEPTSBUFSIZE to the optimal setting is critical to performance. ArcSDE estimates the average size of all features based on the array size (MAXARRAYSIZE) and the size of the points buffer (SHAPEPTSBUFSIZE). If a feature exceeds the average size, it is flagged as truncated and fetched separately.
ArcSDE server array buffers
Connection/Query
ArcSDE data stream>> (returned result set ) << ArcSDE data stream RDBMS ArcSDE Client (for example, ArcMap) Client Transport Buffer ArcSDE Server Server Transport Buffer Array Buffer
For example, suppose SHAPEPTSBUFSIZE is 400,000 (400K), and the array size per fetch (MAXARRAYSIZE) is 100 rows (or 100 features). The features in this example do not have annotation, z-values, or measures, so each point requires eight bytes (four bytes for the x-value and four bytes for the y-value). Dividing SHAPEPTSBUFSIZE by MAXARRAYSIZE returns the maximum space available for each feature’s points within the array buffer. 400000 bytes / 100 = 4000 bytes 400000 bytes / 100 = 4000 bytes 400000 bytes / 100 = 4000 bytes 400000 bytes / 100 = 4000 bytes 400000 bytes / 100 = 4000 bytes
Further dividing the maximum space available to each feature by the number of bytes each point requires returns the total number of points that can be stored in the maximum space available for each feature. 4000 / 16 = 250 4000 / 16 = 250 4000 / 16 = 250 4000 / 16 = 250 4000 / 16 = 250
In this case, ArcSDE expects each feature to have no more than 250 points. When ArcSDE encounters features with 251 or more points, the feature is skipped and fetched separately, forcing an additional I/O fetch from the DBMS.
If the shape contains z-values or measures, divide by 24 instead of 16. If the shape contains both z-values and measures, divide by 32.
Setting the SHAPEPTSBUFSIZE parameter too small results in a higher number of features to be fetched individually, which diminishes performance. Setting this parameter too high wastes memory. Array Buffer 1 2 3 4 98 99 100
=
... ... 250 points per row (feature) 4 KB per row (feature) SHAPEPTSBUFSIZE = 400KMAXARRAYSIZE = 100 ROWS (FEATURES) FEATURE POINT SIZE = 16 bytes
The SHAPEPTSBUFSIZE and MAXARRAYSIZE parameters determine the number of points that can be stored in the maximum space available for each feature within an ArcSDE array buffer.
Determining 90 percent of the most queried feature classes for an optimum
SHAPEPTSBUFSIZE setting
1. Determine the total number of features in the feature class by issuing this SQL query.
2. Determine the largest number of points in the feature class to use as a starting point in the step. 3. Through iteration, determine
the number of points 90 percent of the features have. Start by using an x-value that is 0.75 * maximum numofpts. Increase or decrease the x-value until the feature count is
90 percent of total features. 4. Multiply the number of points
by the number of rows (MAXARRAYSIZE) the array will hold and the number of bytes required by each point. For instance, if
MAXARRAYSIZE is set to 100, 90 percent of the features have 710 or fewer points, and each point requires 8 bytes, set the SHAPEPTSBUFSIZE to 568000.
Queries used under the ArcSDE binary model (SqlServer, Oracle LONG RAW and Oracle LOB). select count(*) “total features” from f<n>; select max(numofpts) “maximum numofpts” from f<n>;
select count(*) 'feature count' from f<n> where numofpts < X;
SHAPEPTSBUFSIZE = (MAXARRAYSIZE) * (number of points) * (point size)
568000 = 100 * 710 * 16
Queries used under the object relational model (DB2, Informix and Oracle Spatial) select count(*) 'total features' from table; select max(numpoints<spatial_column>)) 'maximum numofpts' from <table>; - DB2 and Informix
select max((select count(*)
from table(p.shape.SDO_ORDINATES))/2) * points from <table>;
- Oracle Spatial
select count(*) 'feature count' from table where numpoints(<spatial_column>) < X; SHAPEPTSBUFSIZE = (MAXARRAYSIZE) * (number of points) * (point size)
568000 = 100 * 710 * 8
1
2
3
4
Tip Point sizeThe number of bytes required by each point depends on its coordi-nate type. If the feature class only has x,y coordinates, the number of bytes each point requires is 16. If either z-values or measures are present, 24 bytes are required. If both z-values and measures are present, 32 bytes are required.
1
2
3
4
Estimating ATTRBUFSIZE
ATTRBUFSIZE defines the array buffer size for attribute (non-BLOB) data. Tuning the attribute array buffer is similar to the SHAPEPTSBUFSIZE. The default 50,000 setting allows 100 rows with 500 bytes of attribute data each.
Performance is affected when the number of rows that can be fetched into the attribute buffer does not match the
MAXARRAYSIZE parameter setting. For queries involving multiple columns, add the number of bytes per column to get a total row size. The ATTRBUFSIZE divided by row size cannot exceed the number of rows specified by MAXARRAYSIZE. ArcSDE will automatically reduce the size of the attribute buffer to hold MAXARRAYSIZE rows.
Setting MAXARRAYSIZE
As mentioned, MAXARRAYSIZE sets the number of rows that the server will fetch per request. The recommended default value is 100. Optimal values can range between 20 and 150 on different platforms and DBMSs. Once the shape points data
(SHAPEPTSBUFSIZE) and attribute buffer (ATTRBUFSIZE) are correctly tuned, try several array sizes to determine the optimal setting for each installation.
Setting MAXARRAYBYTES
Control the maximum number of bytes per stream with the MAXARRAYBYTES parameter. This value represents the total bytes that can be allocated to both ATTRBUFSIZE and SHAPEPTSBUFSIZE for each stream.
MAXARRAYBYTES is simply a way to manage the memory allocations for array buffers on the server. The sum of
ATTRBUFSIZE and SHAPEPTSBUFSIZE must be less than or equal to MAXARRAYBYTES. If it isn’t, the ArcSDE application server will not start. If this problem occurs, either increase
MAXARRAYBYTES or decrease either ATTRBUFSIZE or SHAPEPTSBUFSIZE.
This value cannot be changed with the
SE_connection_set_stream_spec function and can only be altered in the sde.server_config table by the ArcSDE administrator.
Managing BLOB data
The BLOB parameters, MAXBLOBSIZE and BLOBMEM, determine the server-side buffer requirements for BLOB data types. MAXBLOBSIZE determines the maximum number of bytes the server will accept. A BLOB is either written to memory or disk, depending on the size. BLOBMEM determines the maximum BLOB size for in-memory storage. That is, the server allocates BLOBMEM bytes to hold the BLOB. A BLOB that exceeds this size is written to disk.
Maximum initial features (Oracle only)
The MAXINITIALFEATS parameter controls the initial features argument that may be submitted to either the sdelayer command or the SE_layer_create function. Based on the value of initial features, sdelayer or SE_layer_create calculates the initial and next extent of the feature and spatial index table. To prevent the possibility of a user inadvertently entering a large initial features value and, thus, acquiring an initial extent beyond what is needed, the administrator can limit the initial features value by setting the MAXINITIALFEATS parameter.
Statistics
The MAXDISTINCT parameter controls the number of distinct values a call to either the SE_table_calculate_stats or the SE_stream_calculate_table_statistics function can return.
Setting this parameter to 0 allows an unlimited number of distinct values to be returned by the applications that call these functions. However, the distinct values are generated in memory on the server and passed to the client’s memory when the list is complete. Calculating the statistics of a large table could pose a threat to the client and server resources. A prudent database administrator will set this value high enough to allow most queries to complete, but not so high as to expose the server or the client application to a memory shortage. Should a user receive the error message SE_TOO_MANY_DISTINCTS, the MAXDISTINCT parameter may be raised, but this should be done cautiously as it impacts both client and server memory. It may be advisable to examine the applications to determine whether queries could be performed more efficiently.
Streams
You have read about streams in the section ‘Transport buffer parameters’ earlier in this chapter. To recap briefly, each time a user accesses a feature class or business table, an ArcSDE stream is created to transfer the information between the client and the server. An ArcSDE stream is a mechanism that transports data between the DBMS database and an application such as ArcMap. The resources allocated to each stream are significant; it is, therefore, important that the ArcSDE administrator understands how they operate and how to control the impact they have on the resources of the ArcSDE host computer.
The administrator controls the behavior with the STREAMPOOLSIZE parameter.
STREAMPOOLSIZE
The process of creating a stream requires the allocation of memory and other resources. These are released when users close the stream, which normally happens when they disconnect from the ArcSDE application server.
If the STREAMPOOLSIZE is set to a value greater than 0, ArcSDE creates a pool of released stream resources for reuse. In this case, when a user releases a stream, the ArcSDE
application server first checks the stream pool to determine if it is full. If it is not, the released stream resources are added to the pool; otherwise, the resources are deallocated. When the same user application creates a stream, the ArcSDE application server checks the stream pool for an available stream. If one exists, ArcSDE removes it from the pool and allocates it to the user application.
Raster parameters
ArcSDE stores images similar to the way it stores features, in a DBMS BLOB column. For more information about raster tables, see ‘Appendix B: ArcSDE table definitions’.
RASTERBUFSIZE
The RASTERBUFSIZE parameter controls raster data transfer, which operates in a fashion similar to streamed data (see ‘Transport buffer parameters’ and ‘Array buffer parameters’ in this chapter).
The RASTERBUFSIZE is specified in bytes and must be large enough to hold the largest raster tile. The default ArcInfo tile size is 128 * 128 pixels. Data that is eight bits per pixel has a tile size of 16,384 bytes (128 * 128).
The raster transfer includes both an array buffer and transport buffers. The raster array buffer is set at two times the
RASTERBUFSIZE parameter, while the raster transport buffers are set to the RASTERBUFSIZE. Therefore, the memory allocated to raster transfer on the server is three times RASTERBUFSIZE. On the client, RASTERBUFSIZE bytes of memory are allocated to the client raster transport buffer. The raster buffers are allocated when raster tiles are accessed by a stream. The raster buffers are not deallocated until the stream is closed (unless the stream is added to the stream pool—see ‘STREAMPOOLSIZE’). In other words, a user can elect to display an image and a feature class together in ArcMap. The raster buffers are allocated on demand when a user selects an image stored in the feature class’s business table.
If the ArcSDE application server encounters a raster tile that does not fit into the transport buffer, the
SE_RASTERBUFFER_TOO_SMALL error is returned. If users report this error, determine the tile size they are using. If they are using a 256 x 256 tile size on an 8-bit image, the RASTERBUFSIZE must be at least 65,536 bytes (256 * 256 * 1). The pixel depth must be taken into account when calculating the number of bytes per pixel. For example, an image with a 64-bit pixel depth and a tile size of 256 * 256 requires a RASTERBUFSIZE of 524,288 bytes (256 * 256 * 8). If memory is at a premium, advise users to specify a smaller tile size rather than raise the RASTERBUFSIZE.
Maximum time difference
The maximum time difference that a client’s system clock can deviate from an application server’s system clock can be set with the giomgr.defs MAXTIMEDIFF parameter. The parameter is specified in seconds. It prevents an unauthorized entry by individuals who may have captured a network packet with a sniffer software that contains an ArcSDE connection string. Because the encrypted password is time stamped, the packet cannot be resent at a later time if MAXTIMEDIFF is set low enough (for example, 60 seconds).
If legitimate connections receive a “-99 password received was sent 7 MAXTIMEDIFF seconds before” error, reset the client machine’s system time to the host’s system time.
To disable MAXTIMEDIFF, set it to -1.
The MAXTIMEDIFF parameter does not affect direct connections.
Controlling transactions
The AUTOCOMMIT parameter lets you control the commit rate of large transactions and prevents the DBMS logfiles that store rollback information from filling. The default value of 1,000 specifies that a transaction is committed whenever it reaches 1,000 updates. Setting this parameter to 0 disables automatic commits and requires the application to explicitly commit the transactions.
Logfiles
ArcSDE provides logfiles to store lists of table rows. ArcMap, for example, stores selection sets exceeding 100 rows in a logfile. Physically, the list of row IDs, referred to as the logfile, are stored in a logfile data table.
At ArcSDE version 9, enhancements to the logfile data storage were introduced to solve problems experienced by some applications using the ArcSDE shared logfile method. ArcSDE 8.3 and prior releases store logfiles in the
SDE_LOGFILES and SDE_LOGFILE_DATA tables created in each users schema. By default these logfile tables were created by ArcSDE the first time the user connected and all deletes from the SDE_LOGFILE_DATA table were deferred until the user
terminates the session. This type of logfile data storage is referred to as the ArcSDE shared logfile method.
ArcIMS users, and to some extent ArcGIS users, experienced problems with the deferred delete functionality of the ArcSDE
shared logfile method. The number of rows contained within the SDE_LOGFILE_DATA table could grow in size, reaching millions of records, until the user disconnected. The deletion of the rows following the disconnect could take several minutes. Starting at ArcSDE 9, all logfile entries are deleted immediately.
Also, since the logfile tables must be created in the user’s schema, DBMS administrators were required to grant privileges to each user that would allow them to create the required data objects.
Another shortcoming of the ArcSDE shared logfile method is that all sessions connecting as the same user wrote to the same table. This contributed to the inflated growth of the
SDE_LOGFILE_DATA table.
Enhancements to the logfile system beginning with ArcSDE 9 allow administrators to create standalone and session logfiles, as well as pools of logfiles owned by the sde user that can be checked out by other users.
Therefore, beginning at ArcSDE 9, logfile data can be stored in one of three kinds of tables that include standalone, session, and shared.
Standalone logfile data tables
Standalone logfile tables contain a single logfile. When the logfile is deleted, the table is immediately truncated to remove the records. The truncation avoids the costly logging of changes to the DBMS.
Standalone logfiles are created in the user’s schema only when none are available for checkout from the sde user’s logfile pool. Standalone logfiles are used until the user’s quota, defined by the MAXSTANDALONELOGS server configuration parameter, is exhausted.
Standalone logfiles are checked out of the logfile pool until the pool is exhausted. The total number of logfiles in the pool is defined by the SESSIONLOGPOOLSIZE server configuration parameter. Set the HOLDLOGPOOLTABLE parameter to TRUE if you want users to retain the checked out logfiles following a logfile delete.
Session logfile data tables
Session logfile data tables are dedicated to a single session and may contain multiple logfiles. When a logfile is deleted, the records are immediately removed from the session logfile data table. The table is truncated if no other logfiles are present; otherwise the logfile records are deleted.
Session logfiles are used if the ALLOWSESSIONLOGFILE parameter is set to TRUE and the user’s quota of standalone logfiles has not been exhausted.
Session logfiles are created in the user’s schema when they cannot be checked out from the sde user’s logfile pool.
Shared logfile data tables
ArcSDE reverts to using the ArcSDE shared logfile data table, the default, if it cannot use a standalone or session logfile data table. This will happen if MAXSTANDALONELOGS is exhausted and ALLOWSESSIONLOGFILE is FALSE. If the shared logfile tables cannot be created, ArcSDE returns an SE_NO_ACCESS (-15).
ALLOWSESSIONLOGFILE
Allows your users to use a session logfile. By default this parameter is set to FALSE.
SESSIONLOGPOOLSIZE
Specifies the number of logfiles that may be borrowed from the sde user’s logfile pool. Once the users have exhausted the pool, ArcSDE will attempt to create either the standalone or session logfile in the user’s schema. By default the logfile pool is set to 0. If you do not want your users creating logfiles in their own schemas, set this value high enough that all logfiles are borrowed.
HOLDLOGPOOLTABLE
Set to TRUE, this parameter directs the user session to retain a logfile table that it has borrowed from the sde user until the session disconnects from ArcSDE. Setting the parameter to FALSE will result in the sde user’s logfiles being shared among a greater number of sessions at the expense of the allocation or deallocation of the logfile tables. By default this parameter is set to TRUE.
MAXSTANDALONELOGS
The maximum number of standalone logfiles a user may use. By default this parameter is set to 0.
States
ArcSDE uses states to manage the chronological order of changes made to the registered tables of the geodatabase. States are added, updated, and deleted as are other objects of the database. As a change is made to a registered table, states are added to the database. States are deleted when versions are removed from the database and the state tree is trimmed. Changes stored in the form of adds and deletes in tables associated with the registered tables are coalesced into the base tables. The intensity with which ArcSDE interacts with the states tree depends on the application and the number of users. Periodically, you need to compress the database using either ArcCatalog or the sdeversion administration command to reduce the size of the state tree. The following parameters help to reduce the interaction with the state tree but should be used to replace or delay
necessary maintenance of the state tree.
STATEAUTOLOCKING
By default the autolocking of the states is disabled (set to FALSE) and should not be set to TRUE if you are editing multiversioned tables with applications constructed by ESRI. ESRI’s applications are designed not to delete or alter states that are currently opened by other sessions. However, if you are using a non-ESRI application that manages states external to
ArcObjects™ you should set this parameter to TRUE. ESRI’s applications will then lock states in use to protect them from being deleted by the third party application. Turning
STATEAUTOLOCKING on adds overhead to management of the state tree and does adversely affect the performance of the ArcGIS desktop applications attempting to perform edits on the geodatabase.