D67016GC20 Exadata Workshop Part2

(1)

O ti i i

D t b

P f

ith

Optimizing Database Performance with

Exadata

(2)

Objectives

After completing this lesson, you should be able to:

• Describe the recommendations for optimizing Exadata

performance

(3)

Optimizing Performance

• Start with best practices for ASM and Oracle Database

• Areas for special consideration:

–

Flash memory usage

–

Compression usage

–

Compression usage

–

Index usage

–

ASM allocation unit size

–

Minimum extent size

Optimizing Performance

Optimizing the performance of your Exadata environment begins with following good practices for database design and application development From an administration

practices for database design and application development. From an administration

viewpoint, you should continue to follow the best practices for ASM and Oracle Database in conjunction with advice and statistics provided by tools such as SQL monitor and SQL Tuning Advisor.

In addition, there are a number of areas for special consideration listed in the slide. These are covered in the remainder of this lesson.

(4)

Flash Memory Usage

• Each Exadata server contains 384 GB of high performance

flash memory.

• Flash memory can be configured in two ways:

–

Exadata Smart Flash Cache:

—

Speeds up access to frequently accessed data

—

Uses all available flash memory by default

—

Can be managed automatically for maximum efficiency

— Users can only provide optional hints to influence caching priorities.

—

Beneficial for OLTP and Data Warehouse workloads

–

Flash-based permanent storage:

—

Uses flash memory as a premium storage device

—

Must be configured by an administrator

—

Requires deliberate planning to use efficiently

—

Additional benefits for specific applications

Flash Memory Usage

Each Exadata server contains 384 GB of high performance flash memory.

B d f lt h E d t i fi t fi d thi fl h i fi d E d t

By default, when Exadata is first configured, this flash memory is configured as Exadata Smart Flash Cache. Exadata Smart Flash Cache provides a caching mechanism for

frequently accessed data on each Exadata cell. It is a write-through cache which is useful for absorbing repeated random reads, and very beneficial to online transaction processing (OLTP). Exadata Smart Flash Cache also benefits data warehousing applications by caching the most frequently accessed data.

Exadata Smart Flash Cache automatically focuses on caching frequently accessed data and y g q y index blocks, along with performance critical information such as control files and file headers. In addition, DBAs can influence caching priorities using the CELL_FLASH_CACHE storage attribute for specific database objects.

For example, to prioritize caching for the CALLDETAIL table use the following SQL command: ALTER TABLE calldetail STORAGE (CELL_FLASH_CACHE KEEP)

Because of this setting, Exadata will cache data for the CALLDETAIL table more aggressively than otherwise. Without this setting, Exadata caches individual table and index blocks. With this setting, Exadata attempts to cache scans on the CALLDETAIL table. This makes it more likely that subsequent scans on the CALLDETAIL table will also be satisfied using Exadata Smart Flash Cache.

(5)

Flash Memory Usage (continued)

This setting also causes Exadata to clear out other tables from the cache before clearing out cached data for the CALLDETAIL table. This means that CALLDETAIL records will remain in Exadata Smart Flash Cache longer than cached data for other tables. ada a S a as Cac e o ge a cac ed da a o o e ab es

If the CALLDETAIL table is spread across several Exadata servers, each one will cache its part of the table in its own Exadata Smart Flash Cache. If the caches are sufficient in size, the CALLDETAIL table is likely to be completely cached over time.

Alternatively, you can configure a smaller Exadata Smart Flash Cache and use the remaining flash memory as a flash disk drive. You can then create grid disks based on flash memory. These flash disks can be used to configure high performance ASM disk groups housing their own tablespaces. In this case, the flash memory is not used to cache data, rather it becomes a high performance permanent store for some of your data. You should consider configuring flash-based permanent storage areas in the following cases:

• Heavy write applications: For applications that perform many updates, or in cases of heavy data ingest, you may consider storing specific tables, partitions or indexes on flash disks. In these cases, the high volume of writes can consume a considerable portion of the physical disk bandwidth. Storing insert- and update-intensive tables, partitions, and

p y g p , p ,

indexes on flash-based storage means that there are no physical disk I/Os associated with those objects. The physical disk bandwidth saved using this configuration can be used by other operations increasing the overall throughput of the system. Flash disks can also be used effectively in conjunction with partitioning. In many time-based partitioning scenarios, the most current data can be quite volatile while older data is subject to few updates, if any. In these situations, you can host your most current partitions in a flash-based tablespace and routinely move these partitions to a disk-based tablespace as their tablespace and routinely move these partitions to a disk based tablespace as their volatility diminishes due to age.

• Star-schemas: In a data warehouse based on a star schema, you may be able to store your entire fact table on flash disk. This effectively acts as a permanent cache for this key table. If your fact table is too large to store on flash disk, you can partition it and store the most referenced partition on flash disk.

The flash memory on each cell can be divided between Exadata Smart Flash Cache and permanent flash-based storage. Both areas can be any size as long as the total does not

exceed 384 GB on each cell. In practice, the effectiveness of Exadata Smart Flash Cache drops if the cache size is very small so in most cases, you would allocate a substantial proportion of flash memory to Exadata Smart Flash Cache.

Note that Exadata Smart Flash Cache is largely automatic. After it is configured, you can only influence caching priorities for specific tables using the CELL_FLASH_CACHE storage attribute. In contrast, configuring permanent flash-based storage requires accurate space planning to In contrast, configuring permanent flash based storage requires accurate space planning to make the most effective use of this premium resource.

(6)

Compression Usage

Compression Method CREATE/ALTER TABLESyntax Compression Ratio CPU Overhead Typical Applications Basic Compression COMPRESS [BASIC]

High for direct path inserts

Conventional path inserts and updates are not compressed

Minimal: Oracle Database performs compression and decompression

DSS

are not compressed OLTP

Compression

COMPRESS FOR OLTP

High for all transaction types

Minimal: Oracle Database performs compression and decompression, compression for DML is performed in batches OLTP and DSS Warehouse Compression COMPRESS FOR QUERY [LOW|HIGH]

Higher for direct path inserts

High for conventional

Higher: Decompression is performed by Exadata

DSS

path inserts and updates Online Archival Compression COMPRESS FOR ARCHIVE [LOW|HIGH]

Highest for direct path inserts

High for conventional path inserts and updates

Highest: Decompression is performed by Exadata

Archiving

Compression Usage

Many people think of data compression as they approach the capacity limits of their storage. Commonly data compression is seen as a performance overhead which must be tolerated in

Commonly, data compression is seen as a performance overhead, which must be tolerated in order to deliver extra storage capacity. However, in cases where I/O bandwidth is limited, data compression can be an effective tool to increase performance by using available CPU

capacity to effectively increase the I/O throughput of a storage system. Oracle Database provides the following modes of data compression:

• Basic compression compresses data at the Oracle block level. It allows more data to be stored in each block by replacing duplicate values with a smaller symbolic

t ti F l if th i l t 50 ti i

representation. For example, if the numeric value 99999 was present 50 times in a block of data it could be replaced by 50 occurrences of the # symbol along with an entry in a symbol table (also stored in the block) defining the use of the symbol. The degree of compression depends on the level of duplication in each block. Higher compression ratios can be achieved by using larger block sizes or by sorting data to increase the coincidence of duplicate values. Data remains in row-major format where the columns in each row are stored together. Compression occurs when data is loaded using a direct path load operation, such as CREATE TABLE AS SELECT or direct path SQL*Loader. Tables that use basic compression support DML operations, however any data that is inserted or updated remains uncompressed. Basic compression is useful in decision support systems (DSS) where the compressed data is subject to minimal change.

(7)

Compression Usage (continued)

• OLTP compression uses the same compression technique as basic compression, but provides support for compressing transactional data from all DML statements, not just direct path loads. This makes it useful in DSS and online transaction processing (OLTP) p p g ( ) environments. Instead of compressing changes as they are written, all the changes to a data block are compressed as a batch whenever a change causes the block to become fuller than an internally controlled threshold. In other words, only the DML statements that trigger compression experience the overhead of compression for the associated data blocks, however most DML statements experience no compression overhead at all. This means that compression can be implemented on OLTP systems without significantly impacting the overall transactional throughput of the system.

impacting the overall transactional throughput of the system.

In addition to the basic and OLTP compression modes provided by Oracle Database, Exadata provides Exadata Hybrid Columnar Compression. Exadata Hybrid Columnar Compression technology uses a modified form of columnar storage instead of row-major storage. Sets of rows are stored in an internal structure called a compression unit. Within each compression unit, the values for each column are stored together along with metadata that maps the values to the rows. Compression is achieved by replacing repeating values with smaller symbolic references. Because a compression unit is much larger than an Oracle block and because column

Because a compression unit is much larger than an Oracle block, and because column

organization brings similar values together, Exadata Hybrid Columnar Compression can deliver much better compression ratios than both basic and OLTP compression. The best rates of compression are achieved using direct path loads.

Exadata Hybrid Columnar Compression provides a choice of compression modes to achieve the proper trade-off between disk usage and CPU overhead:

• Warehouse compression: This type of compression is optimized for query performance,Warehouse compression: This type of compression is optimized for query performance,

and is intended for DSS and data warehouse applications.

• Online archival compression: This type of compression is optimized for maximum

compression ratios, and is intended for historical data and data that does not change. Exadata Hybrid Columnar Compression supports DML operations on compressed data.

However, such operations attract a considerable CPU overhead and yield a lower compression ratio than for direct path loads. In addition, DML on tables using Exadata Hybrid Columnar Compression requires the entire compression unit to be locked, which may reduce concurrency. Finally, updates to rows using Exadata Hybrid Columnar Compression cause rowids to change. As a result, Exadata Hybrid Columnar Compression is recommended for situations where data changes are infrequent or where data sets are reloaded rather than substantially changed. In conclusion, Exadata Hybrid Columnar Compression makes effective use of Exadata server hardware to deliver the highest levels of compression for data in an Oracle database. It is best suited to cases where the data is not subject to substantial change. For transactional data sets, su ed o cases e e e da a s o subjec o subs a a c a ge o a sac o a da a se s, you should consider OLTP compression instead of Exadata Hybrid Columnar Compression. In all cases, you should be aware of the relative merits and overheads associated with each compression type in order to choose the best approach for your situation.

(8)

Index Usage

• With Exadata, scans might perform better than index

lookups for some queries.

• Consider removing indexes where scans deliver

acceptable performance.

p

• Removing unnecessary indexes will:

–

Improve DML performance

–

Save storage space

• Test the effect of removing indexes by making them

invisible:

SQL> ALTER INDEX <index_name> INVISIBLE;

Index Usage

Some queries that require indexes when using conventional storage will perform acceptably without indexes using Exadata Review your queries that use indexes to determine if they

without indexes using Exadata. Review your queries that use indexes to determine if they would run acceptably with Exadata scans. To test if queries run acceptably without an index, you can make the index invisible to the optimizer. An invisible index still exists and is

maintained by DML operations, but it is not used by the optimizer for queries. To make an index invisible, use the following command:

ALTER INDEX <index_name> INVISIBLE

Removing unnecessary indexes will aid the performance of DML operations and will save storage space.

(9)

ASM Allocation Unit Size

• By default, ASM uses an allocation unit (AU) size of 1 MB.

• For Exadata, the recommended AU size is 4 MB.

–

AU size must be set when a disk group is created.

–

AU size cannot be altered after a disk group is created

–

AU size cannot be altered after a disk group is created.

–

AU size is set using the AU_SIZE disk group attribute:

SQL> CREATE DISKGROUP data NORMAL REDUNDANCY DISK 'o/*/data_CD*'

ATTRIBUTE 'compatible.rdbms' = '11.2.0.0.0',

'compatible.asm' = '11.2.0.0.0',p ,

'cell.smart_scan_capable' = 'TRUE',

'au_size' = '4M';

ASM Allocation Unit Size

To achieve fast disk scan rates, it is important to lay out segments with at least 4 MB of contiguous space This allows disk scans to read 4 MB of data before performing another

contiguous space. This allows disk scans to read 4 MB of data before performing another seek at a different location on disk. To ensure segments are laid out with 4 MB of contiguous space, set the Oracle ASM allocation unit size to 4 MB. The allocation unit can be set with the AU_SIZEdisk group attribute when creating the disk group.

(10)

Minimum Extent Size

• Large segments should have initial extents of 8 MB:

–

Stops needless proliferation of small extents in the database

–

Minimizes sub-optimal reads for small extents

–

Compliments the 4 MB ASM AU size recommendation

• Large initial extents can be specified using the:

–

INITIAL

storage parameter for specific objects or for all

objects in a locally managed tablespace

–

CELL_PARTITION_LARGE_EXTENTS

initialization

parameter for all new segments in a partitioned object:

—

TRUE

: Sets an 8 MB minimum extent size, but will trim last

unloaded extents to 64 KB boundary for direct loads

—

ALWAYS

: Sets an 8 MB minimum extent size and also minimizes

the creation of suboptimal extent sizes at the end of a load

—

DEFAULT

: Uses traditional extent allocation

Minimum Extent Size

Extent size is managed automatically in locally managed tablespaces. This option automatically increases the size of the extent depending on segment size available free

automatically increases the size of the extent depending on segment size, available free space in the tablespace, and other factors. By default, the extent size starts at 64 KB and increases in increments of 1 MB, 8 MB, or 64 MB.

Generally speaking, it is recommended that large segments should be defined with larger than default initial extents to minimize the needless proliferation of small extents in the database. For Exadata, the recommendation is to have initial extents of 8 MB for large

segments. This setting complements the recommendation to set the ASM allocation unit size to 4 MB The following options can be used to influence the database to allocate large

to 4 MB. The following options can be used to influence the database to allocate large extents:

• The INITIAL storage parameter will set the size of the first extent for a newly created object. It can be set specifically for individual objects or it can be set at the tablespace level for locally managed tablespaces. For objects that are large, set INITIAL to 8 MB. • The CELL_PARTITION_LARGE_EXTENTS initialization parameter can be used to set

the INITIAL storage parameter to 8 MB for all new segments in a partitioned object. g p g p j This parameter is relevant only to locally managed tablespaces created with the AUTOALLOCATEoption, not the UNIFORM option. The parameter does not affect nonpartitioned objects. The parameter may be set to the following values: TRUE,

(11)

Quiz

For which of the following scenarios would it make sense to consider

altering the default Exadata Smart Flash Cache in order to configure a

flash-based tablespace in your database.

1. When you can fit 10 or more of your tables into a flash-based

t bl

tablespace.

2. When you can fit at least 80% of your most used tables into a

flash-based tablespace.

3. When moving specific tables and indexes to a flash-based

tablespace will soak up heavy write activity, freeing disk I/O

bandwidth for other operations

bandwidth for other operations.

4. When you have key application objects (such as a star-schema

fact table) that you wish to cache all the time, you understand

their space requirements intimately, and you are prepared to

accept the additional management effort of configuring a

flash-based tablespace.

Answer: 3, 4

(12)

Quiz

Which of the following CREATE TABLE compression

clauses are only available to use in conjunction with

Exadata storage.

1 COMPRESS BASIC

1. COMPRESS BASIC

2. COMPRESS FOR OLTP

3. COMPRESS FOR QUERY

4. COMPRESS FOR ARCHIVE

Answer: 3, 4

(13)

Summary

In this lesson, you should have learned how to:

• Describe the recommendations for optimizing Exadata

performance

(14)

Additional Resources

• Lesson Demonstrations (Viewlets)

–

Configuring Flash-Based Disk Groups

—

http://st-curriculum.oracle.com/demos/db/11g/r2/dbmach/071ExadataFla

shDiskGroups/071exadataflashdiskgroups_viewlet_swf.html

–

Examining Exadata Hybrid Columnar Compression

—

http://st-curriculum.oracle.com/demos/db/11g/r2/dbmach/072ExadataCo

mpressionAdmin/072exadatacompressionadmin_viewlet_swf.ht

ml

–

Index Elimination with Exadata

—

http://st-curriculum.oracle.com/demos/db/11g/r2/dbmach/073ExadataInd

exElimination/073exadataindexelimination_viewlet_swf.html

(15)

Practice 7 Overview:

Optimizing Database Performance with Exadata

In these practices, you will explore how to optimize database

performance using:

• Flash-based disk groups

• Exadata Hybrid Columnar Compression

Exadata Hybrid Columnar Compression

• Index elimination

(16)

(17)

D t b

M hi

O

i

d A hit t

Database Machine Overview and Architecture

(18)

Objectives

After completing this lesson, you should be able to:

• Describe the architecture and various configuration options

for Database Machine

(19)

Introducing Database Machine

• Fully integrated platform for Oracle

Database

• Based on Exadata storage

• High-performance and high-availability

High performance and high availability

for all Oracle Database workloads

• Balanced hardware configurations

• Well suited as a database consolidation

platform

Si

l

d f t t i

l

t

• Simple and fast to implement

Introducing Database Machine

Database Machine is a fully-integrated Oracle Database platform based on Exadata storage technology. Database Machine provides a high-performance, highly-available solution for all

database workloads, ranging from scan-intensive data warehouse applications to highly concurrent OLTP applications.

Special attention has been paid to ensure that Database Machine is a well balanced platform. Throughout the hardware architecture of Database Machine, components and technologies have been specially matched to eliminate bottlenecks while maintaining good hardware utilization.

Using the unique clustering and workload management capabilities of Oracle Database Using the unique clustering and workload management capabilities of Oracle Database, Database Machine is well-suited for consolidating multiple databases onto a single Database Machine. Delivered as a complete package of software, servers, and storage, Database Machine is simple and fast to implement.

Note: While Database Machine is a fully-integrated platform solution comprised of specific

hardware and software components, Oracle offers Database Machine as a series of separately purchasable components. Customers can choose from the different hardware configurations that are available Appropriate licensing of Oracle Database and Exadata software is also

Exadata and Database Machine Administration Workshop 8 - 3

that are available. Appropriate licensing of Oracle Database and Exadata software is also required. In addition, Database Machine is highly complementary with clustering and parallel operations, so Oracle Real Application Clusters and Oracle Partitioning are highly

(20)

Database Machine X2-2 Full Rack

8 Intel-based 2 CPU Sun Fire database servers 14 Exadata Storage Servers

(all high performance or all high capacity)

2 Power Distribution Units (PDUs) at rear

3 Sun Datacenter InfiniBand Switch 36

(36-port managed QDR i h 40Gb/ )

Keyboard, Video, Mouse (KVM) hardware

1 Cisco Ethernet switch (48-port)

switch - 40Gb/s)

Database Machine X2-2 Full Rack

Database Machine is offered in 4 configurations. The contents of a Database Machine X2-2 Full Rack configuration is illustrated in the slide.

(21)

X2-2 Database Server Hardware Details

(Sun Fire X4170 M2)

Processors 2 Six-Core Intel® Xeon® X5670 processors (2.93 GHz)

Memory 96 GB (12 x 8 GB)

Local Disks Four 300 GB 10K RPM SAS disks

Disk Controller Disk controller HBA with 512 MB battery backed cache

Network Two InfiniBand 4X QDR (40 Gb/s) ports

(1 dual-port PCIe 2.0 HCA)

Four 1GbE Ethernet ports

Two 10GbE Ethernet SFP+ ports

(1 dual-port 10GbE PCIe 2.0 network card based on the Intel 82599 10GbE Controller technology)

Remote Management 1 Ethernet port (ILOM)

Power Supplies 2 redundant hot-swappable power supplies

X2-2 Database Server Hardware Details (Sun Fire X4170 M2)

Database Machine X2-2 uses standardized database server hardware based on 64 bit Intel-based Sun Fire servers The slide shows a description of the database server hardware

based Sun Fire servers. The slide shows a description of the database server hardware.

(22)

Start Small and Grow

X2-2 Full

Rack

X2-2

Quarter

Rack

X2-2 Half

Rack

Start Small and Grow

In addition to a Database Machine X2-2 Full Rack, which contains eight database servers, 14 Exadata cells and 3 InfiniBand switches, Oracle offers two other Database Machine X2-2

configurations which allow customers to start small and grow:

• A Quarter Rack contains two database servers, three Exadata cells and 2 InfiniBand switches.

• A Half Rack contains four database servers, seven Exadata cells and 3 InfiniBand switches.

All of the Database Machine X2-2 configurations are packaged in a standard 19-inch 42U rack. All configurations are pre-cabled and supplied with a KVM and Ethernet switch to complete the configuration. In addition, Oracle offers hardware upgrade kits that allow customers to upgrade from a Quarter Rack to a Half Rack, or to upgrade from a Half Rack to a Full Rack.

(23)

Database Machine X2-8 Full Rack

2 Intel-based 8 CPU Sun Fire database servers 14 Exadata Storage Servers

(all high performance or all high capacity)

2 Power Distribution Units (PDUs) at rear

3 Sun Datacenter InfiniBand Switch 36

(36-port managed QDR i h 40Gb/ )

1 Cisco Ethernet switch (48-port)

switch - 40Gb/s)

Database Machine X2-8 Full Rack

In addition to the three Database Machine X2-2 configurations, Oracle now offers Database Machine X2-8 Database Machine X2-8 is only offered in a Full Rack There are no smaller

Machine X2 8. Database Machine X2 8 is only offered in a Full Rack. There are no smaller configurations.

In many ways Database Machine X2-8 is essentially the same as Database Machine X2-2 Full Rack. Both contain 14 Exadata cells, 3 InfiniBand switches, 2 power distribution units (PDUs) and an Ethernet switch.

The major difference is that the eight 2 CPU database servers in Database Machine X2-2 Full Rack are replaced by two 8 CPU database servers. Each of the larger database servers p y g contains more RAM and is equipped with eight 10 Gb/sec Ethernet ports for high-speed client network access in addition to the 1 Gb/sec Ethernet ports that support administrative access. Database Machine X2-8 does not come equipped with KVM hardware.

Database Machine X2-8 complements the X2-2 models. Like the X2-2 models, it is designed for database consolidation, large OLTP and data warehouse applications. The large memory configuration of Database Machine X2-8 makes it a particularly good platform for database

lid ti C t th t tl th i li ti l SMP t b

consolidation. Customers that currently run their applications on large SMP systems may be more comfortable choosing Database Machine X2-8.

(24)

X2-8 Database Server Hardware Details

(Sun Fire X4800)

Processors 8 x Eight-Core Intel® Xeon® X7560 Processors (2.26 GHz) Memory 1 TB (128 x 8 GB)

Local Disks Eight 300 GB 10K RPM SAS disks

Disk Controller Disk controller HBA with 512 MB battery backed cache Network Eight InfiniBand 4X QDR (40Gb/s) ports

(4 dual-port PCE 2.0 express modules)

Two Network Express Modules (NEM), providing a total of Eight 1GbE Ethernet ports and

Eight 10 GbE Ethernet SFP+ ports

(4 Fabric Express Modules (FEM) using Intel 82599 10GbE controller technology) Remote Management 1 Ethernet port (ILOM)

Power supplies 4 redundant hot-swappable power supplies

X2-8 Database Server Hardware Details (Sun Fire X4800)

The slide shows a description of the database server hardware used in Database Machine X2-8

(25)

Database Machine Capacity

X2-2

Quarter

Rack

X2-2

Half

Rack

X2-2

Full

Rack

X2-8

Full

Rack

High Performance

21 TB 50 TB 100 TB 100 TB

Raw Disk

1

High Performance

21 TB 50 TB 100 TB 100 TB

High Capacity

72 TB 168 TB 336 TB 336 TB

Raw Flash

1 _{1.1 TB} _{2.6 TB} _{5.3 TB} _{5.3 TB}

User Data

2

High Performance

6 TB 14 TB 28 TB 28 TB

1 - Raw capacity calculated using 1 GB = 1000 x 1000 x 1000 bytes and 1 TB = 1000 x 1000 x 1000 x 1000 bytes. 2 - User Data: Actual space for uncompressed end-user data, computed after single mirroring (ASM normal redundancy) and after allowing space for database structures such as temporary space, logs, undo space, and indexes. Actual user data capacity varies by application. User Data capacity calculated using 1 TB = 1024 * 1024 * 1024 * 1024 bytes.

User Data

High Capacity

21 TB 50 TB 100 TB 100 TB

Database Machine Capacity

The table in the slide gives you some key capacity metrics for the available Database Machine configurations.

g

(26)

Database Machine Performance

X2-2

Quarter

Rack

X2-2

Half

Rack

X2-2

Full

Rack

X2-8

Full

Rack

Raw Disk

High Performance

5.4 GB/s 12.5 GB/s 25 GB/s 25 GB/s

Raw Disk

Throughput

1

g

High Capacity

3 GB/s 7 GB/s 14 GB/s 14 GB/s

Raw Flash Throughput

1 _{11 GB/s} _{25 GB/s} _{50 GB/s} _{50 GB/s}

Raw Disk

IOPS

2

High Performance

10,800 25,000 50,000 50,000

High Capacity

5 400 12 500 25 000 25 000

1 – Bandwidth is peak physical disk scan bandwidth, assuming no compression. 2 – IOs per second (IOPS) – Based on IO requests of size 8K

IOPS

_{High Capacity}

_5,400 _12,500 _25,000 _25,000

Flash IOPS

2 _225,000 _500,000 _{1,000,000 1,000,000}

Database Machine Performance

The above table gives you some key performance metrics for the different Database Machine configurations.

g

Note: These metrics do not take into account compression. As a result, they are quite

conservative. With compressed data, you can achieve much higher throughput rates. In all cases, actual performance will vary by application.

(27)

Database Machine X2-2 Architecture

The diagram in the slide shows how the major components of Database Machine X2-2 are connected to each other The diagram shows the KVM hardware Ethernet switch and

connected to each other. The diagram shows the KVM hardware, Ethernet switch and

InfiniBand switches. For the sake of clarity, only one database server and one Exadata server are shown. Database Machine contains three major networks:

• Management/ILOM Network. This is a standard Ethernet/IP network which is used to manage Database Machine. Using this network, administrators can access the

database servers and the Exadata servers using the KVM hardware or by using remote login facilities such as Secure Shell (SSH). The database servers and Exadata servers

l id Eth t i t f f I t t d Li ht O t M t (ILOM) ILOM

also provide an Ethernet interface for Integrated Lights-Out Management (ILOM). ILOM provides a powerful set of remote administration facilities. Using ILOM, administrators can remotely monitor and control the state of the server hardware. The InfiniBand switches also provide an Ethernet port for remote configuration and management purposes.

• Client Access Network. There is also a standard Ethernet network which is primarily used to provide database connectivity via Oracle Net software. The client access

network can be separate from the management network (as illustrated in the diagram) or both functions can be provided by the same network. Single-network configurations are quite typical.

(28)

Database Machine X2-2 Architecture (continued)

When Database Machine is initially configured, customers can choose to configure the database servers with a dedicated client network interface (NET1) or they can choose to configure a bonded network interface (using NET1 and NET2). A bonded g ( g ) client access network interface can provide protection if a network interface fails however using bonded interfaces may require additional configuration in the customer’s network.

• InfiniBand Network. The InfiniBand network is to provides a reliable high-speed interconnect for Database Machine. The InfiniBand network in described in greater detail later in this lesson.

I dditi th NET2 d NET3 i t f th d t b b ti ll

In addition, the NET2 and NET3 interfaces on the database servers can be optionally configured for additional networks.

Note: Database Machine X2-8 is essentially the same except that each database server

has more physical Ethernet ports and each database server also contains four bonded InfiniBand network interfaces connecting to the storage network.

(29)

InfiniBand Network Architecture

• Sun Datacenter 36-port managed QDR (40 Gb/s) switches

–

2 leaf switches used to connect server InfiniBand ports

–

1 spine switch in Full Rack and Half Rack configurations only

• Exadata Servers and X2-2 Database Servers

–

Each server has one dual-port QDR (40 Gb/s) InfiniBand HCA

–

Each HCA port is connected to a different leaf switch for high

availability

–

Active-passive bonding with single IP address

— Performance is limited by PCIe bus, so active-active not needed

• X2 8 Database Servers

• X2-8 Database Servers

–

Each server has four dual-port QDR (40 Gb/s) InfiniBand HCAs

–

Each pair of HCA ports are connected to different leaf switches for

high availability

–

Active-passive bonding for each bonded pair

• Connections are cabled in the factory

InfiniBand Network Architecture

Within Database Machine, the primary role of the InfiniBand network is to provide a reliable high-speed interconnect to support database server to storage server communications It is

y

high speed interconnect to support database server to storage server communications. It is also used to facilitate communication and coordination between database servers using Oracle Clusterware and Oracle Real Application Clusters (RAC) software. It is important to note that the InfiniBand network is completely internal to Database Machine and there is no requirement for additional InfiniBand infrastructure within the customer’s network.

Each Database Machine configuration uses at least two Sun Datacenter 36-port managed QDR (40 Gb/s) InfiniBand switches. Each database server and each Exadata server is

t d t b th it h i b d d t k i t f T it h

connected to both switches using one or more bonded network interfaces. Two switches are used to provide high availability. If a switch fails, then the InfiniBand network can be

maintained by the remaining switch. In such a case there would be no performance loss because each switch is capable of supporting the full system bandwidth. The switches that connect the database servers and Exadata servers within a rack are known as leaf switches. Full Rack and Half Rack Database Machine configurations contain a third InfiniBand switch, known as the spine switch. The spine switch connects the 2 leaf switches and is also used to p p connect multiple Database Machines. Quarter Rack configurations do not include a spine switch.

(30)

X2-2 Leaf Switch Topology

The diagram in the slide illustrates how the two leaf switches in each Database Machine X2-2 rack are connected to the database servers Exadata servers and to each other Each

rack are connected to the database servers, Exadata servers and to each other. Each database server and each Exadata server is configured with a bonded network interface (BOND0) which uses the two InfiniBand HCA ports on the server. Active/passive bonding is used. On each server, each InfiniBand port is connected to a different leaf switch. The active and passive connections are spread across the leaf switches to balance the workload. In addition, the leaf switches are connected to each other via seven inter-switch links. All the connections are pre-cabled in the factory. The topology described on this page is common to all Database Machine X2-2 configurations contained in a single rack; Quarter Rack Half Rack all Database Machine X2-2 configurations contained in a single rack; Quarter Rack, Half Rack and Full Rack. Multiple rack configurations are discussed later in the lesson.

Note: The switch port mappings shown in the diagram are illustrative only.

Note: Database Machine X2-8 has a similar topology except that each database server has

(31)

Full Rack Spine and Leaf Topology

The diagram in the slide shows the added spine switch which exists in Full Rack and Half Rack Database Machine configurations only As shown in the diagram each leaf switch is

Rack Database Machine configurations only. As shown in the diagram, each leaf switch is connected to the spine switch using a single network link. The purpose of the spine switch is to connect multiple Database Machines together, creating a single large-scale database grid.

(32)

Scale Performance and Capacity

Scalable

• Scale to eight racks by adding

Redundant and Fault Tolerant

• Failure of any component

g

y

g

cables

• Scale beyond eight racks by

adding InfiniBand switches

• Scale to hundreds of storage

servers to support

multi-petabyte databases

y

p

is tolerated.

• Data is mirrored across

storage servers.

Scale Performance and Capacity

If you need to scale beyond the performance or capacity of a single Database Machine configuration, you can scale up to eight Full Racks by simply adding some cabling between

g y g y y g g

them. There are enough free ports in the InfiniBand switches and enough bandwidth for eight Full Rack Database Machines running at full performance and with full redundancy.

If you want to extend beyond eight racks, you need to add InfiniBand switches.

In addition, Oracle now enables customers to interconnect a Half Rack Database Machine with other Half Rack or Full Rack configurations. Interconnecting Quarter Rack Database Machines is not supported.

(33)

Scaling Out to Multiple Full Racks

• Single InfiniBand network based on a Fat Tree topology

–

Up to 8 racks supported with existing switches

–

Database and storage server cabling unchanged

• Two rack example:

Two rack example:

Scaling Out to Multiple Full Racks

When multiple Full Rack Database Machines are connected to form a large-scale database grid, the InfiniBand network is modified to resemble a Fat Tree topology. The database server

g , p gy

and storage server connections to the leaf switches remain unchanged.

In a standalone Full Rack Database Machine, each leaf switch has 8 links that connect it to the other leaf switch in the rack (7 links) and to the spine switch (1 link). When multiple full racks are interconnected, these 8 links are redistributed to the spine switches. The diagram in the slide illustrates this using a two rack example. As shown in the diagram, the seven inter-switch links that originally connected the leaf inter-switches in each rack have been redistributed so that each leaf switch has 4 links to each spine switch. There are no direct links between spine

it h d th di t li k b t l f it h

switches and there are no direct links between leaf switches.

To connect eight Full Rack Database Machines, the eight leaf-to-spine links on each leaf switch are configured to provide exactly one link to each spine switch. For configurations between 2 and 8 racks, the eight leaf-to-spine links on each leaf switch are distributed as evenly as possible amongst the total number of spine switches.

Oracle supports interconnecting up to eight Full Rack Database Machines using the existing switches in each rack. Scaling beyond eight racks requires larger external switches.

Note: Up to three Full Rack Database Machines can be interconnected using the cabling

supplied from the factory. To interconnect more than three Database Machines, additional cabling must be purchased.

(34)

Quiz

Bonded interfaces are used for the InfiniBand storage network

on Database Machine servers to provide:

1. Fault Tolerance and High Availability

2 Increased Bandwidth and Performance

2. Increased Bandwidth and Performance

3. Both of the above

Answer: 1

Each database server and Exadata server is configured with bonded network interfaces which use the InfiniBand HCA ports on the server. Active/passive bonding is used so the aggregate

g gg g

bandwidth of the links is not used simultaneously. For each bonded network interface, each InfiniBand port is connected to a different leaf switch. This configuration provides fault tolerance and high availability by removing the InfiniBand switch as a potential single point of failure.

(35)

Quiz

Which of the following scalability options are supported?

1. Upgrading a Quarter Rack Database Machine to a Half

Rack Database Machine

2 Upgrading a Half Rack Database Machine to a Full Rack

2. Upgrading a Half Rack Database Machine to a Full Rack

Database Machine

3. Interconnecting two Quarter Rack Database Machines

4. Interconnecting two or more Full Rack Database Machines

Answer: 1, 2, 4

Interconnecting Database Machines is only supported for Half Rack and Full Rack configurations

configurations.

(36)

Summary

In this lesson, you should have learned how to:

• Describe the architecture and various configuration options

for Database Machine

(37)

D t b

M hi

C

fi

ti

Database Machine Configuration

(38)

Objectives

After completing this lesson, you should be able to:

• Describe the installation and configuration process for

Database Machine.

• Describe the default configuration for Database Machine

Describe the default configuration for Database Machine.

• Describe supported and unsupported customizations for

Database Machine.

(39)

Database Machine Implementation Overview

Four phases:

1. Pre-installation

–

Various planning and scheduling activities including:

—

Site planning: space power cooling logistics

—

Site planning: space, power, cooling, logistics…

—

Configuration planning: hostnames, IP addresses, databases…

—

Network preparation: DNS, NTP, cabling…

–

Oracle and customer engineers work together

2. Installation and configuration

–

Hardware and software installation and configuration

–

Result is a working system based on a default configuration

–

Conducted by Oracle engineers

Database Machine Implementation Overview

The process of successfully implementing Database Machine requires cooperation and coordination between various Oracle and customer representatives and engineers At a high

coordination between various Oracle and customer representatives and engineers. At a high level the process involves four main phases:

1. Pre-installation involves various planning and scheduling activities including those listed in the slide. Typically, Oracle will guide the customer through this phase by collaborating with them in order to complete a key document known as the Configuration Worksheet. The Configuration Worksheet drives the installation and configuration phase (Phase 2). Once it is completed there are additional preparation tasks that the customer will

t i ll l t Thi h i d ib d i t d t il l t i th l

typically complete. This phase is described in greater detail later in the lesson.

2. Installation and configuration is the process of taking a virgin Database Machine and installing it into the customer environment. It involves numerous steps conducted by hardware and software engineers. It is highly recommended that Oracle hardware and software engineers are used to conduct this phase because of the specialized nature of the tasks involved and to ensure that the Database Machine is configured in a standard and supportable way. The result of this phase is a working system which is configured according to the Configuration Worksheet. An example is shown later in the lesson.

(40)

Database Machine Implementation Overview

3. Additional configuration

–

Additional activities for production readiness including:

—

Reconfigure storage using non-default settings

—

Create additional databases

—

Configure Enterprise Manager

—

Configure backup and recovery

–

Conducted by customer or Oracle services engagement

4. Post-installation

–

Ongoing monitoring and maintenance

g

–

Conducted by customer or Oracle services engagement

Database Machine Implementation Overview (Continued)

3. After installation and configuration, the Database Machine is configured on the customer network Depending on the contents of the Configuration Worksheet the Database

network. Depending on the contents of the Configuration Worksheet, the Database Machine may include one or more running databases. At this point customers can commence the process of loading or migrating data and prepare Database Machine for production use. However, customers may wish to vary the configuration of Database Machine in order to create customized databases or to organize the available storage in a non-default way. Such additional configuration tasks are typically conducted by the customer or Oracle services are engaged to conduct the work. Other examples of additional configuration tasks include configuring Enterprise Manager to assist with additional configuration tasks include configuring Enterprise Manager to assist with ongoing monitoring and maintenance, or the configuration of a backup and recovery environment. Common supported and unsupported additional configuration activities are discussed later in the lesson.

4. During post-installation the focus of activities shifts to on-going monitoring and maintenance of Database Machine. This is usually conducted by the customer.

The remainder of this lesson focuses on topics that will assist you to understand some of the key activities in the implementation process. The aim is not to replace the available

documentation, rather the material is presented to assist you to better understand the available configuration options and to better appreciate the results of making different selections.

(41)

Configuration Worksheet Overview

The Configuration Worksheet:

• Is a Microsoft Excel spreadsheet with macros

• Contains site-specific configuration settings including:

Host and domain names

–

Host and domain names

–

IP addresses

–

Region and time zone information

–

Name servers and NTP time servers

–

Database server OS user and group settings

E d t

ll

tifi ti

tti

–

Exadata cell notification settings

–

ASM diskgroup definitions

–

Oracle database definitions

• Generates a series of configuration files that drive the

installation and configuration programs

Configuration Worksheet Overview

One of the most important Database Machine pre-installation tasks is to complete the Configuration Worksheet Completing the Configuration Worksheet is often a collaborative

Configuration Worksheet. Completing the Configuration Worksheet is often a collaborative effort involving database, network and system administrators. Typically, Oracle engineers will participate to provide assistance. The information in the configuration worksheet is used to configure Database Machine. It also defines the network names and addresses that need to be configured within your network before installing Database Machine. The following section examines the Configuration Worksheet in greater detail.

(42)

Getting Started

The slide shows a brand new Configuration Worksheet. The values in the worksheet are all pre-defined defaults The worksheet version is displayed at the very top of the worksheet

pre defined defaults. The worksheet version is displayed at the very top of the worksheet. Please check with your Oracle representative to ensure that you are using the latest available version.

Notice that the Configuration Worksheet implies a set of conventions for naming the various components of Database Machine. For example, the Compute Node Hostname is derived by concatenating the DB Machine Prefix along with the Compute Node Base Name and then adding a two digit numeric identifier at the end.

Notice also that the private InfiniBand network is configured by default to use addresses in the 192.168.10.Xrange. Because this network is self-contained within Database Machine, in most cases there is no requirement to change this setting.

While the Configuration Worksheet is quite flexible and allows many opportunities for customization, it is highly recommended that customers should stick to the default conventions wherever possible. This not only reduces the possibility for errors or misconfigurations but it also helps to ensure that the resulting configuration is easier to misconfigurations, but it also helps to ensure that the resulting configuration is easier to support.

(43)

Configuration Worksheet Example

The slide shows an example of the first part of a completed Configuration Worksheet. The following slide shows the rest of the example The example illustrates a typical configuration

following slide shows the rest of the example. The example illustrates a typical configuration using as many default values as possible. In this part of the Configuration Worksheet the following fields are set to non-default values:

• Customer Name: Setting the Customer Name to Example Industries has no effect on the configuration of Database Machine. The field simply helps to identify this instance of the Configuration Worksheet.

• DB Machine Prefix: DB Machine Prefix is an identifier used to generate host names for network interfaces for all the systems. Indirectly, this value is also used in other

identifiers such as cell disk and grid disk names. Oracle recommends that this value should be 4 characters or less to avoid possible issues with names that are too long. In this example, eidm is an abbreviation of Example Industries Database Machine.

Customers should choose a setting that reflects their own naming conventions. • dbMachine Model: Select a value from the drop-down list to reflect the environment

being configured. Setting this field modifies the rest of the Configuration Worksheet so be g co gu ed Sett g t s e d od es t e est o t e Co gu at o o s eet so that the correct number of database servers and Exadata cells are configured.

(44)

Configuration Worksheet Example (Continued)

• Domain Name: This field is set to the domain name of the management network where the Database Machine resides.

• Region: This field is set to the geographic region where the database machine resides • Region: This field is set to the geographic region where the database machine resides. • Time Zone: This field is set to the time zone where the database machine resides.

• NameServer: Use these fields to identify up to four DNS servers in the network where the Database Machine resides. While it is possible to implement Database Machine without DNS, Oracle recommends that DNS should be used in conjunction with Database Machine.

• NTP Time Server: Use these fields to identify up to four NTP time servers in the network NTP Time Server: Use these fields to identify up to four NTP time servers in the network where the Database Machine resides. NTP services are a mandatory requirement for Database Machine. NTP provides coordinated timing which synchronizes services across Database Machine. Without NTP, a lack of coordination can lead to database nodes being evicted from the cluster or Exadata cells being excluded from the storage pool. NTP

services also ensure that the timestamps written to the various log files are coordinated across Database Machine. Note that the Cluster Time Synchronization Service (CTSS) introduced in Oracle Clusterware11g Release 2 cannot be used to provide time services to introduced in Oracle Clusterware11g Release 2 cannot be used to provide time services to Exadata cells, hence it is not used for Database Machine.

• First Management/ILOM IP address [eth0]: Set this field to the first IP address in the range of IP addresses allocated for the management network. When this field is set, the

Configuration Worksheet sequentially allocates IP addresses for each network interface as shown in the Specific Network Information section of the example worksheet. You can change the default addressing policy by changing values in the Specific Network

Information section of the worksheet or you can change individual addresses by modifying Information section of the worksheet or you can change individual addresses by modifying the addresses generated after you click the Generate button (discussed later in the

lesson). Beware that if you modify the default addressing policy you must ensure that the resulting addresses are valid and there are no duplicates.

• First Client Access IP address [eth1]: Set this field to the first IP address in the range of IP addresses allocated for the client access network. When this field is set, the Configuration Worksheet sequentially allocates IP addresses for each network interface as shown in the Specific Network Information section of the example worksheet. You can change the default addressing policy by changing values in the Specific Network Information section of the worksheet or you can change individual addresses by modifying the addresses generated after you click the Generate button (discussed later). Beware that if you modify the default addressing policy you must ensure that the resulting addresses are valid and there are no duplicates.

In this example all the other fields contain default values. Other fields that might typically be set In this example all the other fields contain default values. Other fields that might typically be set to non-default values include:

• Cell Disk Size: Set this field to 600G for systems with high performance disks or 2TB for systems with high capacity disks.

• Backup requirements: Select from one of the available options (Tape backup, Disk backup only, Both Disk and Tape backup). This setting influences the default size of the ASM disk groups that are configured later in the worksheet.

• Subnet Mask: If the default values are not suitable, the various subnet mask fields should be adjusted to reflect the network where the Database Machine resides.

• Gateway: If the default values are not suitable, the various gateway fields should be adjusted to reflect the network where the Database Machine resides.

(45)

Configuration Worksheet Example

Configuration Worksheet Example (Continued)

Following on from the previous slide, this slide shows an example of the second portion of a completed Configuration Worksheet In this part of the Configuration Worksheet the following

completed Configuration Worksheet. In this part of the Configuration Worksheet the following fields are set to non-default values:

• Email To Notification Address, Email From Notification Address, Email From Name, Email Server, Notify via Email: This group of fields are set to facilitate email notification of Exadata cell alerts. Depending on the email server setup you may also need to set other fields in the SMTP and SNMP Details section of the worksheet. Exadata cell alerts can also be delivered via SNMP however this option is not configured in the example. • Number of Diskgroups: Set this field to a non-zero value to configure ASM disk groups.

By default, a setting of 2 results in the creation of disk groups named DATA and RECO as shown in the slide. This section of the Configuration Worksheet is described in greater detail on the next slide.

• Number of Databases: Set this field to a non-zero value to create databases on the Database Machine. The example in the slide shows the default settings for one database.

database

(46)

Configuration Worksheet Example (Continued)

Note the limited set of database configuration attributes that are available in the Configuration Worksheet. Using the Configuration Worksheet you cannot set many common database attributes, such as database character set for example. The default common database attributes, such as database character set for example. The default database created during installation uses Unicode database character set AL32UTF8. Oracle recommends Unicode for all new deployments. To use a different database

character set after installation, or to create databases with other non-default attributes, run Database Configuration Assistant (DBCA) to delete the default database, and to create a new database using the desired settings.

Note also that if you do not specify any databases in the Configuration Worksheet, the configuration programs will not install the Oracle Database software on the Database configuration programs will not install the Oracle Database software on the Database Machine. Therefore it is highly recommended that you specify at least one database in the Configuration Worksheet, which can be deleted later if it is not required.

(47)

Configuring ASM Disk Groups

with Configuration Worksheet

DATA

(220G)

or SYSTEMDG (Disks 2-11)

(29G)

System Area (Disks 0-1)

Exadata Cell n Exadata Cell 2 Exadata Cell 1

RECO

(ALL ~ 309G)

Configuring ASM Disk Groups with Configuration Worksheet

The slide illustrates how ASM disk groups are configured using the Configuration Worksheet.

B d f lt ASM di k fi d i th C fi ti W k h t t i d

By default, ASM disk groups configured using the Configuration Worksheet are striped across every disk in every Exadata cell in the Database Machine. The worksheet allows you to specify a range of cells to limit the striping to all the disks those cells. However, you cannot specify a subset of disks within Exadata cells using the Configuration Worksheet. The policy of striping disk groups across all the available disks is recommended by default. Deviating from this policy is recommended only where data isolation is required; such as in

circumstances where security policies govern that data in different databases must be stored

h i ll t di k

on physically separate disks.

Regardless of how many disk groups are specified in the Configuration Worksheet, a disk group named SYSTEMDG is always created. The primary purpose of the SYSTEMDG disk group is to store the shared clusterware files; Oracle Cluster Registry (OCR) and voting disks. The SYSTEMDGdisk group occupies a grid disk that consumes approximately 29 Gigabytes on every disk except the first two disks in each cell. On the first two disks (numbered 0 and 1) in each cell, approximately 29 Gigabytes is used to store the System Area, which contains the , pp y g y y , OS image, swap space, Exadata software binaries, metric and alert repository and various other configuration and metadata files.

(48)

Configuring ASM Disk Groups with Configuration Worksheet (Continued)

Because of the layout of the System Areas and the SYSTEMDG disk group, the remaining space on every disk is equal.

As shown in the slide specifying two disk groups in the Configuration Worksheet results in a As shown in the slide, specifying two disk groups in the Configuration Worksheet results in a configuration with disk groups named DATA and RECO. The disk group names can be modified. The amount of space allocated to each disk group on each disk can also be set. The default size values are provided as a guide only and are influenced by Backup Requirements setting which is near the top of the worksheet. In this example the default sizes are used, so the DATA disk group occupies a grid disk that is allocated 220 Gigabytes on each cell disk, and the RECO disk group occupies another grid disk that is allocated the remaining space (approximately 309

Gi b t i hi h f di k ) h di k

Gigabytes assuming high performance disks) on each disk.

Specifying only 1 disk group results in a disk group named DATA which contains all of the non-system disk space on each disk (approximately 529 Gigabytes assuming high performance disks).

If you wish to specify 3 or more disk groups, the first two disk groups will be named DATA and RECO by default and you will need to supply names for the additional disk groups. You will also need to specify the per-disk space allocation for each disk group

need to specify the per disk space allocation for each disk group.

Note that the grid disks that are created to allocate disk group space on each cell disk are created in the order in which the associated disk groups appear in the Configuration Worksheet. Because of this ordering, it is recommended that you list the disk groups with the highest

performance requirements first in the Configuration Worksheet.

Note also that the values generated by the worksheet under the heading Estimated Size of DG represent the approximate space available for user data factoring in the default 2-way

mirroring policy (ASM NORMAL redundancy) that is applied during disk group creation. The Estimated Size of DGvalues also factor in the loss of one Exadata cell. For example, in the example worksheet the Estimated Size of DG value for the DATA disk group is approximately 2.6 TB, however the actual disk space allocated to the disk group is

approximately 7.9 TB (220 GB x 12 disks per cell x 3 cells) and the space available for user data is approximately 3.9 TB after 2-way mirroring is factored in.

(49)

Generating the Configuration Files

1 2

Generating the Configuration Files

After the colored fields in the Configuration Worksheet are set, click the Generate button to generate individual host names and IP addresses in the area below the colored fields. An

g

example is shown in the slide. You can customize the entries in this area of the worksheet to set specific host names and IP addresses, however it is recommended that the default

settings should be used wherever possible to minimize the risk of errors or misconfigurations. If you change one of the colored fields in the Configuration Worksheet you must click the Generate button again for the change to be reflected in the generated section.

After you are satisfied with the generated settings, click the Create Config Files button to generate the configuration files that are used to drive the Database Machine installation and configuration programs A series of files is written to a directory named according to the configuration programs. A series of files is written to a directory named according to the following convention:

c:\dbmachine_<Customer Name>\<DB Machine Prefix>\

While you can examine the entries in the generated files it is not recommended that the files are modified directly because doing so may result in errors during the execution of the

installation and configuration programs. If you make a change to the Configuration Worksheet after creating the configuration files, you must re-create the configuration files for the change to be incorporated in them

to be incorporated in them.