Troubleshooting installation errors 185

Each SPARCS N4 component installer creates log files in the while it runs that record progress, file locations, file names, and errors. You can review these files for details if there are errors upon installation.

They are located in: C:\Program Files\Navis\[component name]\uninstall\Logs

Any errors that happen as you start each component are captured in the following component log files:



SPARCS N4 nodes - Review the navis-apex.log file



XPS - Review the xps_yyyymmddhhmmss file at C:\ProgramData\Navis\xps\log



Bridge daemon - Review the navis-bridged file at C:\ProgramData\Navis\bridge\logs See Review the navis-bridged.txt file (on page 185).



ECN4 - Review the navis-ecn4 file at C:\ProgramData\Navis\ecn4\logs



ECN4Web - Review the navis-ecn4web file at C:\ProgramData\Navis\ecn4web\logs



N4 Billing - Review the navis-billing file at:

Windows - C:\ProgramData\Navis\n4billing\logs

Linux - /opt/navis/application/n4billing/uninstall/Logs



Hyperic server - Review the server.log and boostrap.log in ServerHome\logs\



Hyperic agents - Review agent.log, agent.log.startup, and wrapper.log in AgentHome\logs See also, Hyperic Known Issues

Review the navis-bridged.txt file

The navis-bridged.txt file on the XPS host builds the first time you start a newly installed Navis XPS Bridge Daemon service, and continues to collect data about that service as it runs. The XPS install wizard assigns a default location, C:\ProgramData\Navis\bridged\logs. Review navis-bridged when the Bridge daemon service does not start, or when there is a crash.

The file 'berths.txt' was not found in the MISC cache

In the Bridge daemon, if a file (yard.nyd, berths.txt, Sparcs settings.txt) is not found in the MISC cache, the Bridge logs a warning and sends error response to XPS. XPS then determines how to handle the response, if necessary, for the specific operational need. Customers can ignore the warning in the Bridge daemon logs.

Not connected to N4 Center node yet...

If the Bridge daemon service will not start, with error in the Bridge log

SPARCS N4: Installation Guide 2.4 185

(C:\ProgramData\Navis\bridged\logs\navis-bridged.txt):

ERROR | pool-10-thread-1 | [HttpServerResolver:56] Not connected to N4 Center node yet...

And displays in the Cluster Services view as Connecting…

The possibilities are:

 There is an inconsistency in the n4-settings.xml file that is causing the Bridge daemon to see the Center node, but the Center node is using an invalid address to find an invalid, non-existing Bridge.

 The port for the non-connecting Bridge could be blocked by a firewall or anti-virus program.

To troubleshoot when the Bridge daemon is not connecting to the Center node:

1 For the facility with the troubled Bridge, is there an active yard?

2 Can the troubled Bridge host ping the Center node host and the Standby Center node host? Can the Center node host ping the troubled Bridge host?

a. Center node host troubled Bridge?

b. Standby Center node host troubled Bridge?

c. Troubled Bridge host Center node host?

d. Troubled Bridge Standby Center node host?

 If all of these pings are possible, move to the next step.

 If any of these pings are not possible, it is a network DNS issue. If you provided fully qualified host

186 SPARCS N4: Installation Guide 2.4

names for the SPARCS N4 components when you ran the install wizards, try using numeric IP addresses instead (on page 188). This may get around the issue that network DNS cannot resolve host names. If the numeric addresses still do not work, then your IT department must resolve why a remote server cannot be reached in a network.

3 Open JConsole at the troubled facility to observe. If there is no connection, it looks like the image below:

Open the JConsole for the Center node to observe. If there is a connection to the troubled Bridge, it looks like the image below:

The above shows the low-level tcp connection from the Bridge to the Center. The connection is suffixed with the yard coordinate.

4 Using the netstat command to view active connections,

SPARCS N4: Installation Guide 2.4 187

a. From the troubled Bridge host, check to see if the port used by activemq (the wizard-installed default is 61613) is open on the Bridge for the Center node to connect. Its State should be

‘LISTENING’.

b. From the Center node host, check to see if the Bridge host’s port is acknowledging the connection from the Center node. It’s state should be, ‘ESTABLISHED’.

If the state is "SYN_SENT", it means either the package did not reach the Bridge, or the Bridge ignored it. This indicates that the port for the non-connecting Bridge could be blocked by a firewall or anti-virus program.

Use a search engine to research "SYN_SENT" for details.

Could not create Transport

If the Bridge daemon service appears to start, but it does not appear in the Cluster Services view, with error in the Bridge log (C:\ProgramData\Navis\bridged\logs\navis-bridged.txt):

Error creating bean with name '_muleNotificationManager': FactoryBean threw exception on object creation; nested exception is

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqBroker' defined in bridgeToN4: Invocation of init method failed;

nested exception is javax.jms.JMSException: Could not create Transport.

Reason: java.io.IOException: Transport not scheme specified

It is probable that there is a typo in one of the component host names in the n4-settings.xml file (also known as the esb.settings.file) located in the Shared Configuration Folder. Check the

n4-settings.xml file and fix if necessary.

Change from fully qualified host names to numeric IP addresses

The n4-settings.xml file contains the host names or the IP addresses of the SPARCS N4 nodes and the Bridge daemon. When you run the SPARCS N4 install wizard to install the Center node, it creates this file and puts it into the Shared Configuration Folder. You specify the values it has for the SPARCS N4 nodes and the Bridge when you run the install wizards for those components. You can enter fully-qualified host names or numeric IP addresses into the install wizards. However, if your network DNS is having trouble resolving host names, one work-around could be to change the host names to numeric IP addresses. The code snippets below show the relevant components.

To change from fully qualified host names to numeric IP addresses:

1 Stop all services.

2 On the host for the Shared Configuration Folder that you specified when installing the Center node (on page 47), open up ..\conf\n4-settings.xml for editing.

3 Replace the Center Node host name and the Standby Center node host name with their respective IP addresses. In the example below, the Center node is KRMYCenter, and the Standby Center node is KRMYSTCenter.

4 Replace the Bridge host name with the Bridge host IP address. In the example below, the Bridge host is GOMBridge.

5 Save, and restart the system by starting up a cluster node first. See Sequence the start-up for the SPARCS N4 components (on page 155) for detailed start-up instructions.

188 SPARCS N4: Installation Guide 2.4

n4-settings.xml file with host names

n4-settings.xml file with IP addresses

Sometimes the n4-settings.xml file might be referred to as the "esb.settings.file". This is because the esb.settings.file variable, found in the Java tab of the Windows services for all of the SPARCS N4 nodes and the Bridge daemon, specifies the path to the n4-settings.xml file.

SPARCS N4: Installation Guide 2.4 189

Hyperic Known Issues

The following known issues are reprinted here from Hyperic documentation. For more information, see Hyperic documentation and the Hyperic user forum: http://communities.vmware.com

Out of memory error

A known problem (https://jira.hyperic.com/browse/HHQ-4640) with the Hyperic Agent 4.6, installing the agent on Windows 2008 x64 (Microsoft Windows Version 6.1.7600) causes the agent's Java Service Wrapper to crash and issue error messages. However, the agent service is successfully installed, despite the errors.

The error you see is similar to:

C:\Program Files (x86)\Hyperic\4.6.0-EE-110\hyperic-hqee-agent-4.6.0.BUILD-SNAPSHOT\bin>hq-agent.bat install

Out of memory in logging code (PPB1) Out of memory in logging code (PPB1) Logging mutex wait failed.

Logging mutex wait failed.

...

Oracle startup problem

Hyperic does not start with Oracle 10.x => jdbc error

Suggestion from Hyperic: Copy the ojdbc jar to "hq-engine/hq-server/lib"

The n4-settings.xml file is empty

You may see empty n4-settings.xml files if you have attempted to re-use a n4-settings.xml file from a previous release, rather than properly uninstalling and reinstalling with the install wizards, or properly upgrading with the install wizards.

You can resolve this by following the instructions for installation in this Installation Guide.

190 SPARCS N4: Installation Guide 2.4

Appendix

The information below is provided for your reference.

Sample SPARCS N4 Installation Worksheet ... 191

Post-configure for multiple facilities ... 194

Software and config file prerequisites for SPARCS N4 installation ... 195

Scaling the number of SPARCS N4 application hosts by TEU ... 198

Manually configure the database connection ... 199

Add context information to Oracle database reports ... 206

Database troubleshooting with unreliable networks ... 208

Shut down or restart the SPARCS N4 system ... 209

Cluster Services view information ... 211

Model Directory Structure: SPARCS N4 hosts... 219

Log Files on the SPARCS N4 nodes ... 219

Verify the memory pool and garbage collection settings ... 220

Set up a solo workstation ... 220

Find instructions for setting privileges for the server-room XPS clients 221 Compile the yard source file ... 221

Version Rollbacks ... 226

How to gather information if there is a performance problem ... 227

Subscribe to a document in the Customer Support site ... 228

Subscribe to a 'Workspace' in the Customer Support site ... 229

Sample SPARCS N4 Installation Worksheet

Use this sample installation worksheet to plan and record the details of your installation environment. It will come in handy when as you enter values into the install wizards, and set your network protocols for the SPARCS N4 system.

Do not include underscores or dashes in your host names. There is a known issue with using this and other unregistered characters (including _!~'()*) in host machine names.

The node names for the SPARCS N4 Cluster Services view must not contain spaces.

SPARCS N4 Installation Worksheet for STC (Sample Container Terminal) Test Environment

Function Host Name IP Address Node

N4StCenter101 111.11.111.101 N4-STCenter 16 GB 8 Standby none

Pair of SPARCS N4 Cluster Nodes

N4INT102 111.11.111.102 N4102-1 N4102-2

N4INT103 111.11.111.103 N4103-1 N4103-2

N4INT104 111.11.111.104 N4-104ST-1 N4-104ST-2

N4EDI105 111.11.111.105 N4105EDI-1 N4105EDI-2

N4EDI106 111.11.111.106 N4106EDI-1 N4106EDI-2

N4GATE108 111.11.111.108 N4108GATE-1 N4108GATE-2

16 GB 8 Main Main gate

clerks

192 SPARCS N4: Installation Guide 2.4

Function Host Name IP Address Node

N4CUST110 111.11.111.110 N4110CUST-1 N4110CUST-2

Function Host Name IP Address Node

N4ECBKFAC1 111.11.111.118 EC-STFAC1 12 GB 8 Standby none

ECN4 and

If you have multiple facilities in your complex, you do not need more than one Center node or Standby Center node. Instead, you only need to configure the n4-settings.xml file (located in the conf directory in the Shared Configuration Folder) to have multiple sections in the bridge component, as in the example below.

It is possible to configure the AMQ port numbers to something other than 61613.

194 SPARCS N4: Installation Guide 2.4

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE settings PUBLIC "-//Navis LLC//ecn4//settings" "settings.dtd" >

</component>

</component>

</component>

</component>

</settings>

Software and config file prerequisites for SPARCS N4 installation

The tables below describe the software installers and and configuration files provided with SPARCS N4 2.4 in a Windows OS. Depending on the scope of your license, you need some or all these files.

It is essential for the clocks in the SPARCS N4 system (SPARCS N4 hosts, XPS, ECN4, dispatching clients, etc.) to be time synchronous. If you do not enforce time synchronization across your hosts, SPARCS N4 will not function properly. Navis recommends that you use Network Time Protocol (NTP), but any method that keeps all the host clocks within one second of each other should suffice.

The installation instructions assume you have a time server in place to assure that all server clocks are in sync.

SPARCS N4 2.4 runs on 64-bit systems. All new and existing customers installing version 2.4 are required to install with 64-bit hardware. SPARCS N4 versions above 2.1 are not intended to run in a 32-bit environment, and Navis does not test any part of 2.4 in a 32-bit environment.

Download the required executables and zipped archives from the source location, and put them into a

SPARCS N4: Installation Guide 2.4 195

temporary install subdirectory.

Subscribe to the SPARCS N4 Apps Workspace to be notified of release schedules and newly posted files. See instructions for how to subscribe (on page 228) in the Appendix.

Install Wizards and Configuration Files for SPARCS N4 2.4 and Licensable Components

File Name Purpose/Contents Source

sparcsn4-install.exe Installer for the SPARCS N4 application servers on four or six nodes.

The licenses for the Tomcat application server and the JDK that install with this executable are included with your Navis SPARCS N4 license.

Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

xps-install.exe Installer for the XPS application for planning and control. Includes the Bridge daemon.

Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

Sparcs N4 Client.zip The XPS Client application (Sparcs N4 Client.exe), along with the application extension executables and files needed to run the XPS client.

Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

ecn4-install.exe Installer for ECN4. Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

ecn4web-install.exe Installer for ECN4Web. Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

n4billing.exe Installer for N4 Billing. (One node on a dedicated host.)

(Conditional - Necessary only if you have licensed N4 Billing)

Customer Support Site - SPARCS N4 Apps workspace. Bundled with the .zip archive for the release.

JDBC driver A software component that lets a Java application interact with a database

The specific file depends on what type of database you are running (MySQL, SQL Server, Oracle, or Oracle RAC). See JDBC drivers for SPARCS N4 (on page 197).

xps.zip XPS server files, including the contents of the data folder (including license.xml, SPARCS Settings.txt, yard.nyd), the database folder, and the settings.xml file

Provided by your Navis Professional Services representative.

settings.xml Settings that define the configuration of XPS against SPARCS N4, such as data sizes and log levels.

See xps.zip

196 SPARCS N4: Installation Guide 2.4

File Name Purpose/Contents Source

SPARCS_Settings.txt Settings that define the many of the operational behaviors of XPS.

See xps.zip

yard.nyd Compiled XPS yard configuration See xps.zip. If you have a services engagement, your Navis

representative creates this file on site during installation.

You can generate a new yard.txt file by decompiling the current yard.nyd file in XPS. See Decompile a yard file in the SPARCS N4

Administrator's Guide for instructions.

The yard code in yard.nyd needs to exactly match the yard code you set in the basic topology (on page 134). Otherwise, XPS does not connect to the Bridge daemon, and will log an error (which you can see in C:ProgramData\Navis\xps\logs on the XPS server).

JDBC drivers for SPARCS N4

The JDBC driver is a software component that lets a Java application interact with a database. The type of JDBC driver you have is bound to the type of database you have (Oracle, Oracle RAC, SQL Server, or MySQL). The drivers are the same whether you are installing on a Windows or Linux operating system.

The table below describes the JDBC drivers required for currently supported databases.

Navis includes only Oracle and SQL Server JDBC drivers with the SPARCS N4 component installation wizards.

If you are using a MySQL database, you must contact your Enterprise MySQL vendor for the correct jar. After you run the SPARCS N4 install wizard on each SPARCS N4 host, you must drop the driver into:

For Window environments: the C:\\Program Files\Navis\lib directory.

For Linux environments: the opt/navis/application/sparcsn4/lib directory.

Configuring a system data source is recommended for SPARCS N4 and N4 Billing.

SPARCS N4 2.4 JDBC Drivers, Relative Log-ins, and Example URLs for Each Database Type

The driver class name may change in SPARCS N4 from version to version. Check to make sure yours is up to date.

SPARCS N4: Installation Guide 2.4 197

Type of Database

Driver Class Name Username and

You must replace <SID> (the Oracle Service ID) with your internal SID.

=(ADDRESS = (PROTOCOL = TCP)(HOST =

<db01.example.com>)(PORT =

1521))(ADDRESS = (PROTOCOL = TCP)(HOST

= <db02.example.com>)(PORT =

1521))(LOAD_BALANCE = yes)(CONNECT_DATA

=(SERVER = DEDICATED)(SERVICE_NAME =

<Service Name>)))

You must replace <Service Name> (the Oracle Service ID) with your internal Service Name.

MySQL com.mysql.jdbc.Dri

Scaling the number of SPARCS N4 application hosts by TEU

The application layer consists of four or more hosts running Windows Server 2008 R2 or Red Hat Enterprise Linux 5. Each host typically runs two dedicated SPARCS N4 application nodes to maximize memory use. You can easily scale the cluster by adding more dedicated SPARCS N4 application hosts. For specific recommendations for each host, see SPARCS N4 Application Hosts.

The diagram below shows a rough estimate of the number of applications nodes for different terminal sizes (measured in annual container volume in twenty-foot equivalent units (TEU)). Use this as a guideline for understanding the number of hosts you should dedicate to the SPARCS N4 application.

198 SPARCS N4: Installation Guide 2.4

Application Host Scaling for SPARCS N4 Deployment

**For extra large sites, SPARCS N4 can run on x86_64 CPUs (AMD64).

Sites with high EDI volume (more than 500,000 annual TEU) should configure SPARCS N4 to process EDI on a separate host.

Manually configure the database connection

For each node of the SPARCS N4 Webapp using the SPARCS N4 application server, SPARCS N4 must establish a link between the JDBC driver and the database. The SPARCS N4 install wizards configure this for you, based on the database type you select.

However, if you must ever manually configure this information, the instructions below include a table with specific values to use, depending on the type of database and the driver you are using.

Use separate users for each data source.

To manually configure the database connection:

1 In the ProgamData\Navis directory, locate two files that specify the configuration of each node:

 C:\ProgramData\Navis\node1\conf\Catalina\localhost\apex.xml

 C:\ProgramData\Navis\node2\conf\Catalina\localhost\apex.xml

In a typical installation, you configure four nodes to serve as application servers in the cluster.

This is based on the size of the terminal and the RAM capacity you have available. If the terminal is particularly large, you install and add more nodes--two nodes on each additional SPARCS N4

In document SPARCS.N4_2.4_InstallGd (Page 185-200)