3. Design
3.3. Premium Hot-Standby System
This chapter describes the different features and specifications of a redundant Premium system.
3.3.1 Premium PAC Specifications
Primary and Standby PACs
The Primary PAC executes the application program, controls Ethernet network and In-Rack I/Os and synchronizes the Standby PAC at the beginning of each program cycle.
The Standby PAC does not run the whole program but only the first section (section 0). Moreover, it does not handle the redundant In-Rack and Ethernet I/Os but just checks the state of the Primary PAC.
In case of an anomaly, the Standby PAC takes over the control from the Primary PAC (see switchover time measurements in Performance chapter).
Primary and Standby PACs permanently exchange data in order to check the system integrity via the synchronization link.
A Premium Hot-Standby system necessarily comprises Monitored ETY modules (one in each rack). These modules handle the diagnosis of Premium CPU redundancy configuration status. This diagnosis is achieved through Sync ETY link.
Note: Sync ETY and Synchronization link are different and are not used for the same purpose.
Monitored ETY modules
As for the CPUs, the position in the rack and the firmware version of the Ethernet modules must be identical.
Note: A firmware version 4.0 or earlier is required.
The monitored ETY module allows the swap of the Ethernet services as well as the automatic permutation of the IP addresses between Primary and Standby TSX ETY.
ETY modules are linked with Ethernet switches (one switch per ETY) or via an Ethernet crossover cable. An optical connection is also possible in the case of a long distance communication.
Sync ETY link also allows handling of Ethernet I/O devices with the proper Ethernet I/O Scanning service configuration.
In order to initiate a switchover when a Sync ETY link stops operating on the Primary PAC, Ethernet I/O Scanning service must be configured on the monitored ETY module. In addition to the service activation, an I/O Scanning line must also be declared. If the service is not configured in the monitored ETY module, a switchover will not occur if a Sync ETY ceases to operate.
In case a monitored ETY module ceases to function, the CPU sends a status modification command to all the configured ETY modules populating the X-Bus and the monitored ETY module populating the Standby PAC to switch their IP addresses.
Hardware Constraints
The following table lists the only modules that can be used in a Premium Hot-Standby configuration:
Power Supply All available power supply modules Rack Non-expendable Racks only Ethernet
Communication TSX ETY4103 or TSX ETY5103 (firmware version v4.0 or earlier)
Modbus Communication
- Modbus communication module TSX SCY21601 (firmware version 2.3 or earlier) equipped with multiprotocol communication board TSX SCP114 (firmware version 1.7 or earlier) (slave or master)
-Modbus communication module TSX SCY21601 (firmware version 1.1 or earlier) (Master Modbus only)
Note: The TSX SCY 21601 associated with multiprotocol communication board TSX SCP114 allows the redundant Premium PAC systems to run as Modbus Slave or Master. This configuration allows using Modbus Masters from other suppliers.
TSX SCY 11601 module can only be used in Modbus Master.
Digital I/Os No restrictions apply Analog I/Os No restrictions apply
Software Constraints
The following constraints apply at the application level
•
The use of event tasks is not recommended. An event might be lost if it occurs just before or during the switchover.•
The use of FAST tasks handling dedicated outputs is not recommended as output status modifications might be lost during the switchover.•
The use of counting modules is not recommended. Following the frequency, some pulses might be lost during the switchover.•
The use of fronts is not recommended. They might not be accounted during the switchover.•
The use of the SAVE_PARAM function is not recommended in a CPU redundancy application. This function erases the initial value of a module parameter saved in the program code. This code is not transferred from the Primary PAC to the Standby.More generally, explicit instructions like WRITE_CMD and WRITE_PARAM must be well defined before use.
•
Initial values declared with a recorded attribute (for example DFB variables) can not be replaced with actual values: Do not use%S94 bit.•
Following inherited functions blocks can not be used:PL7_COUNTER PL7_DRUM
PL7_MONOSTABLE PL7_REGISTER_32 PL7_REGISTER_255
PL7_TOF, PL7_TON, PL7_TP PL7_3_TIMER
The use of TON, TOFF and TP blocks is not allowed in the first section
3.3.2. Premium Hot-Standby DFBs Library
The following table summarizes the different DFBs created for our application.
DFB FUNCTION
HSBY_RD Reading Command word (%SW60) hot-standby system HSBY_WR Writing Command word (%SW60) hot-standby system HSBY_ST Reading Status word (%SW61) hot-standby system ETHERNET ETY_MONITOR Monitoring ETY Ethernet Module
SYNTH_FAULT Synthesis Fault monitored elements SYNTH_OR_ETY Synthesis Fault ETY module (Logic OR) SYNTH_AND_ETY Synthesis Fault ETY module (Logic AND) SWITCHOVER SWITCH_MANG Switchover Managment
SYSTEM
SYNTHESIS
System DFBs
In order to manage the different registers of a Premium Hot-Standby system, we have created blocks that allow reading and writing registers %SW60 and %SW61.
•
HSBY_RD_P: Read the command register %SW60BOOL Run Mode Controller A BOOL Run Mode Controller B BOOL OS Versions Mismatch HSBY_RD_P
PLCB_RUN PLCA_RUN Offline_if_OS_Mismatch
•
HSBY_WR_P: Write the command register %SW60Manual Control BOOL Manual_Control_Enable Command Run Mode Controller A BOOL PLCA_RUN
Command Run Mode Controller B BOOL PLCB_RUN
Forced Command OS no Mismatch BOOL Offline_if_OS_Mismatch HSBY_WR_P
This block allows sending switch commands from the program (PLCA_RUN,
PLCB_RUN), also, in order to be able to update the CPU OS, the ETY module or the coprocessor, it allows to set the OS mismatch bit to 1 to avoid switching in offline mode.
A dedicated input allows sending switch orders for example, during maintenance activities.
•
HSBY_ST_P: Hot-Standby system, status checkThis block allows to process data from register %SW61. It gives information about each PAC role (Primary, Standby, and Offline), OS version, and so on.
BOOL Hot-Standby System active BOOL This Pac is PAC A
BOOL This Pac is PAC B BOOL This Pac is Offline BOOL This Pac is Primary BOOL This Pac is Standby
BOOL Remote state Pac undefined BOOL Remote Pac is Offiline BOOL Remote Pac is Primary BOOL Remote Pac is Standby BOOL Identical Logic Pac A et Pac B BOOL CPUs synchronized
BOOL Same CPUs OS BOOL Same Copro OS BOOL ETY version ok
BOOL Monitored ETY OS Mismatch LOGIC_OK
Ethernet link monitoring DFBs
ETY_Monitor: Ethernet module monitoring
External default, Ethernet cable unplugged BOOL BLK Fault BOOL Module Fault Module Error BOOL MOD_ERROR
Command Run Mode Controller A (T_COM_X103) IODDT COM_ETY5103 COM_ETY5103 IODDT
Monitoring Rate value INT Monitoring_Rate Enable BOOL Reading Pulse READ_STS function Pulse computer in the Standby Section BOOL Pulse
Monitoring Rate current value INT RateEt RateEt INT ETY_Monitor
The “ETY_Monitor” DFB monitors the status of the Ethernet link provided by the TSX ETY 5104 (or TSX ETY 4103). We use as inputs the BLK and MOD_ERROR
information from IODDT T_GEN_MOD.
•
BLK: external default, Ethernet cable unplugged•
MOD_ERROR: Module errorThe IODDT T_GEN_MOD is updated by the READ_STS function. This function reads the status word of a ETY module. The execution rate is controlled by the Monitoring Rate parameter configured by the user (see Chapter 5: Implementation).
BOOL BLK Fault BOOL
BOOL MOD_ERROR
IODDT COM_ETY5103 COM_ETY5103 IODDT
INT Monitoring_Rate Enable BOOL Reading Pulse READ_STS function EN ENO
BOOL Pulse %CHx.X.MOD CH
INT RateEt RateEt INT
ETY_Monitor
READ_STS
The structure of the IODDT T_GEN_MOD is detailed in the table on the next page:
ETY3_State T_GEN_MOD
NO_MOD_EXT BOOL Module absent or power down (only FIPIO extension) Internal fault: Module failure (only FIPIO extension) Faulty channel(s) (only FIPIO extension)
External fault: Terminal Block (only FIPIO extension)
Hardware or software configuration fault (only FIPIO extension) External fault: Terminal Block
Hardware or software configuration fault Module absent or power down
FIPIO extension module fault Error while reading module status Module Faults
Internal fault: Module failure Faulty channel(s)
Module error Exchange status
Status parameter read in progress Channel report
The MOD_ERROR bit is set to 1 when an ETY module ceases operation. One frequent cause is the cessation of communication of a device on the I/O Scanning which, in our case, should not initiate a switchover. Therefore, in order to filter this occurrence, we use the T_COM_X103 function to monitor the I/O Scanning status and validate the MOD_ERROR value.
When implementing a Hot-Standby system, this block is used once for each ETY module in the configuration.
Switchover Management
SYNTH_FAULT: Performs the defaults synthesis
Synthesis Fault ETY Module BOOL Faulty_ETY Synthesis Fault SCY Module BOOL Faulty_SCY
Synthesis Fault Scada BOOL Faulty_SCADA Fault Mask word WORD Fault_Mask
Fault_Synth INT Synthesis Fault Word Fault BOOL OS Versions Mismatch SYNTH_FAULT
This block aims at processing the faults that would lead to a switchover. We find in input the results of the ETY and SCY modules failure detection. “Faulty_SCADA” is an input pin in the case of the communication between the SCADA and the PAC is monitored.
This DFB also processes:
•
Battery faults %S67 = application memory card battery
%S68 = processor battery
%S75 = data storage memory card battery
•
CPU fault %S12 = CPU running
•
General In-Rack I/O fault %S119 = fault of one or several I/O modules in the rack
•
Slots 3 to 10 fault %SW160 = operating status of Premium modules installed on station 1 The faults processing is performed using the mask value set on the input pin
“Fault_Mask”. This mask allows to select which fault to take into account according the configuration and to the user’s settings.
Each fault corresponds to one bit of the “Fault_Synthesis” word:
BIT Element monitored Bit 0 Battery Fault
Bit 1 Fault CPU
Bit 2 General In-Rack I/O fault Bit 3 Fault on Slot 3 Bit 10 Fault on Slot 10
Bit 11 Ethernet Adapter(s) ETY Fault Bit 12 MODBUS Adapter(s) SCY Fault Bit 13 SCADA Fault
The result of this synthesis is saved in a word and set as an output on the
“Fault_Synth_Plc” pin. If there is at least one fault, the output pin “Fault” is set to 1.
During the implementation of the system, this block is used twice: once for the Primary PAC and once for the Standby PAC.
In order to be able to compute the status of several ETY modules, logical “OR” and
“AND” processing DFBs have been created:
BOOL FLT_ETY_1 BOOL
SWITCH_MANAG: Approve or deny a switchover
INT PRIM_DIAG Synthesis Fault word Standby INT STBY_DIAG
Switchover Number Reset BOOL SWITCH_NB_Reset SWITCH_NB UNIT Switchover request
Manual Switchover BOOL FORCE FORCE BOOL Manual Switchover
Synthesis Fault word Primary
SWITCH_MANAG
The “Switch_Manag” DFB manages and counts switchover queries. The switchover approval is computed from the Primary and Standby PACs diagnosis coming from the
“Fault_Synthesis” DFBs as seen above.
A switchover is allowed if:
•
The Standby PAC diagnosis is OK.•
More than 30s have elapsed since the previous switchover.Note: The time delay before the switchover takes place can be adjusted using variables of the DFB (Delay_Time_Before_Switchover). This delay is set to 1s by default.
The switchover counter can be reset using the input pin “Switch_N_Reset”.
For maintenance reasons, the input pin FORCE allows a manual switchover of the system.
During the implementation, this block is used only once.
Switchover Time
Remote Pac is Primary BOOL Remote_is_Primary Sw_Timer TIME Switchover Time This Pac is Primary BOOL This_is_Prima
Switch_Over_Time
The time gap during the switchover is a very important feature of the Hot-Standby system. A DFB has been defined to measure this time. The principle is based on the measurement of the time when the Primary PAC loses its Primary status and when the Standby turns Primary. This block, placed in the section 0, processes the system word %SW61 information and uses the ITCNTRL block function which allows event time measurements. The accuracy of the switchover time depends on the PAC scan time, for more accuracy, other measurement can be performed as described in the performance chapter.
3.3.3. In-Rack I/O System
This paragraph describes the management of the I/Os populating the main rack.
Inputs acquisition is performed locally by both Primary and Standby PACs, whereas the Primary PAC outputs are mirrored on the Standby PAC (provided that there is no specific action programmed in the section 0).
Redundant Digital I/Os Implementation
Digital input and output signals are connected to the PAC through an ABE7 connection block. These signals are multiplexed/de-multiplexed by a Telefast connection device as seen on the above diagram (ABE7 ACC11 for the inputs and ABE7 ACC10 for the outputs). Exceptions detected on Digital inputs cannot initiate a switchover.
The digital I/Os implementation is illustrated on the diagram below.
Digital Outputs in the section 0
As the Standby PAC executes the first section (section 0) of the application program and then applies the object image %Q received from the Primary PAC, it is
important not to modify the redundant output status in this section. A
modification of the output bits in the section 0 can lead to an inconsistent status of the outputs as they are modified twice in the same MAST task.
Digital Outputs fall-back mode
In general, the outputs fall-back mode must be similar to their current mode in order to avoid an operation discrepancy during the switchover.
Pulse Triggered Actuators
Digital output redundancy implies a distortion of the command signal. Output modules are connected in parallel of the physical output, via a connection block. The result of a command is based on the length of the pulse and the delay after which the pulse is applied on the Standby PAC.
These mechanisms have to be taken into account in order to handle In-Rack digital output redundancy.
Positive pulse trigger
As shown on the above diagram, the length of the output is longer than the Pulse Time. This does not have any impact on the device behavior.
In the case where the delay is greater than the pulse length and using an actuator with a low response time, the signal received by the actuator might be composed of 2 commands, as shown on the diagram below:
Negative pulse trigger
As shown on the above diagram, the length of the output is shorter than the Pulse Time. This does not have any impact on the device behavior unless it cannot handle a shorter command.
The following diagram presents the case where the delay is greater than the pulse time of the output signal:
Because the delay is greater than the Pulse Time, the device will not receive any command.
Analog Inputs implementation
Analog signals are connected to the PACs through a signal duplicator. For our application, we use a JMConcept TELIS9000U2 module (which replaces reference JK3000N2)
The table below describes the signal range handled by the TELIS9000U2:
Standard scales
0/1mA; 0/10mA; 4/20mA; +/-1mA; +/-10mA; +/-20mA User defined scales
from -22mA to 22mA
Standard scales
0/100mV; 0/1V; 0/5V; 1/5V; 0/10V; 2/10V; 2/10V; 0/50V 0/100V; 0/200V
User defined scales
from -110mV to 110mV; from 2V to 11V; from -200V to 220V PT100; PT1000
Ni100; Ni1000
Thermocouple J, K, R, S, T, E, B, N, W3, W5, NiMo Potentiometer from 100Ω to 100kΩ
Resistance 0/200Ω; 0/1kΩ; 0/10kΩ;
Sensor Power Supply 2 or 3 wires, 24V - 29mA max
Output 1 Current 0/20mA; 4/20mA; from 0 to 20mA Output 1 Voltage 0/10V; +/-10V - from 0 to 10V Output 2 Current 0/20mA; 4/20mA; from 0 to 20mA Output 2 Voltage 0/10V; +/-10V - from 0 to 10V Digital Output USB connector in front panel
RS485 Modbus Jbus isolated from input and output 1 Relay Output Relay: 1RT; 2RT; 3RT; 4T; 1RT & 1T
INPUTS
OUTPUTS Current (continuous)
Voltage (continuous)
Probe
As seen on the next figure the signal from the process is duplicated and wired on both PACs thanks to the JMConcept module.
Analog Outputs implementation
The implementation is handled by a low-level commutation interface, in our case a JMConcept GK3000D1 module.
The principle is to select the output coming from the Primary PAC. This selection is performed by 2 relays controlled by a PAC digital output. The management of the relays can be performed either by 1 non-redundant output or by 2 redundant outputs to increase reliability. In our architecture, we choose to manage the relays with 2 redundant digital outputs.
The following table describes the GK3000D1 interface relays logical operation:
Relay A 1 0 1 0
Relay B 0 1 1 0
Output Channel Analog Input 1 Analog Input 2 last correct channel Analog Input
Analog Input 1 4/20 mA Analog Input 2 4/20 mA
Digital Input 1 on optocoupler 30V max Digital Input 2 on optocoupler 30V max
Analog Output 4/20 mA
Digital Output RS 485 isolated from input Modbus, Jbus
Digital link allows programing of the module Digital link allows acquisition of the measurements
INPUTS
OUTPUTS
Analog Outputs controlled by non-redundant Digital Outputs
The 2 PACs analog outputs are connected to a GK3000 D1 module. This module, controlled by digital signals, routes to its output, using 2 relays, one of its 2 inputs.
In our case, the digital signals that control the GK3000D1 module are 2 digital outputs of the PACs. The main benefit of this solution is that it only uses 1 digital output to route analog outputs.
A wiring example is presented in the diagram below:
The Primary PAC must set to 1 the digital output that controls the relay in order to route its own analog output to the output channel of the commutation interface. The Standby PAC then sets to 0 the digital output that controls the other relay.
Note: This logical operating principle must be coded in section 0. The fall-back mode of the digital output module must be set to 0.
For example, according to the diagram above, if PAC_A is the Primary PAC, the digital output connected on the relay A is set to 1 while PAC_B digital output
connected to relay B is set to 0. This leads to route the analog signal A on the output of the communication interface.
Note: This solution is not used in our architecture.
Analog Output Digital Output
Analog Outputs controlled by redundant Digital Outputs
An analog output redundancy is performed, thanks to a GK3000D1 communication interface and redundant digital outputs, using 2 digital outputs per PAC. Each relay of the GK3000D1 interface is connected to a digital output.
Note: This logical operating principle must not be coded in the first section (section 0).
For example, according to the diagram above, if PAC_A is the Primary PAC, the digital output No 0 (relay A) is set to 1 and the digital output No 1 (relay B) is set to 0.
Thus, the analog signal ANA_ A is routed to the output of the communication If… Then…
PAC A is Primary Digital output number 0 is set to 1 (relay A) Digital output number 1 is set to 0 (relay B)
PAC B is Primary
Digital output number 0 is set to 0 (relay A) Digital output number 1 is set to 1 (relay B) Analog Output
Digital Output