Resource-Efficiency Testing
Introduction to Resource‐
Efficiency Testing
•
Introduction
•
Metrics Acquisition - Windows Performance Monitor
•
OPC Server test plan model
•
OPC Client test plan model
•
Network Failure Simulation Batch File
•
Test Reports
Resource-Efficiency Testing
Contents
Resource-Efficiency Testing
Introduction
(1 of 3)
What is it?
36-hour stress-test to verify no resource leaks exist
in your product, e.g. CPU, memory, threads etc.
More info?
Windows Performance Monitor is used in the lab; the
performance counter details are available online:
www.opcfoundation.org/ua/ctt/help/testcases/server/all/testcases.html#Efficiency technet.microsoft.com/en-us/library/cc768048.aspx
What is tested?
UA Servers/Clients, OPC Classic Servers/Clients
Easy to test?
Yes, but several other OPC products are required to
generate a load on the product being tested.
Resource-Efficiency Testing
Introduction
(2 of 3)
How it works:
UA and Classic products are tested the same way:
Servers
:
- multiple device (PLC) connections; 1 is intermittent
- 5 or more OPC Client connections, 1 is intermittent
Clients
:
- 5 or more OPC Server connections, 1 is intermittent
Load is placed on the product for stress-testing.
Resources are measured over 36-hours.
Resource-Efficiency Testing
Introduction
(3 of 3)
How do I pass?
No resource leaks or performance degradation.
Am I Certified?
No. It means that your product is a good candidate
for certification testing in a Test Lab.
Lab tools used?
Windows Performance Monitor, network fault
simulation BAT file, also:
Servers
: 2 or more PLCs; 5 or more OPC Clients.
Clients
: 5 or more OPC Servers.
Test when?
As frequently as possible, and on any official build
whether it is alpha, beta, or release etc.
Resource-Efficiency Testing
Windows Performance Monitor
(1 of 6)
Configure
the
Windows
Performance
monitor
to
record
the
performance
counters
for
36
‐
hour
duration
•
Click
Start,
click
in
the
Search
box,
type
perfmon.msc
,
and
then
press
ENTER
.
Perfmon.msc
Resource-Efficiency Testing
Windows Performance Monitor
(2 of 6)
7
Create
a
Data
Collector
Set
to
store
the
data
values
recorded
for
the
36
‐
hour
test.
Specify the directory
you want to store the
file
Create a new Data
collector set to
record the data
Click finish to create
the Data collector
Resource-Efficiency Testing
Windows Performance Monitor
(4 of 6)
Configure
the
Process
,
Processor,
Threads,
Handles,
and
other
counters
for
the
instances
of
your
product:
Step 4: Choose the
performance counters
from the list
Step 1: Double
click “Performance
Counter ” to select
the counters
Step 2: Click “Add”
to specify the
resources to
monitor
Step 5: Select the log
format. Test lab uses
comma separated
format to record the
values for the counters
Resource-Efficiency Testing
Windows Performance Monitor
(5 of 6)
9
Setup
a
36
‐
hour
stop
condition
in
the
Data
Collector
Set
properties
to
stop
recording:
Setup the
duration to 36
hours
Click OK to apply the
Resource-Efficiency Testing
Windows Performance Monitor
(6 of 6)
Setup
the
stop
condition
in
the
Data
Collector
Set
properties
to
stop
recording
the
data
values
after
36
hours.
Measurements
can
be
displayed
in
a
chart,
as
well
as
being
logged
to
a
file
(as
previously
configured).
Click
the
Triangle
icon
to
start
Resource-Efficiency Testing
Test Plan Model – Server
(1 of 2)
11
Device
1
(e.g.
PLC)
Device
2
(e.g.
PLC)
Remote
PC1
Classic/UA Clientwith max no of Tags
Supported
•
An
OPC
Server
is
configured
with
2
devices
(PLCs)
•
Typically,
5
Clients
are
connected,
with
approx.
1000+
items
each
•
Make
1
Device
and
1
Client
go
online/offline
every
2
‐
minutes
Remote
PC2
Classic/UA Client
with max no of Tags
Supported
Remote
PC3
Classic/UA Client
with max no of Tags
Supported
Remote
PC4
Classic/UA Client
with max no of Tags
Supported
Remote
PC5
Classic/UA Client
with max no of Tags
Supported
Local Client
(if available)
UA/Classic
Server
Resource-Efficiency Testing
Test Plan Model – Server
(1 of 2)
Remote
PC1
Classic/UA Server
with max no of Tags
Supported
•
An
OPC
Client
is
connected
to
5
or
more
OPC
Servers,
each
approx.
1000+
items
•
Make
1
Server
go
online/offline
every
2
‐
minutes
Remote
PC2
Classic/UA Server
with max no of Tags
Supported
Remote
PC3
Classic/UA Server
with max no of Tags
Supported
Remote
PC4
Classic/UA Server
with max no of Tags
Supported
Remote
PC5
Classic/UA Server
with max no of Tags
Supported
Local Server
(if available)
UA/Classic
Client
@echo off
SETDTIME=120
CLS
ECHO Delay time is: %DTIME%
ECHO Network Connection Enabler/Disabler started at: %date% %time%
:start
ECHO Disabling network adapter at %time%
netshinterface set interface "Local Area Connection" DISABLED
timeout/t %DTIME%
ECHO Enabling network adapter at %date% %time%
netshinterface set interface "Local Area Connection" ENABLED
choice/c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d y
ECHO %ERRORLEVEL%
IF ERRORLEVEL 2 GOTO end
IF ERRORLEVEL 1 GOTO start :end
ECHO.
ECHO Network adapter recycle complete at %date% %time%
@echo off
SETDTIME=120
CLS
ECHO Delay time is: %DTIME%
ECHO Network Connection Enabler/Disabler started at: %date% %time%
:start
ECHO Disabling network adapter at %time%
netshinterface set interface "Local Area Connection" DISABLED
timeout/t %DTIME%
ECHO Enabling network adapter at %date% %time%
netshinterface set interface "Local Area Connection" ENABLED
choice/c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d y
ECHO %ERRORLEVEL%
IF ERRORLEVEL 2 GOTO end
IF ERRORLEVEL 1 GOTO start :end
ECHO.
ECHO Network adapter recycle complete at %date% %time%
Resource-Efficiency Testing
Batch File
13
