HP Service Virtualization

HP Service Virtualization

Fixed Length Protocol Virtualization

SV Training

September 2014

Fixed Length Protocol Virtualization

• Technology Description

• Use Cases

• Supported Message Structures

• SV Service Description Format

• Virtual Service Summary

• Exercise

Technology Description

• TCP/IP layer is used directly

• Message is introduced by a field describing its length

• The same message type has usually the same length

• No delimiters are present

• Each field is described by offset and length pair

• Usually used as a legacy protocol or in embedded devices

Use Case

Use Case

How to Recognize Fixed Length Message (While Viewing in Hex Editor)

Use Cases

• Unsupported characters are present in the beginning of the file (underlined)

• Lots of spaces (hex value 0x20)

• Lots of zeros (hex value 0x30)

• No visible data value separation

• Can read ASCII strings and numbers

• Free Hex Editor

http://frhed.sourceforge.net

00000000 00 00 00 14 53 51 52 54 20 20 30 30 30 30 30 30 ....SQRT 000000

00000010 30 30 36 34 0064

How to Recognize Fixed Length Message (From Provided Description)

Use Cases

• Offset and Length columns

• Format column

• Message Length field

• Alphanumeric and Numeric fields are mostly present

• Numeric fields are usually left zero padded to defined length

• Alphanumeric fields are usually right space padded to defined length

• Message Type field can be present

• All messages have a message length field on the same offset position

Name Offset Length Type Description Value

Length 0 4 Binary (network byte order) Message length

Type 4 6 String Message type SQRT

Number 10 10 Numeric Number

Supported Message Structures

• No occurring structures are supported (arrays of some type, defined field occurrence)

• Message can be shorter than defined, unspecified fields are set to “not present” value in SV

• Supported field data types:

Format SV Format Note

ASCII, Alphanumeric, String String Just ASCII chars are valid

UTF-8, Unicode, … Binary Any non-ASCII charset

Numeric Numeric Numbers in ASCII (“000010” or “-000020”)

Decimal Decimal Numbers in ASCII + decimal separator (“0005.4” or “-005.4”) Fixed Decimal String Fixed decimal places (i.e. financial applications: “00100.00”) Integer, Short

(big-endian, network byte order) IntBE

Supported byte length is 4 bytes or less.

More info: http://en.wikipedia.org/wiki/Endianness Integer, Short

(little-endian, Intel x86) IntLE

Binary, other non-specified Binary If some of parent formats is not working, use Binary instead.

SV Service Description Format (structure)

• Tab-separated table in *.txt file. Can be prepared in Excel and copy & pasted to *.txt file.

• Following table format is required (at any order), just bold fields are mandatory:

Name Offset Length Type Description Default Value Attributes

Length 0 4 IntBE Message Length 1

Type 4 4 String Message Type 2

Request Type REQ

Response Type RESP

MsgID 8 6 Numeric Message ID

HasError 14 1 Numeric Has Error Flag 0

No Error 0

Error 1

Data 15 30 String Message Payload

Ev al uat io n Di re cti on

Enumeration of

possible values

for the parent field

Enumeration of

matching message

types

SV Service Description Format (file name)

• Defined service description is used for request and response calls but if the file name matches one of the following patterns:

Request File Name Pattern Response File Name Pattern

*_req.txt *_resp.txt

*_request.txt *_response.txt

*_rq.txt *_rs.txt

*-req.txt *-resp.txt

*-request.txt *-response.txt

*-rq.txt *-rs.txt

*.req.txt *.resp.txt

*.request.txt *.response.txt

*.rq.txt *.rs.txt

Virtual Service Summary

• Multiple responses are supported

• TCP/IP protocol is only supported

• Multiple service description files can be used to create a virtual service

• Simulation from the scratch is not supported, learning is needed

• Just one virtual service can be deployed on a Fixed-Length TCP Agent

• Occurring structures are not supported

• Variable message structures (defined by a precedent field) are not supported

• Cobol strings are not supported (in a readable way)

Summary

• Introduced simple to use fixed-length protocol in SV

• No code changes needed for introducing customer-specific fixed-length protocol

• Easy maintenance of the protocol in customer environment

• Service description format can be used for further extensions (like Text Based

protocol over TCP)

Exercise

Fixed Length Protocol Virtualization Exercise

• Use Case

• Application Analysis & Packet Dumps

• Service Description Creation

• Virtualization

• Troubleshooting

• Summary

Use Case – Demo Application

• Simple calculator that supports SQRT and PLUS operations

• Responses are either RESULT or ERROR

Demo Application Package 

Client Application (connecting to localhost:10000)

Real Service (listening on localhost:10000)

Application Analysis &

Packet Dumps

• Open *.pcap file in Wireshark

• Select the first TCP/IP packet with payload data

• Check the packet structure (the TCP part is the last part)

• Find the TCP segment data

• Check payload of the data

• Use the protocol recognition steps from the Lesson 5: Fixed Length Protocol Virtualization

 Example *.pcap file

SQRT Operation

Service Description Creation

Name Offset Length Type Description Default Value Data Attributes

Length 0 4 IntBE Message length 1

Type 4 6 String Message type 2

SQRT

Number 10 10 Numeric Number

Service Description File for SV 

Example – Excel with Service Descriptions 

PLUS Operation

Service Description Creation

Name Offset Length Type Description Default Value Data Attributes

Length 0 4 IntBE Message length 1

Type 4 6 String Message type 2

PLUS

Base 10 2 Numeric Base

Binary 2

Octal 8

Decimal 10

Hexadecimal 16

Number1 12 10 String Number 1

Number2 22 10 String Number 2

Service Description File for SV 

RESULT Operation

Service Description Creation

Name Offset Length Type Description Default Value Data Attributes

Length 0 4 IntBE Message length 1

Type 4 6 String Message type 2

RESULT

Base 10 2 Numeric Base

Binary 2

Octal 8

Decimal 10

Hexadecimal 16

Result 12 10 String Result

Service Description File for SV 

ERROR Operation

Service Description Creation

Name Offset Length Type Description Default Value Data Attributes

Length 0 4 IntBE Message length 1

Type 4 6 String Message type 2

ERROR

ErrorCode 10 4 Numeric Error code

ErrorMessage 14 1000 String Error message

Output error_rs.txt file opened in Notepad++ 

Service Description File for SV 

Agent Configuration

Virtualization

Virtual Service Port Number

Real Service Hostname And Port Number

(our example:

“localhost” and “10000”)

Creating a Virtual Service & Learning

Virtualization

2.

1.

3.

4.

After Learning

Virtualization

Protocol

operations

Data Model Refinement – Message Length Recognition

Virtualization

Note the length

Data Model Refinement – Fallback to the Error Message Type

Virtualization

Create a new rule and set the

message type to

“error_rs”

Data Model Refinement – Default Rule Update

Virtualization

Set all fields meaningfully, especially the length field must have the same value as the learned message

When you hit

the default rule,

data model

accuracy will be

decreases

Troubleshooting

• The following namespaces can be set to TRACE log level to debug the protocol:

– HP.SV.MessageProcessing.GenericProtocol

– HP.SV.TcpAgent.FixedAndText

– HP.SV.MessageProcessing.ProtocolBuilder.GenericProtocolServiceDescriptionBuilder

• The protocol uses XML message format internally in SV

– The message logger also displays XML format, so don’t be surprised

• Service description format is tab-separated, so check the format first when you get errors while creating a virtual service

• Fixed Length TCP Agent can be used just by one virtual service at the same time

Summary

• The same approach applicable also for Text Based protocol over TCP extension in HP SV Extensibility SDK

• No empty line should be present in a service description file

• Use Designer/Server logs for debugging

• Analyze messages in the message logger in Designer

Thank you!