OpenFlow Conformance Test Program
Criterion Network Labs - Overview
Vendor neutral SDN interoperability test laboratory
Conformance, benchmarking, interoperability, proof-of-concepts and independent validation services 4thONF certified international SDN Interoperability Test Facility & only SDN test lab in India
First lab to be ISO 17025 accredited with scope as Open Flow Conformance
CNLabs is member of the Test Leadership Council (TLC) and help develop test programs at ONF We lead the Interoperability charter within the Test & Interoperability WG at ONF
Backed by strong academic and industry relationships
Led by industry veterans & staffed by team very familiar with conformance & interoperability needs 2
OpenFlow conformance certification
OpenFlow switch conformance certification assures product compliance with the OpenFlow specification
Issued by ONF and applicable to all OpenFlow enabled networking hardware and software switches
Successfully certified products can use “Open Flow
Conformant” logo on products and collaterals. Certified products will also be listed on the ONF website
Switch conformance certification program is available from the ONF for OpenFlow 1.0 specification
Switch conformance certification program for OpenFlow 1.3 specification is currently in pilot phase
Open Flow 1.0 conformance program
Open Flow Conformance for Full, Layer 2 or Layer 3 profile
depending on which of the 12 match fields are implemented by the switch vendor
All mandatory test scenarios for a given profile needs to pass for a device to be considered conformant
https://www.opennetworking.org/sdn-resources/onf-specifications/openflow-conformance
https://www.opennetworking.org/images/stories/downloads/sdn -resources/onf-specifications/openflow-test/conformance-test-spec-openflow-1.0.1.pdf
Program recently expanded to accommodate non-ONF members
Open Flow 1.3 conformance program
Open Flow 1.3 conformance test cases for a “base profile” Core Open Flow 1.3 test cases (single table only)
Details on additional profiles to be announced in future Pilot phase until Q3 2015
Four ONF labs, test equipment vendors, NEMS participate in the pilot Vendors who are ONF members can participate in the pilot program and
get their products certified at the end of the period
Significant publicity is planned by the ONF for NEMs who take their products through the pilot program for 1.3 conformance testing
Open Flow 1.3 conformance test coverage
Sections Title # of Test Cases
10 Control channel 13
40 Controller to Switch Messages 21
50 Flow table miss 5
60 Flow table matching 14
80 Flow table match prerequisites 7
90 Flow table match combinations 1
100 Flow table actions 9
130 Flow table action set 2
140 Flow table modification 19
150 Flow table errors 21
180 Counters 6
200 Protocol messages 13
210 Port structure protocol messages 1
230 Action Header Protocol Messages 3
250 Stich configuration protocol messages 2
260 Flow mod protocol messages 23
300 Multipart Reply Protocol Messages 13
310,320,330,340,380 Multipart reply section one, two, three, four, five 84
390 Packet out protocol messages 6
410 Packet in protocol messages 12
420 Flow removed protocol messages 10
430,440 Error messages 55
450 Symmetric messages 3
6
Work Flow
Vendor Develops Product
Identify preferred tool(s) for in-house testing or alternately do pre-test with the lab Vendor Contacts Test Lab
Submit completed conformance application form to test lab Sign agreement with test lab
Schedule test
Submit completed equipment and test information form to test lab Deliver Equipment
Product Passes
Vendor authorizes results for submission to ONF Lab submits results to ONF for review
ONF awards conformance certificate to the vendor Product Fails
Vendor may submit for retest
Vendor may dispute results with Lab and/or advisory board
Test labs
The ONF has chosen to allow 3rd party testing labs perform
official conformance testing. Authorized test labs sign a testing service agreement with the ONF
The ONF will maintain a list of authorized test labs along with contact information. Device manufacturers will contract
directly with authorized test labs to perform official
conformance testing. The vendor and lab will be responsible for scheduling and agreeing when to submit test results to the ONF
The lab will provide results to the ONF in a standard report format as defined by the test specification along with a
CNLabs OF conformance test program
A mutual NDA is signed along with a testing agreement
Vendor submits the required forms and provides the product with candidate image to the lab
Its quite common for test cycles to be iterative and a product might require multiple rounds of testing before actually passing the
conformance test. Hence CNLabs offers three different models for the product vendors to engage with the lab (summarized in next slide) The vendor selects the program which best meets their needs
On successful completion of a conformance test, CNLabs will share test results with vendor and ONF along with recommendation for
certification to the ONF
The ONF does an internal review and reverts back to the vendor with their decision to award conformance (typically 2 weeks)
* Conformance logos are applicable for each product family
CNLabs OF conformance test program
Option1
One test cycle
Vendor submits DUT with candidate image for conformance testing to lab
If test passes, lab submits result to ONF to seek certification. If test fails
results are sent to vendor Vendor has to submit new application
for re-test efforts
Option 2
Three test cycles
Vendor submits DUT with candidate image for conformance testing to lab
CNLabs will do up to 2 cycles of pre-testing on the candidate image
before initiating final test cycle
If test passes, lab submits result to ONF to seek certification. If test fails
results are sent to vendor Vendor has to submit new application
for subsequent re-test efforts
Option 3
Multiple test cycles
Vendor submits DUT with candidate image for conformance testing to lab
CNLabs will do up to 2 cycles of pre-testing on the candidate image before
initiating final test cycle Additional pre-test efforts would be invoiced weekly (up to max 12 weeks)
When test passes, lab submits test results to ONF (after vendor approval)
Confidentiality
All test results are kept confidential during the conformance test program , however to obtain a certification, the test results will need to be released to ONF along with the
recommendation from the lab for product conformance.
Vendor will need to provide consent for the same as part of the testing agreement
To protect the confidentiality of other customers, vendors are typically not allowed in the lab during official testing. However during the test period, they can chose to monitor the test
progress with prior request. The lab will take additional precautions as needed to protect customer interest.
Other programs from CNLabs
Offerings
SDN Application Interoperability Program SDN Switch and Controller Interoperability Proof-of-Concepts
Benchmarking
Independent test reports
Under development
SDN controller conformance program USGv6 IPv6 conformance programs
Contacting us
Email: [email protected] Phone: +91-80-41529444 +1-408-715-7754 Address:#3 Ground Floor, Explorer, ITPL, Whitefield Main Road
Whitefield, Bangalore – 560 066 India
