V A Procedure to Verify and Validate an FPGA Level Testing as Per DO-254

(1)

www.rsisinternational.org Page 48

A Procedure to Verify and Validate an FPGA Level Testing as Per DO-254

Dr. Manju Nanda

^#

, P Rajshekhar Rao

^*

#,*Aerospace Electronics & Systems Division, CSIR- National Aerospace Laboratories, Bangalore, Karnataka, India

Abstract:-This paper describes the how advanced verification like constrained random verification or directed tests which improves the system performance in FPGA level testing or hardware target level testing which outcomes safety analysis and performance based scenarios. Basically recent avionics industries facing while testing the hardware design using FPGA level mode of testing in to different feeded inputs, capturing with different techniques, which will improves the system verification on random based approach methods. Finally this paper tells about to improve the hardware design data (HDD) needs to trace the hardware requirement document (HRD) accurately using the randomization methods as carried out by constrained random verification.

Keywords: DO-254, FPGA, directed tests, verification

I. INTRODUCTION

alidation and Verification Process conducts the hardware requirements review to determine whether the captured hardware requirements satisfy the objectives of RTCA/DO- 254. This review also reduces the risk of an applicant producing hardware requirements that does not meet RTCA/DO-254 objectives or other certification criteria.

Hardware requirements review should take place after the initial completion of the hardware requirements capture process. The reviewed and approved hardware requirements documents serve as primary driving factor for design, implementation and testing processes[1].

To ensure the completeness and correctness of requirements, all requirements of the hardware will be reviewed as per RTCA/DO-254.

A. Hardware processes.

The processes involved in the hardware design lifecycle process are:

 Hardware Planning Process

 Hardware Design Processes

 Requirement Capture Process

 Conceptual Design Process

 Detailed Design Process

 Implementation Process

 Product Transition Process (Not Applicable in this case)

 Verification/Validation Process

 Integral Processes

 Process Assurance Process

 Configuration Management Process

 Certification Liaison Process

Integral processes are parallel processes that shall be adopted throughout the lifecycle of the project. While traversing in the standard hardware lifecycle processes, the transition criteria shall be met before transiting to next process in the sequence.

Means that, before start of next process the previous process’s output must be available as an input. So transition criteria shall define the minimum data/outcome of any process to start next process [2].

Hardware Planning Process

Requirement Capture Process

Conceptual Design Process

[High-Level]

Detailed Design Process [Low-Level]

Implementation Process

Planning Development Process

PLANNING REQUIREMENTS DESIGN IMPLEMENTATION

/INTEGRATION

Integral Processes

Fig. 1. Hardware Design Lifecycle Processes [4]

B. Relationship between Processes and Activities The processes established as per RTCA/DO-254 guidelines are carried out by different teams. During the planning process, the activities to be performed in every process are defined in the planning document. After review, they are brought under configuration control. The activities related to certification liaison process and system process is not brought under the configuration control process established for hardware development [3].

The system team provides the Functional requirements document and system design. The design and development team designs the IP-core as per the system requirement. The system requirements allocated to hardware are translated to high level and low level requirements. The output of hardware development process is verified by an independent verification team.

Table 1: Hardware process and activities Hardware Design Processes Planned Activities

Planning Process

Plan to meet all objectives of lifecycle process

Define Hardware Development Lifecycle Define Hardware

Development lifecycle standards

Define Program Milestones

V

(2)

www.rsisinternational.org Page 49

Hardware Design Processes Planned Activities and Reviews

Define Transition Criteria among processes Define Development &

Testing Environment and assisting tools

Define Means of Compliances of hardware design assurance objectives for certification authority

Define problem reporting and tracking resolution mechanisms

Design Process

Requirement Capture Process

Define/Document system's allocated and derived requirements for hardware item

Define/Document design constraints

Traceability to be maintained across the different levels of requirements

Conceptual Design Process

Design Hardware items architecture

Define functional blocks and its interfaces etc.,

Define Electrical interface characteristics between design blocks

Detailed Design Process

Design HDL code from hardware item requirements and conceptual design data Develop test methods and

interface data for verification of design

Implementation Process

Perform simulation and synthesis

Capture design constraints Perform target timing analysis

Integral Processes

Verification/

Validation Process

Generate compliances with respect to design requirements and Design assurance level.

Generate design deviation report (if any) and raise &

track the problem till it gets resolved

Verify the traceability of all requirements from top to bottom and vice versa Review of design artifacts Generate review checklists Generate verification

plans/methods and compliance documents Identify the acceptance test criteria

Generate test cases, execution of test cases and log test results Perform formal

simulation/testing on target C. Tasks

Hardware Verification Plan should include following details in the document.

 Methods to be adopted for Verification & Validation of hardware items independently

 Identification of evidence reports to be produced out of verification and validation independently

 Identification of analysis/test tools and equipment to be used to meet verification and validation process objectives.

 Describes the means of independence to ensure verification purpose for those objectives requiring independence.

 Identification of organizational responsibilities for implementing independent verification process

II. DETAILED DESIGN PROCESS

The Detailed Design Process produces detailed design data using the hardware item requirements and conceptual design data as the basis for the detailed design such as HDL/RTL.

Hardware Design Standard shall be followed during the Detailed Design phase of development lifecycle [4].

A. Detail Design Process Objectives

 Detailed design is developed from the hardware item requirements and conceptual design data

 Trace Data: Derived requirements are fed back to the conceptual design process or other appropriate processes.

 Trace Data: Requirement omissions or errors are provided to the appropriate processes for resolution.

B. Detail Design Process Inputs and Outputs

Fig. 2. Shows the inputs and outputs of detail design process

Detailed Design Process

Integral Process Records

INPUTS OUTPUTS

Integral Process

Hardware Design Description Hardware Block

Diagram

Hardware Coding Standard

Hardware VV Standard

HDL /RTL

Design Constraints

Fig. 2. Detail Design Process Inputs and Outputs [4]

C. Detail Design Process Activities

 Generates the detailed design data for the hardware item includes interconnection data, component data, HDL and test methods.

 Test features should be designed in, where necessary, to facilitate verification

 Constraints on the design should be identified.

 Derived requirements produced during the detailed design process should be fed back to the conceptual design or other appropriate processes.

(3)

www.rsisinternational.org Page 50

 Requirement omissions and errors discovered during the detailed design process should be provided to the appropriate process for resolution

III. TESTING PROCESS

The Mentor Graphics Questa Sim shall be used for following testing purpose and same is represented in Fig. 3.

 RTL Behavioral Simulation

 Post Synthesis Simulation

 Post Place & Route Simulation

RTL Behavioural Simulation Design VHDL

Code & Test Bench

Verification Results/Coverage

Reports

Post Synthesis Simulation Synthesis Model

of Netlist & Test Bench

Reports

Post Place/Route

Simulation Place/Route Model

of Bitstream & Test Bench

Reports

INPUTS OUTPUTS

Questa Tool

Fig. 3. : Testing Tool Usage A. Simulation and On-Target Testing

Mentor Graphics Questa Sim shall be utilized for performing the simulation activities of the designed IP-core. On target the testing is limited for performing functionality test of the IP- core.

B. Test Execution and Test Results Compilation

Test cases execution for Behavioral Simulation, Post Synthesis Simulation, Post Translate simulation and Post PnR (Place & Route) simulation will be performed and associated results will be generated.

C. Elemental Analysis Resolution

Elemental Analysis (code coverage) will be performed on RTL code and on Post Synthesis verification model at integrated levels.

IV. INTEGRAL PROCESSES

Integral processes are integral part of any hardware development lifecycle process. It is a parallel path from start of planning process till the product conformity process. The Integral processes include Validation/Verification, Configuration Management, Process Assurance and Certification Liaison processes.

A. Validation & Verification Process Objectives and Activities

Validation and Verification Process ensures that the test cases, methods, procedures, pass/fail criteria and results/coverage are reviewed to determine whether the developed test cases, methods, procedures, pass/fail criteria satisfy the objectives of RTCA/DO-254.

Verification of the implementation is the verification (e.g.

post layout simulations) of the detailed design after place and route and of the device itself.

B. Reviews and Analysis

The entry criteria to perform the review of test cases, methods, procedures, pass/fail criteria and results/coverage are release of initial version of those data items and other supporting hardware verification data will be under configuration control as appropriate. Designers will perform the review of these test cases, methods, procedures & pass/fail criteria and provide the feedback to the testing team.

The verification on device level, to assess unacceptable robustness defects through RBT (requirement based testing) to cover the normal and abnormal input conditions and operating conditions. HVVP document will be prepared to justify for level of implementation (RTL, post layout, physical device, board level) and the type of the planned verification activities (test, simulation, analysis, inspection etc.,) [5].

C. Requirements-Based Test Coverage Analysis

Mentor Graphics Reqtracer shall be used along with HDL designer to generate requirement coverage analysis. Test cases shall be generated to perform requirement test coverage for all individual requirements.

D. Elemental Analysis

Elemental Analysis (code coverage) will be performed on RTL code and on Post Synthesis verification model at integrated levels.

V. HARDWARE DESIGN METHODS

 Take the design under test (DUT) and simulate the design using the test bench provided.

 Create the required input and output text files to store the input and output values for the design respectively.

 Simulate the behavioral model using ISim tool.

A. *To compare and verify :

 Create a new test bench where the inputs are to be given manually and their corresponding outputs are obtained.

 The objective is to view the results obtained and compare them with the outputs of the given test bench.

B. *Procedure to create the test bench:

(4)

www.rsisinternational.org Page 51

 After a thorough analysis of the design, add a new source to the top module.

 Select the VHDL test bench option, where the test bench created that is the<.tb file> of the given design.

 Within the test bench, create a process for the reset, clock and/or other periodic signals.

 Within the stimulus process, force values to the inputs and give a delay depending on the clock and reset signals.

C. *Procedure to force values:

* Without constraint random verification



For std_logic_vector the syntax is as follows:

 <input_name> <= ”<value>”;

(value must have length equal to the vector length )

 For std_logic the syntax is as follows:

 <input_name> <= ‘<value>’;

 After ending the process , end the architecture, save the test bench and do the behavioral check syntax.

 Obtain the output waveforms via simulation using the ISim tool.

 Compare the results obtained from the new test bench with the results obtained from the previous test bench .

 If the results of the two test benches match, then we can conclude that the former test bench is functioning properly.

* With constraint random verification

1) Create a vhdl package file func_pkg.vhd and copy paste the code given below:

library IEEE;

use IEEE.STD_LOGIC_1164.all;

use IEEE.STD_LOGIC_arith.all;

use IEEE.STD_LOGIC_signed.all;

use ieee.numeric_std.all;

use ieee.math_real.all;--(contains the functions uniform and trunc)

package func_pkg is

shared variable seed1:integer:=844396720;

shared variable seed2:integer:=821616997;

impure function RANDOM_NUM_GEN(constant

lower_value:in integer;

constant upper_value:in integer;

constant busWidth:in integer)return integer;

-- type <new_type> is -- record

-- <type_name> : std_logic_vector( 7 downto 0);

-- <type_name> : std_logic;

-- end record;

--

-- Declare constants --

-- constant <constant_name> : time :=

<time_unit> ns;

-- constant <constant_name> : integer :=

<value;

--

-- Declare functions and procedure --

-- function <function_name> (signal <signal_name> : in

<type_declaration>) return <type_declaration>;

-- procedure <procedure_name> (<type_declaration>

<constant_name> : in <type_declaration>);

--

end func_pkg;

package body func_pkg is

---- Example 1

<type_declaration> ) return <type_declaration> is -- variable <variable_name> : <type_declaration>;

-- begin

-- <variable_name> := <signal_name> xor

<signal_name>;

-- return <variable_name>;

-- end <function_name>;

---- Example 2

<type_declaration>;

--

signal <signal_name> : in

<type_declaration> ) return <type_declaration> is -- begin

-- if (<signal_name> = '1') then -- return <signal_name>;

-- else -- return 'Z';

-- end if;

-- end <function_name>;

---- Procedure Example

-- procedure <procedure_name> (<type_declaration>

<constant_name> : in <type_declaration>) is --

-- begin --

-- end <procedure_name>;

impure function RANDOM_NUM_GEN(constant lower_value:in integer;

constant upper_value:in integer;

constant busWidth:in integer)return integer is

(5)

www.rsisinternational.org Page 52

variable result :integer;

variable tmp_real:real;

begin

assert(lower_value<(2**busWidth))

report"RANDOM_NUM_GEN():lower_value RANGE is exceeded"

severity failure;

assert(upper_value<(2**busWidth))

report"RANDOM_NUM_GEN():upper_value RANGE is exceeded"

severity failure;

uniform (seed1,seed2,tmp_real);

result:=integer( trunc ((tmp_real*real(upper_value- lower_value))+real(lower_value)));

return result;

end RANDOM_NUM_GEN;

end func_pkg;

2) Check the syntax of the code.(if there are modifications to the range)

3) To use the given code in a test bench

EXAMPLE:



Inside the architecture block (after the begin statement) a process is added. Say A,B are input buses of std_logic_vector (7 downto 0) to operate a random number for each buses ,call the function RANDOM_NUM_GEN() and then call conv_std_logic_vector() to convert it to bit vector stream.

That is:

use work.func_pkg.all; --library

………

Use ieee.std_logic_arith.all;

Testing:process Begin

A<=

conv_std_logic_vector(RANDOM_NUM_GEN(20,256,A’LE NGTH),A’length);

B<=

conv_std_logic_vector(RANDOM_NUM_GEN(50,200,B’LE NGTH),B’length);

End process;

…….

If any parameter i.e.upper_value or lower_value is exceeded ,the severity failure is indicated and the simulation is stopped.

(NOTE: the code given above will generate random values in an a periodic fashion and no value is repeated, only values between the specified range is generated (CONSTRAINED)).

To get the random numbers in a periodic manner for a given period of time use a loop with a sensitivity list.

VI. HARDWARE ITEM DATA A. Hardware Design Data

Hardware design data includes plans, standards, design, procedures, methods and results/analysis produced in project lifecycle. Hardware lifecycle data should follow some characteristics as listed below:

 Unambiguous

 Complete

 Verifiable

 Consistent

 Modifiable

 Traceable

VII. RESULTS

A. Trace Data Objective Evidence of Compliance Traceability connects between two or more elements in a project such as functions, requirements, concept, design, verification data and test results shall be prepared across all traceable data. The outputs of all traceability activities are the traceability reports showing the correlation between the project elements. Traceability reports are reviewed for correctness internally within organizations, and audited by certification authorities.

Tracing hardware lifecycle data right from requirements till the test results shall be carried out using COTS tools (ReqTracer). Hierarchy of the traceability for different artifacts is shown in Fig. 4.

System Requirements

Hardware Requirements

Conceptual Design Description

RTL/HDL and constraints

Unit level test Plan Integration level test Plan

Unit level Test Bench

Integration level Test

Bench Unit level

Test Report

Integration level Test

Report

Coverage Report

Fig.4. : Hardware traceability hierarchy VIII. CONCLUSION

At a conceptual design stage, a block-level description of the design consistent with the requirements documents, detailing all of the major components and potential sources will be included in the FPGA/CPLD design document. Any derived requirements and any errors or omissions detected in the requirements document from this process are fed back to the requirements document.

At detail design stage, detailed design work will begin on creation of the HDL for major components as well as the definition of test features and design failure mitigation. Errors or omissions detected in the earlier steps are fed back to the requirements document.

(6)

www.rsisinternational.org Page 53

During the implementation phase, the actual system is

implemented. In the case of programmable logic, the design is synthesized, place-and-route completed, and Bitstream is generated. Errors or omissions detected in the earlier steps is fed back to the requirements document.

REFERENCES

[1]. X. Y. Chen W. J. Xu X. Yang Y. W. Xia "SystemVerilog assertions and its application" China Integrated Circult vol. 16 no.

19 pp. 19-24 2007.

[2]. Y. W. Xia Verilog Digital System Design Tutorial Beijing:Press Of Beihang University 2003.

[3]. X. Jin W. Y. Ding C. Yan "Verification and testing of IP core with a RISC architecture inside" Semiconductor Technology vol. 28 no.

11 pp. 32-35 2003.

[4]. T. J. Foster D. L. Lastor P. Singh "First silicon functional validation and debug of multicore microprocessors" IEEE Trans.

Very Large Scale Integr. (VLSI) Syst. vol. 15 no. 5 pp. 495-504 May 2007.

[5]. J. Rose J. Luu C. W. Yu O. Densmore J. Goeders A. Somverville K. B. Kent P. Jamieson J. Anderson "The VTR project:

Architecture and CAD for FPGAs from Verilog to routing" Proc.

ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays pp. 77- 86 2012.

[6]. S. Tasiran K. Keutzer "Coverage metrics for functional validation of hardware designs" IEEE Design Test Comput. vol. 18 no. 4 pp.

36-45 Jul./Aug. 2001.

[7]. M. Karimibiuki K. Balston A. J. Hu A. Ivanov "Post-silicon code coverage evaluation with reduced area overhead for functional verification of SoC" Proc. IEEE Int. High Level Design Validation Test Workshop pp. 92-97 2011.