Input/Output Interface

Format and Timing

The input/output interface must parse the data transmitted on the input bus, load the appropriate frames into the DDR-2 memory for use by the two encoder instances, and output the bitstream data on the output bus. In addition, it must be able to pass on the frames to be encoded to the output bus, as well as any additional frames transmitted on the input bus.

Data is transmitted to, and from, the FPGA in meta-frames. The meta frame format is used to ensure the FPGA is compatible with the other devices in the system. Each meta-frame consists of 625 lines of 3200 bytes. An 8 byte horizontal synchronisation period occurs at the end of each line. A 5 line vertical synchronisation period occurs at the end of each frame. The first 128 bytes of each input meta-frame contains the information required for encoder control.

This is then followed by up-to 3 video frames, the first of which is the frame to

Figure 4.10: Example input meta frame

be encoded. An example input meta-frame is shown in Figure 4.10.

In the example shown in Figure 4.10, the 720p frame must also be passed on to the output bus as well as being encoded. Whether the frame to be encoded must also be passed on is dependent on the combinations of frames contained within the meta-frame. A complete list of the input frame combinations which had to be supported is given in table 4.4.

The requirement to pass on frames complicates the output meta-frame format.

There are no spare resources available to store the additional pass-on frames in the FPGA. The DDR-2 memory cannot be used as there is no spare memory bandwidth available. Therefore, the frames must be passed on with minimal delay. As a consequence the bit-stream data must be transmitted in the empty

Encoded Frame Size Encoded Frame Passed On ? Pass On Frame Sizes

space available within each output meta frame. For meta-frames which require the encoded frame to be passed on, the bitstream is located at the end of each meta-frame, as indicated in Figure 4.11. For meta frames which do not require the encoded frame to be passed on, the bitstream data starts at the end of a meta-frame and, if necessary, continues at the start of the next meta-frame as indicated in Figure 4.12.

From Figure 4.11, it can seen that the encoding latency with this type of meta-frame is over 30 milliseconds. The bitstream is transmitted on the output bus over 30 milliseconds after the corresponding input frame has been transmitted.

This is unavoidable given the time needed to encode larger frame sizes and the inability to store the pass on frames. The latency for meta-frames where the frame to be encoded is not passed on is much less, as shown in Figure 4.12. This is a result of the smaller frame sizes which have to be encoded and the larger frame sizes used for pass on frames in this case. As a result the bitstream can be outputted in the same meta-frame as the corresponding input frame.

Figure 4.11: Timing relationships between the input and output meta frames when the encoded frame is passed on to the output bus

Figure 4.12: Timing relationships between the input and output meta frames when the encoded frame is not passed on to the output bus

Figure 4.13: Structure of I/O interface

Input/Output Interface Design

The structure of the input/output interface is shown in Figure 4.13. It consists of three main components. The input/output controller communicates with the two external buses. It parses the header data from the input bus, writes frame data to the input frame memory buffer, and reads the bitstream data from the bitstream memory buffer and puts it on the output bus. The input frame memory controller schedules the transfer of the input frame from the input memory buffer to the external memory. The bitstream memory controller schedules the transfer of the bitstream data to the bitstream memory buffer. The memory buffers are used to transfer the bitstream and input frame data efficiently between the capture and memory clock domains.

Similarly to the design described in section 4.1, input images are received in raster scan order. For this design, the input image memory format shown in Figure 4.4 is not used. Instead, the simple macroblock ordered format shown in Figure 4.3 was used. The format shown in Figure 4.4 was not used because the Altera DDR-2 controller used only supports a burst length of 4. If an Avalon

Logic Elements 63929 out of 68416 Embedded Rams (M4K) 240 out of 250 Maximum Memory Clock Frequency 127.8 MHz Maximum Encoder Clock Frequency 66.77 MHz

Table 4.5: Usage of Cyclone-2 resources and maximum operating frequency master requires a larger burst size than that supported by the DDR-2 controller, the Avalon switch fabric instantiates additional logic to support it. It was found that this additional logic could not operate at 125 megahertz. Consequently, there would have been no benefit in using the memory format shown in Figure 4.4 in this system.

4.2.3 Results

The total resources used by the system are shown in table 4.5. From it it can be seen that, even with the architecture used, the memory clock frequency require-ment is met with a minimal timing margin. Similarly to the system presented in section 4.1 the most heavily used resource in the design is the embedded RAMs.

Although in this system the LUT utilisation is only slightly less than the embed-ded RAM utilisation.

4.3 Summary

The implementation of two FPGA video encoding systems has been described.

Both systems used readily available intellectual property provided by the FPGA manufacturers to provide the video encoder with access to an external memory. In both cases this proved problematic. The effective bandwidth provided by the IP in each case only being suitable due to custom modifications to the encoding system.

In the case of the system presented in section 4.1, a custom memory format was

used for input frames. In the case of the system presented in section 4.2 the overall system was specifically designed around the memory/bus architecture’s limitations. This is not a surprising result, given the significant amount of data an encoder needs to write to, and read from, external memory.

Another common issue in the two systems is the high utilisation of FPGA embedded rams. This is unfortunate. The easiest method to reduce an encoder’s memory bandwidth requirements is to store more data in the embedded RAMs available on the FPGA. The results presented here suggest that in many cases this will increase the size of the FPGA required to implement the encoding function.

Thus, making the integration of the encoder into a complete system easier will, in many cases, increase the size of the FPGA required to implement the system.

Chapter 5

FPGA H.264 Video Encoder