1 Project Description

The class will be assigned to teams to do the various components of an industry-strength system design. The intent of the project is to do a HW/SW co-design of an embedded SoC. The design is a low-power, real-time SoC implementation of the public domain Tiny YOLO visual object detection system. The platform will consist of an ARM processor, memory components, custom hardware accelerators, etc. The ARM processor was chosen because it is used for a majority of the components developed for the target market. The teams will be given a market requirements document (MRD) and will then generate a product requirements document (PRD) which will be used to complete the implementation of the Tiny YOLO design.

The project activities include:

• Develop a PRD for the HW/SW implementation of the low-power SOC

• Profile the public domain software implementation to determine bottlenecks and optimize software performance (example in Lab 1)

• Partition the software into components which will run on the ARM processor and on the hardware accelerator (example in Lab 2)

• Convert the hardware code to fixed point (example in Lab 1)

• Verify that the code maintains its target specification (example in Lab 1 and Lab 2)

• Synthesize the hardware accelerator to RTL (example in Lab3)

• Prototype the hardware accelerator on the FPGA

• Establish proper communication between the software and hardware (example in Lab 2 and the board tutorial below)

• Co-verify the prototype HW/SW implementation (example in Lab 2 and board tutorial below)

• Develop a high-level performance and power model of the SoC

• Estimate the timing, area and power consumption and validate product metrics described in the product MRD

The main tasks for the final part of the project are as follows:

• Integrate, optimize and prototype the Tiny YOLO SoC on the ARM+FPGA board. You can come up with functions other than the GEMM to accelerate in hardware. Please refer to profiling results and develop your own hardware accelerator, if you want to do so.

• Interface the hardware accelerator to the ARM processor, and develop and optimize the communication architecture.

• Implement the remaining functionality on the ARM processor and interface the ARM software to the hardware accelerator prototype in the FPGA.

2 Market Requirements Document (MRD)

We are targeting an ASIC implementation of the Tiny YOLO SoC in 45nm or 32/28nm technology. The marketing requirements are derived from the Tiny TOLO specification and Tincy YOLO reference implementation by Xilinx.

Specifically, the cost metrics for the project are:

• Performance

- Real-time operation with at least 10 frames per second (fps)

- Object detection accuracy of at least 50% mAP

• Area

- 0.5mm² for accelerators

- On board memory TBD

• Power

- Additional system power for accelerators < 8mW

3 Product Requirements Document (PRD)

Some PRD examples:

• PRD Overview

• IP Camera

4 Prototyping Board

We are using an ARM- and FPGA-based development board for prototyping our ASIC SoC design:

• Ultra96 development board tutorial