Germany

This work focuses on the time-predictable execution of Deep Neural Networks (DNNs) accelerated on FPGA System-on-Chips (SoCs). The modern DPU accelerator by Xilinx is considered. An extensive profiling campaign targeting the Zynq Ultrascale+ platform has been performed to study the execution behavior of the DPU when accelerating a set of state-of-the-art DNNs for Advanced Driver Assistance Systems (ADAS). Based on the profiling, an execution model is proposed and then used to derive a response-time analysis. A custom FPGA module named DICTAT is also proposed to improve the predictability of the acceleration of DNNs and tighten the analytical bounds. A rich set of experimental results based on both analytical bounds and measurements from the target platform is finally presented to assess the effectiveness and the performance of the proposed approach on ADAS applications.

IEEE RTSS 2021

Time-Predictable Acceleration of Deep Neural Networks on FPGA-based SoC

fpga autonomous syste

dnn fpga acceleration

time predictable dnn acceleration

field programmable gate arrays

system-on-chip

timing

analytical models

autonomous systems

advanced driver assistance systems

deep learning

real-time systems

 **RTSS 2021** has successfully concluded. Hope you have enjoyed it, and look forward to seeing you next year in person.
    Announcement for the Awards:
 * Best Paper Award goes to: Monte Carlo Response-Time Analysis, Sergey Bozhko, Georg von der Brüggen, and Björn Brandenburg
 * Best Student Paper Award goes to: Automatic Energy-Hotspot Detection and Elimination in Real-Time Deeply Embedded Systems, Mohsen Shekarisaz, Lothar Thiele, and Mehdi Kargahi.
 * Best Solution Award for Industry Session goes to: Toward Real-Time Guaranteed Scheduling for Autonomous Driving Systems. Jinghao Sun, Tianyi Wang, Kailu Duan, Bin Lu, Jiankang Ren, Zhishan Guo, and Guozhen Tan.



The IEEE Real-Time Systems Symposium (RTSS) is the premier conference in the field of real-time systems, and is a venue for researchers and practitioners to showcase innovations covering all aspects of real-time systems, including theory, design, analysis, implementation, evaluation, and experience. 

technical paper

Industrial Challenge: Opening, Introduction, and Overview of Solutions

In this paper we address the timing verification for processing graphs used to implement autonomous driving systems by the company Perceptin. We demonstrate how to extend our previous results on end-to-end timing analysis in terms of maximum reaction time and maximum data age for cause-effect chains in automotive systems for the posed challenge.

End-To-End Processing Chain Analysis

The 2021 RTSS Industry challenge considers timing constraints in autonomous driving systems. One real-time requirement is that the differences of timestamps among all source data of an output must be upper-bounded by a pre-defined value, to make sure that data from different sources are correctly fused, e.g, object detection by using different sensing data. In this paper, we formally describe the challenge system as well as the requirements. In particular, we present timing analysis to bound the maximum differences of timestamps among all sources for an output. We show that an output may possibly be caused by two different data from the same source, when fork-join structure exists. This result suggests that data fusion after different long pipelined processing procedure must be avoided to meet required timing constraints.

Analysis of Time difference among Sensing Data in Autonomous Driving Systems

Given the problem proposed in RTSS 2021 Industry Challenge, we formally define a system model where dependent tasks are organized in a direct acyclic graph structure and communicate with an unblocking manner. For such a system, we present solutions to estimate three time latencies: maximum reaction time, maximum data age, and maximum difference among the timestamps of ”source” sensor data. One major contribution made in this paper is that the clock shifts among different processing units have been taken into account during the estimations without a mandatory schedule simulation process. By doing this, fairly tight upper bounds can be calculated for all three latencies.

Timing Analysis for Dependent Tasks in Distributed Embedded Systems

We propose a SMT method to derive a real-time guaranteed schedule for the autonomous driving system de- scribed in the industry challenge. All timing constraints proposed as the industry challenge are considered in our SMT model.

Toward Real-Time Guaranteed Scheduling for Autonomous Driving Systems

Considering the instance of autonomous driving system mentioned in RTSS 2021 Industry Challenge, a mixed integer linear programming (MILP) model is built to formulate this specific instance. Based on the specific instance, system and scheduling models are proposed correspondingly.

A Linear Programming Solution to Real-time Scheduling in Autonomous Driving System

In this paper, we propose a general solution to the problem of scheduling real-time applications on heterogeneous hardware platforms. To fully utilize the computing capacity of heterogeneous processing units, we model the real-time applica- tion as a heterogeneous Directed Acyclic Graph (DAG) which specifies the types of processors (CPU, GPU, etc.) where each task should run. In this well-known DAG context, we propose a novel extension aimed at safety-critical systems that operate in unpredictable environments: the coupling of conditional DAG nodes with stochasticity. Specifically, conditional DAG nodes en- able the modeling of systems that execute computational pipelines based on environmental context, while stochasticity of DAG edges captures the uncertain nature of a system’s environment or the reliability of its hardware. Furthermore, considering the pessimism of deterministic worst-case execution time (WCET) in scheduling processes, we model execution times of tasks (DAG nodes) as probability distributions which yields a novel stochastic conditional DAG model. Coupled with a novel S-bottleneck heuris- tic and safety-performance (SP) metric, our proposed framework allows for efficient online scheduling in complex computational pipelines, with more flexible representation of timing constraints, and ultimately, safety-performance trade offs.

S-Bottleneck Scheduling with Safety-Performance Trade-offs in Stochastic Conditional DAG Models

We regard the industry challenge as a multi- objective real-time scheduling problem. Different from tradi- tional methods, we model it as a nonlinear program (NLP) and use gradient-based methods for efficient solutions. Different requirements can be freely added as constraints, which gives the proposed method effective for a large range of problems. Furthermore, the proposed NLP scheduling method can utilize available scheduling algorithm as initialization method, and improve their performance even further. Preliminary evaluation shows advantages against simple heuristic methods in compli- cated problems.

A General Scheduling Framework for Multi-objective Real-time Systems

As a feasible solution for the RTSS 2021 industry challenge, we propose System-Level Logical Execution Time (SL- LET) in combination with a modified middleware layer.

Premium content

Time-Predictable Acceleration of Deep Neural Networks on FPGA-based SoC

Downloads

Next from IEEE RTSS 2021

Industrial Challenge: Opening, Introduction, and Overview of Solutions

Similar lecture

HIART-MCS: High Resilience and Approximated Computing Architecture for Imprecise Mixed-Criticality Systems