Germany

Deep neural networks (DNNs) have shown remark- able success in various machine-learning (ML) tasks useful for many safety-critical, real-time embedded systems. The foremost design goal for enabling DNN execution on real-time embedded systems is to provide worst-case timing guarantees with limited computing resources. Yet, the state-of-the-art ML frameworks hardly leverage heterogeneous computing resources (i.e., CPU, GPU) to improve the schedulability of real-time DNN tasks due to several factors, which include a coarse-grained resource allocation model (one-resource-per-task), the asymmetric nature of DNN execution on CPU and GPU, and lack of schedulability- aware CPU/GPU allocation scheme. This paper presents, to the best of our knowledge, the first study of addressing the above three major barriers and examining their cooperative effect on schedulability improvement. In this paper, we propose LaLaRAND, a real-time layer-level DNN scheduling framework, that enables flexible CPU/GPU scheduling of individual DNN layers by tightly coupling CPU-friendly quantization with fine-grained CPU/GPU allocation schemes (one-resource-per-layer) while mitigating ac- curacy loss without compromising timing guarantees. We have implemented and evaluated LaLaRAND on top of the state-of-the- art ML framework to demonstrate its effectiveness in making more DNN task sets schedulable by 56% and 80% over an existing approach and a baseline (vanilla PyTorch), respectively, with only up to -0.4% of performance (inference accuracy) difference.

IEEE RTSS 2021

LaLaRAND: Flexible Layer-by-Layer CPU/GPU Scheduling for Real-Time DNN Task

meetings

quantization (signal)

couplings

timing

embedded systems

graphics processing units

real-time systems

 **RTSS 2021** has successfully concluded. Hope you have enjoyed it, and look forward to seeing you next year in person.
    Announcement for the Awards:
 * Best Paper Award goes to: Monte Carlo Response-Time Analysis, Sergey Bozhko, Georg von der Brüggen, and Björn Brandenburg
 * Best Student Paper Award goes to: Automatic Energy-Hotspot Detection and Elimination in Real-Time Deeply Embedded Systems, Mohsen Shekarisaz, Lothar Thiele, and Mehdi Kargahi.
 * Best Solution Award for Industry Session goes to: Toward Real-Time Guaranteed Scheduling for Autonomous Driving Systems. Jinghao Sun, Tianyi Wang, Kailu Duan, Bin Lu, Jiankang Ren, Zhishan Guo, and Guozhen Tan.



The IEEE Real-Time Systems Symposium (RTSS) is the premier conference in the field of real-time systems, and is a venue for researchers and practitioners to showcase innovations covering all aspects of real-time systems, including theory, design, analysis, implementation, evaluation, and experience. 

technical paper

This work focuses on the time-predictable execution of Deep Neural Networks (DNNs) accelerated on FPGA System-on-Chips (SoCs). The modern DPU accelerator by Xilinx is considered. An extensive profiling campaign targeting the Zynq Ultrascale+ platform has been performed to study the execution behavior of the DPU when accelerating a set of state-of-the-art DNNs for Advanced Driver Assistance Systems (ADAS). Based on the profiling, an execution model is proposed and then used to derive a response-time analysis. A custom FPGA module named DICTAT is also proposed to improve the predictability of the acceleration of DNNs and tighten the analytical bounds. A rich set of experimental results based on both analytical bounds and measurements from the target platform is finally presented to assess the effectiveness and the performance of the proposed approach on ADAS applications.

Time-Predictable Acceleration of Deep Neural Networks on FPGA-based SoC

In this paper we address the timing verification for processing graphs used to implement autonomous driving systems by the company Perceptin. We demonstrate how to extend our previous results on end-to-end timing analysis in terms of maximum reaction time and maximum data age for cause-effect chains in automotive systems for the posed challenge.

End-To-End Processing Chain Analysis

The 2021 RTSS Industry challenge considers timing constraints in autonomous driving systems. One real-time requirement is that the differences of timestamps among all source data of an output must be upper-bounded by a pre-defined value, to make sure that data from different sources are correctly fused, e.g, object detection by using different sensing data. In this paper, we formally describe the challenge system as well as the requirements. In particular, we present timing analysis to bound the maximum differences of timestamps among all sources for an output. We show that an output may possibly be caused by two different data from the same source, when fork-join structure exists. This result suggests that data fusion after different long pipelined processing procedure must be avoided to meet required timing constraints.

Analysis of Time difference among Sensing Data in Autonomous Driving Systems

Given the problem proposed in RTSS 2021 Industry Challenge, we formally define a system model where dependent tasks are organized in a direct acyclic graph structure and communicate with an unblocking manner. For such a system, we present solutions to estimate three time latencies: maximum reaction time, maximum data age, and maximum difference among the timestamps of ”source” sensor data. One major contribution made in this paper is that the clock shifts among different processing units have been taken into account during the estimations without a mandatory schedule simulation process. By doing this, fairly tight upper bounds can be calculated for all three latencies.

Timing Analysis for Dependent Tasks in Distributed Embedded Systems

We propose a SMT method to derive a real-time guaranteed schedule for the autonomous driving system de- scribed in the industry challenge. All timing constraints proposed as the industry challenge are considered in our SMT model.

Toward Real-Time Guaranteed Scheduling for Autonomous Driving Systems

Considering the instance of autonomous driving system mentioned in RTSS 2021 Industry Challenge, a mixed integer linear programming (MILP) model is built to formulate this specific instance. Based on the specific instance, system and scheduling models are proposed correspondingly.

A Linear Programming Solution to Real-time Scheduling in Autonomous Driving System

In this paper, we propose a general solution to the problem of scheduling real-time applications on heterogeneous hardware platforms. To fully utilize the computing capacity of heterogeneous processing units, we model the real-time applica- tion as a heterogeneous Directed Acyclic Graph (DAG) which specifies the types of processors (CPU, GPU, etc.) where each task should run. In this well-known DAG context, we propose a novel extension aimed at safety-critical systems that operate in unpredictable environments: the coupling of conditional DAG nodes with stochasticity. Specifically, conditional DAG nodes en- able the modeling of systems that execute computational pipelines based on environmental context, while stochasticity of DAG edges captures the uncertain nature of a system’s environment or the reliability of its hardware. Furthermore, considering the pessimism of deterministic worst-case execution time (WCET) in scheduling processes, we model execution times of tasks (DAG nodes) as probability distributions which yields a novel stochastic conditional DAG model. Coupled with a novel S-bottleneck heuris- tic and safety-performance (SP) metric, our proposed framework allows for efficient online scheduling in complex computational pipelines, with more flexible representation of timing constraints, and ultimately, safety-performance trade offs.

S-Bottleneck Scheduling with Safety-Performance Trade-offs in Stochastic Conditional DAG Models

We regard the industry challenge as a multi- objective real-time scheduling problem. Different from tradi- tional methods, we model it as a nonlinear program (NLP) and use gradient-based methods for efficient solutions. Different requirements can be freely added as constraints, which gives the proposed method effective for a large range of problems. Furthermore, the proposed NLP scheduling method can utilize available scheduling algorithm as initialization method, and improve their performance even further. Preliminary evaluation shows advantages against simple heuristic methods in compli- cated problems.

A General Scheduling Framework for Multi-objective Real-time Systems

As a feasible solution for the RTSS 2021 industry challenge, we propose System-Level Logical Execution Time (SL- LET) in combination with a modified middleware layer.

Premium content

LaLaRAND: Flexible Layer-by-Layer CPU/GPU Scheduling for Real-Time DNN Task

Downloads

Next from IEEE RTSS 2021

Time-Predictable Acceleration of Deep Neural Networks on FPGA-based SoC

Similar lecture

TimeWall: Enabling Time Partitioning for Real-Time Multicore+Accelerator Platforms