Partly due to their exceptional performance in visual perception, deep neural networks (DNNs) are currently used in a wide range of tasks in safety-critical systems.During the execution of a neural network (NN), transient faults can cause errors, such as bit-flips, that affect the classification performance of the NN. This poses a serious threat to the functional safety of systems that utilize DNNs. By incorporating checksums into the computation of a NN, its reliability and resilience against faults can effectively be enhanced through selective protection instead of complete redundancy.However, designing optimal network architectures is a difficult and time-consuming task, especially when error resiliency and hardware efficiency are considered simultaneously. Recent approaches in hardware-aware neural architecture search (NAS) show that optimizing NN architectures automatically with respect to given objectives can lead to good performing yet efficient NNs. Therefore, extending NAS to additionally consider the error resilience of networks poses a good method to obtain optimal NNs for safety-critical applications.In this paper, we present an evolutionary multi-objective optimization approach to optimize NN architectures for classification accuracy, error-resilience and hardware-efficiency. By using a novel surrogate model to estimate the classification performance of NNs as well as functions with low computational costs for the remaining objectives, our method is highly efficient and outperforms current NAS approaches in terms of search cost.Besides optimizing the inherent error resilience of NNs, we incorporate checksums into their computation as an external measure to increase the error resilience. The utilized checksums leverage a NN’s linear properties to provide error detection capabilities at low costs. Depending on the configuration, they are guaranteed to detect up to three errors and close to 100% for higher error rates.We conduct two NAS experiments on the CIFAR-10 and GTSRB classification benchmarks and evaluate whether optimizing the checksum configuration for a NN is more advantageous when done during NAS or sequentially. Additionally, the error resilience of the discovered optimal network architectures is investigated through extensive fault injection simulations.The experiments show that NN architectures discovered on the CIFAR-10 dataset can be transferred to GTSRB and still achieve competitive performance. Furthermore, it can be observed that their error resilience characteristics are also maintained.With an additional computational overhead as low as 17% for the found architectures, the results show that checksums are an efficient method to effectively enhance the error resilience of NNs. Moreover, since determining the checksum configuration for a NN is more advantageous when done after NAS, also pre-trained NN can benefit from a significantly improved error resilience.

Automated Hardening of Deep Neural Network Architectures

Paper Proceedings: https://asmedigitalcollection.asme.org/IMECE/IMECE2021/volume/85543

ASME 2021 International Mechanical Engineering Congress and Exposition (IMECE2021)

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.