Tutorial Sessions

Abstract
The brain is the perfect place to look for inspiration to develop more efficient neural networks. While the computational cost of deep learning exceeds millions of dollars to train large-scale language models, our brains are somehow equipped to process an abundance of signals from our sensory periphery, to provide feedback control to motor systems, and to ensure our involuntary biological processes do not shut down. All within a power budget of approximately 10-20 watts.
One of the main differences with modern deep learning is that the brain encodes and processes information as spikes rather than continuous, high-precision activations. The dominant cost of deep learning hardware accelerators arises from regular memory access and data communication. But distributing information in the temporal domain in the form of sparse spiking activity has demonstrated 100-1000x reduction of energy consumption in deep learning workloads.
This tutorial will dive into the intersection between neuroscience, deep learning, and hardware acceleration, and how spanning across these various layers of abstraction can drive forward the next generation of deep learning algorithms. We will explore how to adopt basic principles of neurobiology into modern deep learning algorithms trained via error backpropagation. Several practical case studies of spike-based processing will be explored, from front-end event-based sensing, to back-end processing, and how all of these concepts come together to enable ultra-efficient cognitive computation. The tutorial will conclude with a hands-on coding session, and an overview of best practices for ensuring optimal performance of spike-based algorithms on neuromorphic hardware.

Abstract
Many platforms, from data centers to edge devices, started to deploy multiple DNNs to provide high-quality results for diverse applications in various domains such as computer vision, speech, language, and so on. Unlike single task-oriented applications that were dominant in the past, emerging applications such as AR/VR deploy multi-DNN pipelines performing different tasks to enable more complex applications. Also, such workloads are often dynamic where some models are activated based on the results of another model’s output (e.g., keyword detection – speech recognition pipeline). In addition, some models in such applications need to be run constantly at a target processing rate (e.g., object detection on an autonomous vehicle). Such new features of emerging workloads such as AR/VR introduce new design challenges to accelerator systems.

This tutorial will introduce such real-time and multi-DNN workloads from emerging applications and discuss their unique features and challenges to accelerator systems. As a case study, this tutorial will also present our recent work, Herald, which proposed an accelerator architecture tailored for real-time multi-DNN workloads.

Abstract
Deep learning algorithms gathered serious attention due to their outstanding performance in various tasks. Their application areas are fast expanding from computer vision and speech recognition to multi-modal understanding. While power-saving techniques such as quantization, network compression, and pruning have been successfully adopted in pre-trained models, they often become next to useless when applied to the training process. This talk will discuss various algorithmic and hardware optimization techniques enabling energy-efficient training processors. While backpropagation is a very powerful tool for gradient computation, its computational graph does not match the human brain very well, and it requires high-precision computations for reliable training convergence. In the second part of the talk, alternative algorithms based on neuromorphic computing for training deep neural networks will be presented. Recent advances in neuromorphic computing will be discussed, followed by design examples of neuromorphic processors.

Abstract
Low-Power Computer Vision: Algorithms and Practice, I will introduce our team’s winning solution of IEEE low-power computer vision (LPCV) challenge 2021, Track 1 (UAV video). The track targets detecting and tracking multiple moving objects in video captured by an unmanned aerial vehicle (UAV), and the algorithm solution needs to be both accurate and energy-efficient (as measured by Raspberry Pi 3B+). The task poses challenges for reducing temporal-spatial redundancy at all three levels: data, model, and hardware. I will describe the efforts our team made for optimizing each aspect, and how the three levels work together as an efficient and effective pipeline in the wild.

Abstract
Realizing a 5X Computer Vision Inference Speedup with PyTorch, I will conduct an interactive demonstration of popular low-power computer vision techniques for embedded devices. The techniques covered will include (a) Parameter Quantization, (b) Neural Network Pruning, and (c) Knowledge Distillation. Starting from one of the state-of-the-art “unoptimized” deep neural networks, we will apply the different techniques in PyTorch, to see reductions in latency, energy, and memory requirements. I will also discuss the combination of the various techniques and describe the accuracy-efficiency tradeoff. Before concluding, I will explain the open problems in the area and highlight directions for future research.

Abstract
Machine Learning Reproducibility: Guidance for Practitioners, Machine learning techniques are becoming a fundamental tool for scientific and engineering progress. These techniques are applied in contexts as diverse as astronomy and spam filtering. However, correctly applying these techniques requires careful engineering. Much attention has been paid to the technical potential; relatively little attention has been paid to the software engineering process required to bring research-based machine learning techniques into practical utility. Technology companies have supported the engineering community through machine learning frameworks such as TensorFLow and PyTorch, but the details of how to engineer complex machine learning models in these frameworks have remained hidden. To promote best practices within the engineering community, academic institutions and Google have partnered to launch a Special Interest Group on Machine Learning Models (SIGMODELS) whose goal is to develop exemplary implementations of prominent machine learning models in community locations such as the TensorFlow Model Garden (TFMG). The speech defines a process for reproducing a state-of-the-art machine learning model at a level of quality suitable for inclusion in the TFMG. We define the engineering process and elaborate on each step, from paper analysis to model release. We report on our experiences implementing the YOLO model family with a team of 26 student researchers, share the tools we developed, and describe the lessons we learned along the way.

Abstract
Wearable intelligent health monitoring devices can detect the abnormality in users’ biomedical signals and generate real-time alerts. Compared with the general health monitoring equipments in the hospital, they can be used for long-term health monitoring with automatic abnormality detection without involving the doctor, which is very suitable for health monitoring at home. A key component of the wearable intelligent health monitoring device is the biomedical AI processor. By embedding the biomedical AI processing capability in the wearable health monitoring device, real-time abnormality detection with low data transmission can be achieved. A major challenge of biomedical AI processor design is that the AI computation involves large computational complexity causing high power consumption, which is not suitable for size- and energy-constrained wearable devices. Therefore, how to reduce the power consumption of the biomedical AI processor while achieving high accuracy and real-time performance is an urgent need. This talk introduces the design techniques combining algorithm, architecture, circuit and device to address the above challenges.

Abstract
Computing in memory (CIM) processor is quite promising to improve energy efficiency of ML applications. Plenty of CIM circuit designs are emerging, such as SRAM, DRAM and RRAM, which demonstrates obvious advantages over digital counterparts in device and macro level. Recently, how to exploit CIM in system-level processor design is attracting more and more attentions. This tutorial will introduce CIM processor designs from a holistic approach, covering device, macro and system level techniques to design programmable and scalable CIM processors. Hardware and software co-design techniques will also be covered to support realistic ML applications, including considerations of device/circuit non-idealities, system architecture, and model structures suitable for CIM processors with compression techniques such as pruning and quantization.

Abstract
Voice-activated wake-up functions such as voice activity detection (VAD) and keyword spotting (KWS) are prevalent for always-on edge audio devices. This tutorial will cover the design trends of power-efficient analog signal processing circuits for the edge audio tasks. These continuous-time (CT) feature extraction circuits are bio-inspired, especially from the biological cochlea, building up an architecture of a set of frequency-selective channels which include a band-pass filter (BPF), rectifier and spike generator. Starting with an early generation of bio-inspired audio feature extractors, we will provide a new figure-of-merit (FoM) to compare the state-of-the-art designs. We will give a demonstration example that combines a bio-inspired spiking cochlea front-end with a deep-neural-network (DNN) classifier showing the end-to-end audio-inference.

Abstract
AI with IoT (AIOT) application is widely appear in human’s life such as self-driving car, smart door bell. These applications show the AI is moving forward from cloud to edge device. However, the edge device for AI have the challenges of low to medium computing power, small form factor, long battery life time. These challenges are all related the limitation of computing energy efficiency. The conventional computer architecture (Von-Neumann machine) is not efficient due to the memory bandwidth limit and data movement frequently. A new computing architecture in-memory computing (IMC) is proposed to improve the computing efficiency. The IMC can directly compute (multiple and accumulate) in memory to avoid the data movement between the CPU and memory. However, IMC still have many challenges need to be overcome, such as linearity, throughput, energy and area efficiency. The design challenges of IMC will be discussed first. This challenges also need to consider the system requirement. Then the characteristic of memory devices for IMC will be introduced. According the characteristic of memory devices, the designer can select the suitable memory device for weight storage. Finally, the previous works of IMC macro will be analyzed and discussed.