Energy efficiency

This research pertains to the energy-efficient determination of characteristics of a physical process using neuromorphic computer vision.

Determining the speed and direction of fan blade rotation was chosen as a test problem. The need to solve this problem can arise, for example, in automated monitoring of industrial rotating equipment.

Our prototype neuromorphic computer vision system solves the test problem using a DVS camera, a neuromorphic computing module powered by AltAI chips, and a hybrid SNN consisting of an encoder, a spiking block, and a decoder.

We chose a DVS camera as the video sensor for its event-driven and asynchronous nature (pixels operate independently, responding only to changes in the scene), which aligns well with similar information processing principles in SNNs and neuromorphic processing units.

The operating cycle of the neuromorphic computer vision
system consists of the following main steps

Structure of the hybrid SNN

The neural network is trained using the backpropagation method with weight quantization (ANN2SNN). The spiking component of the neural network contains ~16 500 trainable ternary weights {-1, 0, 1}. The Heaviside function is used for neuron activation.

Deploying a pre-trained spiking component of the neural network requires 15 AltAI chips in the prototype configuration, featuring 16 cores per chip.

The event-driven principle of neuromorphic computing in the sensor–chip–neural network chain is clearly illustrated by the activity of the cores within the AltAI chip when the SNN processes the data from the DVS camera (see the figure below). The cores are only active when there are changes in the monitoring scene (the fan blades are rotating). Moreover, only the local areas of the network (neuromorphic cores) currently involved in the processing are active.

Example of neuromorphic core activity depending on the input image generated by the DVS camera. 15 AltAI chips used for deploying the SNN are shown. Each chip contains 16 neuromorphic cores, each represented by a square element within the chip. The higher the activity of a neuromorphic core, the higher the brightness of its corresponding element in the chip

Example of the stand in operation

Charts

Angular velocities

Angular velocity of the fan blades: actual and predicted by the AltAI-based SNN

Angular velocity of the fan blades: actual and predicted by the GPU-based neural network

Power consumption

Power consumption of the AltAI-based SNN in milliwatts

Power consumption of the AltAI-based SNN and the GPU-based neural network in watts, on a logarithmic scale

Video stream from the video camera

Video stream from the DVS camera

Activity panel for the AltAI neuromorphic chip cores Deploying a pre-trained spiking component of the neural network requires 15 AltAI chips in the prototype configuration, featuring 16 cores per chipand comparison of the performance of neural networks running on AltAI and on the GPU in terms of accuracy, power consumption, and speed.

As seen from the power consumption graphs, the power consumption of the SNN running on 15 AltAI chips averaged 0.07 W. A similar neural network with real-valued weights, running on a GTX 1650 Ti, consumed an average of 13 W.

Furthermore, the neuromorphic solution has the potential for further improvements in energy efficiency by having the SNN work directly with individual DVS camera pixel events, rather than frames (images).

Contact

To discuss collaboration opportunities, please email us at neuro@kaspersky.com,
or use the contact form below