Exploiting Inhomogeneities of Subthreshold Transistors as Populations of Spiking Neurons

As machine learning applications are becoming increasingly more powerful and are deployed to a growing number of different devices, the need for energy-efficient implementations is rising. To meet this de- mand, a promising field of research is the use of spiking neural networks in combination with neuromorphic hardware, as energy is only consumed when information is being processed. The approach that maximizes en- ergy efficiency, an analog layout with transistors operating in subthresh- old mode, suffers from inhomogeneities such as device mismatch which makes it challenging to create a uniform threshold necessary for spiking neurons. Furthermore, previous work mainly focused on spiking feedfor- ward or convolutional networks, as neural networks based on rectified linear unit translate well to rate coded spiking neurons. Consequently, the processing of continuous sequential data remains challenging, as neu- ral networks, with long short-term memory or gated recurrent units as recurrent cells, are based on sigmoid and tanh activation functions. We show how these two disadvantages can compensate for each other, as a population of spiking neurons with a normally distributed threshold can reliably represent the sigmoid and tanh activation function. Based on this finding we present a novel method on how to convert a long short- term memory recurrent network to a spiking neural network. Although computationally expensive in a simulation environment, this approach offers a significant opportunity for energy reduction and hardware fea- sibility as it leverages the often unwanted process variance as a design feature.     «

As machine learning applications are becoming increasingly more powerful and are deployed to a growing number of different devices, the need for energy-efficient implementations is rising. To meet this de- mand, a promising field of research is the use of spiking neural networks in combination with neuromorphic hardware, as energy is only consumed when information is being processed. The approach that maximizes en- ergy efficiency, an analog layout with transistors operating in subthresh-...     »