Ternary Stochastic Neuron – Implemented with a Single Strained Magnetostrictive Nanomagnet
Rahnuma Rahman, Supriyo Bandyopadhyay · 2024
Read Ternary Stochastic Neuron – Implemented with a Single Strained Magnetostrictive Nanomagnet by Rahnuma Rahman, Supriyo Bandyopadhyay (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04246.txt.
Introduction
A single nanomagnet can act as a three-way random neuron, Rahnuma Rahman and Supriyo Bandyopadhyay show in a 2024 paper. Random neurons power efficient brain-like hardware, but most flip between two states or slide across a range. Three-state versions excel at sorting patterns such as digits and images.
Their device is a tiny magnet that changes shape under strain, with almost no energy barrier to hold a state. Heat noise then tosses its magnetization among three values on its own. Strain applied along one axis sets the random behavior. A cheap, tiny three-state neuron could shrink the hardware cost of probabilistic computing chips.
Catalog record
| CREATOR | Rahnuma Rahman, Supriyo Bandyopadhyay |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04246
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ternary Stochastic Neuron – Implemented with a Single Strained Magnetostrictive Nanomagnet Rahnuma Rahman, Supriyo Bandyopadhyay Abstract Stochastic neurons are extremely efficient hardware for solving a large class of problems and usually come in two varieties – “binary” where the neuronal state varies randomly between two values of ± \pm 1 and ”analog” where the neuronal state can randomly assume any value between -1 and +1. Both have their uses in neuromorphic computing and both can be implemented with low- or zero-energy-barrier nanomagnets whose random magnetization orientations in the presence of thermal noise encode the binary or analog state variables. In between these two classes is n n -ary stochastic neurons, mainly ternary stochastic neurons (TSN) whose state randomly assumes one of three values (-1, 0, +1), which have proved to be efficient in pattern classification tasks such as recognizing handwritten digits from the MNIST data set or patterns from the CIFAR-10 data set. Here, we show how to implement a TSN with a zero-energy-barrier (shape isotropic) magnetostrictive nanomagnet subjected to uniaxial strain. Keywords: ternary stochastic neurons, activation functions, zero barrier nanomagnets, magnetostriction, strain 1 Introduction: Ternary stochastic neurons (TSNs) Deep neural networks (DNN) are essential ingredients of artificial intelligence (AI), but deploying them at the edge or for embedded applications (wearable electronics, personal communicators, etc.) faces significant challenge because of the need for massive storage and computational power. Numerous compression techniques have been proposed to reduce the computational burden [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ] and have performed admirably well. A different approach that departs from conventional compression methods is to use ternary quantization to reduce storage and computational cost [ 13 ] . Here every neuron has three states instead of the usual two in binary quantization. Hence, there is an obvious saving in area and energy cost which is beneficial for edge applications. Ternary quantization has been shown to perform very well in image classification and detection tasks involving MNIST, CIFAR-10 and Imagenet [ 13 , 14 ] . It also has very little accuracy degradation and can improve the accuracy of some models [ 14 ] . When both the weights and activations assume ternary values, the DNNs can be reduced to sparse binary networks with significant space and energy savings [ 15 ] . Ever since the dawn of digital electronics, binary devices with two states to encode digital information have ruled the roost. While they are easy to implement, they provide the least information content per device. As a result, even in the digital logic community, there is a strong push to switch to a higher radix/logic number by employing multi-state logic [ 16 ] . If the same device can encode n states ( n>2 )instead of two, it obviously can process more data in the same area of a chip, resulting in improvements in computational density . When it comes to low power and energy consumption, low complexity, low on-chip and off-chip interconnections, and high speed, the n -ary system ( n>2 ) is inevitably superior to the binary system [ 16 ] . This is true in all forms of information processing. Ternary stochastic neurons (TSNs) have three output states instead of the two in BSNs. These three states can be represented by -1, 0 and +1, and the probability of being in any of these states will be manipulated by an external agent. While BSNs (with two states -1 and +1) are implemented with low-barrier nanomagnets [ 17 , 18 ] and analog stochastic neurons (with any state between -1 and +1) are implemented with even lower barrier nanomagnets [ 19 ] , the implementation of a TSN is more tricky. To understand the challenge, consider the activation function of a BSN which is shown in Fig. 1 (a). It is a sigmoid or tanh(x) function, which is very appropriate for a BSN, but not a TSN [ 20 ] . This is because the slope near zero activation strength is highest and hence the state 0 will not be stable if such an activation function is adopted for a TSN. The neuron will tend to get pushed away from this state. Instead, we would want an activation function that looks like Fig. 1 (b) which allow the intermediate state with 0 output to be stable [ 20 ] . Ref. [ 20 ] proposed a function to replace tanh(x) : f(x)=1.5tanh(x)+0.5tanh(-3x), (1) which replicates the shape in Fig. 1 (b). Needless to say that the standard low barrier nanomagnet of refs. [ 17 , 18 ] cannot implement an activation function of this shape, and so we have to devise a new strategy to implement a TSN with an activation function that looks like the one in Fig. 1 (b). Figure 1: Activation function of: (a) a binary stochastic neuron, and (b) a ternary stochastic neuron. The latter looks like two staircases with a landing. The width of the “landing” or the plateau can be increased by increasing the magnitude of the strain. Figure 2: (a) A circular disk of a magnetostrictive material into which a spin-polarized current is injected perpendicular to the plane. The current is spin polarized in the \pm y-direction with the current’s sign denoting the spin polarization and not the current polarity. (b) The y-component of the magnetization averaged over time (which is the activation function) versus the spin polarized current (which is the activation agent) when no strain is present. (c) The y-component of the magnetization averaged over time versus the spin polarized current when uniaxial strain of the correct sign is applied along the y-axis. (d) The configuration of the TSN. The magnetic tunnel junction is used to inject a spin polarized current of either polarization into the soft layer which acts as the TSN and the two shorted gate electrodes are used to apply uniaxial strain of the correct sign along the y-axis. The sign of the strain depends on the polarity of the gate voltage. If the electric field generated by the gate voltage is opposite to the direction of poling of the piezoelectric layer, then compressive (negative) strain will be generated along the axis joining the two gate electrodes (y-direction) and tensile (positive) strain will be generated in the transverse direction (x-direction). Reversing the polarity will reverse the signs of the strains. The MTJ resistance denotes the neuronal state; “high” is the state -1, “intermediate” is the state 0 and “low” is the state +1. 2 Strained magnetostrictive nanomagnets for TSN Consider a magnetostrictive nanomagnet with in-plane anisotropy made of, say, Galfenol (FeGa) that is shaped like a circular disk as shown in Fig. 2 . There is no in-plane shape-anisotropy potential barrier. At room temperature, the magnetization will fluctuate randomly because of thermal perturbations and all in-plane orientations will be equally likely. If we now inject a spin polarized current into the nanomagnet perpendicular to the plane, with the spin polarized along a chosen in-plane direction, then we will bias the probability of the magnetization to point in that direction because the spin polarized electrons in the current will transfer their