Intel has released the latest iteration of its neuromorphic hardware, called Loihi. Intel Loihi processors feature electronics that behave much like neurons.
Despite their name, neural networks are highly related to the kinds of things you will find in a brain. While their organization and the way they transfer data through processing layers may have similarities to real neural networks, the data and calculations made on it will be very familiar to a typical CPU.
But neural networks are not the only way people have tried to learn lessons from the nervous system. There is a separate branch called neuromorphic computing that is based on the approach to the behavior of individual neurons in hardware. In neuromorphic hardware, calculations are performed by many small units that communicate with each other through bursts of activity called peaks and adjust their behavior based on the peaks they receive from others.
On Thursday, Intel released the latest iteration of its neuromorphic hardware, called Loihi. The new version comes with the kinds of things you'd expect from Intel: a better processor and some basic computing improvements. It also comes with some fundamental hardware changes that will allow it to run completely new classes of algorithms. And while Loihi remains a research-focused product for the time being, Intel is also releasing a compiler that it hopes will lead to wider adoption.
To make sense of what exactly Loihi does and what's new in this version, we started by looking at some information about neurobiology.
The basis of the nervous system is a type of cell called a neuron. All neurons share some common functional characteristics. At one end of the cell there is a structure called dendrites, which you can think of as a receptor. This is where the neuron receives inputs from other cells. Nerve cells also have axons, which act as transmitters, connecting to other cells to pass signals.
The receiving cell incorporates a variety of information and uses it to determine its own state of activity. Once a limit is exceeded, it will activate a downward spike on its own axes and possibly activate activity in other cells.
Usually, this leads to sporadic, random distant activity peaks when the neuron does not receive much input. Once it starts receiving signals, however, it will go into active mode and trigger a bunch of peaks in quick succession.
Neural networks can be implemented in software on traditional processors. But it is also possible to implement them through hardware, as Intel does with Loihi. The result is a processor that looks a lot like anything you might know.
The previous generation Loihi chip contains 128 individual cores connected to a communication network. Each of these nuclei has a large number of individual "neurons". Each of these neurons can receive input in the form of spikes from any other neuron - a "neighbor" in the same nucleus, a unit in a different nucleus on the same chip, or from another chip altogether. THE neuron incorporates the peaks it receives over time and, based on the behavior with which it is programmed, uses it to determine when it will send its own peaks to whichever neurons it connects to.
All "spike signaling" happens asynchronously. At specified intervals, the built-in x86 cores on the same chip force a synchronization. At that point, the neuron will redo the weights of its various connections - essentially, how much attention must be paid to all the individual neurons sending signals to it.
In real neuron terms, part of the execution unit on the chip acts as a dendrites, processing incoming signals from the communication network based in part on the weight derived from the previous behavior. A mathematical formula was then used to determine when the activity had exceeded a critical threshold and to activate its own peaks when it did. The "axis" of the execution unit then looks for which other execution units it communicates with and sends a spike to each one.
In Loihi's previous iteration, a spike merely conveyed a bit of information. A neuron registered as soon as they received one.
Unlike a regular processor, there is no external memory RAM. Instead, each neuron has a small cache dedicated to its use. This includes the weights it assigns to inputs from different neurons, a recent activity cache, and a list of all the other neurons to which spikes are sent.
One of the other big differences between neuromorphic chips and traditional processors is energy efficiency, where neuromorphic chips are "far ahead". Mike Davies, director of Intel's Neuromorphic Computing Lab, said Loihi could beat traditional processors by 2.000 in a given workload.
Source of information: arstechnica.com