Power/Performance Bits: Aug. 8

Energy-efficient neural nets
Researchers from MIT developed new techniques for more energy-efficient neural networks, offering as much as a 73% reduction in power consumption over the standard implementation of neural networks, and as much as a 43% reduction over the best previous method for paring the networks down.

First, they developed an analytic method that can determine how much power a neural network will consume when run on a particular type of hardware. Then they used the method to evaluate new techniques for paring down neural networks so that they’ll run more efficiently on handheld devices.

“One of the questions that people had is ‘Is it more energy efficient to have a shallow network and more weights or a deeper network with fewer weights?’ This tool gives us better intuition as to where the energy is going, so that an algorithm designer could have a better understanding and use this as feedback,” said Vivienne Sze, associate professor of electrical engineering and computer science at MIT. “The second thing we did is that, now that we know where the energy is actually going, we started to use this model to drive our design of energy-efficient neural networks.”

To reduce energy, the team focused on reducing the number of weights, the connections between nodes which determine how much a given node’s output will contribute to the next node’s computation.

Low-weight connections between nodes contribute very little to a neural network’s final output, so many of them can be safely eliminated, a technique called pruning.

Although cutting even a large number of low-weight connections can have little effect on a neural net’s output, cutting all of them probably would, so pruning techniques must have some mechanism for deciding when to stop.

The researchers thus began pruning those layers of the network that consume the most energy. That way, the cuts translate to the greatest possible energy savings.

Weights in a neural network can be either positive or negative, so the researchers’ method also looks for cases in which connections with weights of opposite sign tend to cancel each other out. The inputs to a given node are the outputs of nodes in the layer below, multiplied by the weights of their connections. So the method looks not only at the weights but also at the way the associated nodes handle training data. Only if groups of connections with positive and negative weights consistently offset each other can they be safely cut. This leads to more efficient networks with fewer connections than earlier pruning methods did.

Harvesting energy from low-frequency movements
Engineers at Vanderbilt University developed an ultrathin energy harvesting system built of layers of 2D black phosphorus. The device generates small amounts of electricity when it is bent or pressed, even at the extremely low frequencies characteristic of human motion.

“Compared to the other approaches designed to harvest energy from human motion, our method has two fundamental advantages,” said Cary Pint, assistant professor of mechanical engineering at Vanderbilt. “The materials are atomically thin and small enough to be impregnated into textiles without affecting the fabric’s look or feel and it can extract energy from movements that are slower than 10 Hertz–10 cycles per second–over the whole low-frequency window of movements corresponding to human motion.”

Extracting usable energy from low frequency motion is challenging. A number of research groups are developing energy harvesters based on piezoelectric materials that convert mechanical strain into electricity. However, these materials often work best at frequencies of more than 100 Hertz. This means that they don’t work for more than a tiny fraction of any human movement so achieve efficiencies of less than 5-10% even under optimal conditions.

Graph showing the operating ranges of different types of energy harvesting devices. The red stars denote piezoelectric devices that use crystals which produce electricity when deformed. The blue circle represents another solid-state device called an ionic diode that generates electricity when compressed. The orange triangles depict triboelectric nanogenerators that produce electricity by sliding friction. The purple circles show the performance of the ultrathin strain harvester developed at Vanderbilt. (Source: Nanomaterials and Energy Devices Laboratory/Vanderbilt)

“Our harvester is calculated to operate at over 25% efficiency in an ideal device configuration, and most importantly harvest energy through the whole duration of even slow human motions, such as sitting or standing,” Pint said.

Both electrodes of the harvester are black phosphorus nanosheets. While this prevents the device from storing energy, it allows it to fully exploit the voltage changes caused by bending and twisting, producing up to 40 microwatts per square foot while sustaining current generation over the full duration of movements as slow as 0.01 Hertz.

The researchers acknowledge that one of the challenges they face is the relatively low voltage, in the millivolt range, that their device produces. However, they are trying to step up the voltage as well as exploring the design of electrical components, like LCD displays, that operate at lower than normal voltages.

Reconfigurable crystal circuits
Physicists at Washington State University found a way to write an electrical circuit into a crystal, opening up the possibility of transparent, three-dimensional electronics that can be erased and reconfigured.

The work began when it was noticed a strontium titanate crystal that was left exposed to light saw a 400-fold increase in electrical conductivity. The latest research involved using a laser to etch a line in the crystal. With electrical contacts at each end of the line, it carried a current.

Ordinarily, a crystal does not conduct electricity. But when the crystal strontium titanate is heated under the right conductions, it is altered so light will make it conductive. The phenomenon, called “persistent photoconductivity,” also occurs at room temperature, an improvement over materials that require cooling with liquid nitrogen.

Washington State University researchers used light to write a highly conducting electric path in a crystal. This opens up the possibility of transparent, three-dimensional electronics that, like an Etch-A-Sketch, can be erased and reconfigured. On the left, a photograph of a sample with four metal contacts. On the right, an illustration of a laser drawing a conductive path between two contacts. (Source: WSU)

“We’re still trying to figure out exactly what happens,” said Matt McCluskey, a WSU professor of physics and materials science. He surmises that heat forces strontium atoms to leave the material, creating light-sensitive defects responsible for the persistent photoconductivity.

The recent work increased the crystal’s conductivity 1,000-fold. The phenomenon can last up to a year. “We look at samples that we exposed to light a year ago and they’re still conducting,” said McCluskey. “It may not retain 100% of its conductivity, but it’s pretty big.”

Moreover, the circuit can be by erased by heating it on a hot plate and recast with an optical pen.

“It opens up a new type of electronics where you can define a circuit optically and then erase it and define a new one,” said McCluskey. “It’s exciting that it’s reconfigurable. It’s also transparent. There are certain applications where it would be neat to have a circuit that is on a window or something like that, where it actually is invisible electronics.”