Power/Performance Bits: Mar. 19

Explainable AI
Researchers from Technische Universität Berlin (TU Berlin), Fraunhofer Heinrich Hertz Institute (HHI), and Singapore University of Technology and Design (SUTD) propose a pair of algorithms to help determine how AI systems reach their conclusions.

Explainable AI is an important step towards practical applications, argued Klaus-Robert Müller, Professor for Machine Learning at TU Berlin. “Specifically in medical diagnosis or in safety-critical systems, no AI systems that employ flaky or even cheating problem solving strategies should be used.”

The first algorithm, developed previously by TU Berlin and Fraunhofer HHI, is called the Layer-wise Relevance Propagation (LRP) algorithm and allows visualizing according to which input variables AI systems make their decisions. The second, called Spectral relevance analysis (SpRAy), extends LRP to identify and quantify a wide spectrum of learned decision making behavior. The two make it possible to detect undesirable decision making even in very large data sets, according to the team.

“We were very surprised by the wide range of learned problem-solving strategies,” said Wojciech Samek, group leader at Fraunhofer HHI. “Even modern AI systems have not always found a solution that appears meaningful from a human perspective, but sometimes used so-called ‘Clever Hans Strategies’.”

Clever Hans was a horse renown in the early 1900s for the supposed ability to count and tap out the answers to math problems. However, Hans was later shown to not actually solve the math problems, but determine the correct answer by reading involuntary clues in the body language of his handler.

In their research, the team found several “Clever Hans” strategies used by various AI systems.

One system they tested, which won several international image classification competitions a few years ago, classified images mainly on the basis of context. Images were assigned to the category “ship” when there was a lot of water in the picture. If rails were present, images were classified as “train.” Other pictures were assigned the correct category by their copyright watermark. The researchers argue that the real tasks (detecting the concepts of ships or trains) were not solved by the AI, even if it classified most images correctly.

Even state-of-the-art deep learning systems used such faulty problem-solving strategies. These networks based their classification decision in part on artifacts that were created during the preparation of the images and have nothing to do with the actual image content, the team said.

“It is quite conceivable that about half of the AI systems currently in use implicitly or explicitly rely on such ‘Clever Hans’ strategies. It’s time to systematically check that, so that secure AI systems can be developed,” said Müller.

Along with simply understanding AI, the team thinks their algorithms could be used to improve datasets and reduce the learning of flawed strategies. The technology is open source and available to all scientists.

Low-power ASIC for small robots
Researchers at the Georgia Institute of Technology built a low-power ASIC capable of controlling miniature robotic cars and other small, swarm-type robots while only needing milliwatts of power to operate.

The neural network IC accommodates both model-based programming and collaborative reinforcement learning. To conserve power, it uses a hybrid digital-analog time-domain processor in which the pulse-width of signals encodes information.

“The size of the chip is reduced by half, and the power consumption is one-third what a traditional digital chip would need,” said Arijit Raychowdhury, associate professor in Georgia Tech’s School of Electrical and Computer Engineering. “We used several techniques in both logic and memory designs for reducing power consumption to the milliwatt range while meeting target performance.”

It is slower than digital or analog devices, but still sufficiently fast for the robots. “For these control systems, we don’t need circuits that operate at multiple gigahertz because the devices aren’t moving that quickly,” noted Raychowdhury. “We are sacrificing a little performance to get extreme power efficiencies. Even if the compute operates at 10 or 100 megahertz, that will be enough for our target applications.”

In a demonstration, the robot cars navigated through an arena floored by rubber pads and surrounded by cardboard block walls. As they search for a target, the robots must avoid traffic cones and each other, learning from the environment as they go and continuously communicating with each other.

The cars use inertial and ultrasound sensors to determine their location and detect objects around them. Information from the sensors goes to the hybrid ASIC, which determines the robot’s next course of action. Instructions then go to a Raspberry Pi controller, which sends instructions to the electric motors.

The researchers noted that in palm-sized robots, power is consumed by the motors and controllers used to drive and steer the wheels, the processor, and the sensing system. The use of the team’s low power ASIC, however, meant the bulk of the power was consumed by the motors.

To reduce power further, the team is exploring MEMS motors. “We would want to build a system in which sensing power, communications and computer power, and actuation are at about the same level, on the order of hundreds of milliwatts,” said Raychowdhury. “If we can build these palm-sized robots with efficient motors and controllers, we should be able to provide runtimes of several hours on a couple of AA batteries. We now have a good idea what kind of computing platforms we need to deliver this, but we still need the other components to catch up.”

Ionic transistor
Researchers at Columbia University developed a biocompatible ion driven transistor that is fast enough to enable real-time signal sensing and stimulation of brain signals.

Called an internal-ion-gated organic electrochemical transistor (IGT), it operates via mobile ions contained within a conducting polymer channel to enable both volumetric capacitance (ionic interactions involving the entire bulk of the channel) and shortened ionic transit time. The team said the IGT has large transconductance (amplification rate), high speed, and can be independently gated as well as microfabricated to create scalable conformable integrated circuits.

“We’ve made a transistor that can communicate using ions, the body’s charge carriers, at speeds fast enough to perform complex computations required for neurophysiology, the study of the nervous system function,” said Dion Khodagholy, assistant professor of electrical engineering at Columbia Engineering. “Our transistor’s channel is made out of fully biocompatible materials and can interact with both ions and electrons, making communication with neural signals of the body more efficient. We’ll now be able to build safer, smaller, and smarter bioelectronic devices, such as brain-machine interfaces, wearable electronics, and responsive therapeutic stimulation devices, that can be implanted in humans over long periods of time.”

Silicon-based transistors used in bioelectronics need to be carefully encapsulated, making them bulky and rigid. Other organic devices have been too slow for neurophysiology applications.

To speed up their device, the team modified the material to have its own mobile ions. By shortening the distance that ions needed to travel within the polymer structure, they improved the speed of the transistor by an order of magnitude compared to other ionic devices of the same size.

“Importantly, we only used completely biocompatible material to create this device. Our secret ingredient is D-sorbitol, or sugar,” said Khodagholy. “Sugar molecules attract water molecules and not only help the transistor channel to stay hydrated, but also help the ions travel more easily and quickly within the channel.”

A primary application identified by the researchers is electroencephalography (EEG) procedures. In tests, the IGT enabled a five order of magnitude decrease in contact area, meaning it could be placed between hair follicles. Additionally, it conforms to the scalp without chemical adhesives.

The researchers say the device could be used to make implantable closed loop devices, such as those currently used to treat some forms of medically refractory epilepsy. They could also be used to record heart, eye, and muscle movement.