Power/Performance Bits: Dec. 11

Internet of Ears for smart buildings
Scientists at Case Western Reserve University proposed a new way for smart homes to determine building occupancy: sensors that ‘listen’ to vibration, sound, and changes in the existing ambient electrical field.

“We are trying to make a building that is able to ‘listen’ to the humans inside,” said Ming-Chun Huang, an assistant professor in electrical engineering and computer science at Case. “We are using principles similar to those of the human ear, where vibrations are picked up and our algorithms decipher them to determine your specific movements. That’s why we call it the ‘Internet of Ears.'”

The team believes that with a few small, hidden sensors, smart home technologies could adapt power usage without the need for cameras to determine occupancy.

“The first advantage will be energy efficiency for buildings, especially in lighting and heating, as the systems adjust to how humans are moving from one room to another, allocating energy more efficiently,” Huang said.

Aside from vibration, changes in the electrical field can be a good indicator of someone’s presence.

“There is actually a constant 60 Hz electrical field all around us, and because people are somewhat conductive, they short out the field just a little,” said Soumyajit Mandal, an assistant professor of electrical engineering and computer science at Case. “So, by measuring the disturbance in that field, we are able to determine their presence, or even their breathing, even when there are no vibrations associated with sound.”

The team has tested the technology in conference rooms in the electrical engineering department on campus and in the Smart Living Lab at Ohio Living Breckenridge Village, a senior-living community in Willoughby, Ohio.

Mandal said they have used as few as four small sensors in the walls and floor of a room. As for privacy concerns, Mandal said the system would not be able to identify individuals, although it could be calibrated to recognize the different gaits of people.

Additionally, the system could be used to track and measure a building’s structural integrity and safety, based on human occupancy in case of an earthquake or hurricane.

“This hasn’t really been explored as far as we’ve seen, but we know that humans create a dynamic load on buildings, especially in older buildings,” Huang said. “In collaboration with our colleague YeongAe Heo in the Civil Engineering department, we are trying to predict if there is going to be structural damage because of the increased weight or load based on the number of people on the floor or how they are distributed on that floor.”

Loose-fitting smart clothing
Researchers at the University of Massachusetts Amherst developed a wearable fabric-based triboelectric joint sensing system that can be integrated with loose-fitting clothing to sense a variety of joint movements.

Wearable devices are a hot area, with ‘smart clothing’ applications expected to grow dramatically in the next decade. Smart clothing, however faces a number of problems when it comes to integrating sensors with loosely worn garments. Deepak Ganesan, a professor of computer science at Amherst, pointed out that many sensors, like inertial sensors and electromyography require a tight fit to reduce motion artifacts and obtain a meaningful signal. But tight clothing is uncomfortable to wear and not appropriate in many applications such as elder care and patient care.

For the new device, which the researchers dub Tribexor, they used a functionalized triboelectric fabric comprised of layers that transfer surface charge from one layer to another and generate a voltage or current when compressed, tugged or twisted due to joint motion. This translates movement into an electrical signal and extracts useful information from loosely worn smart textiles.

The Tribexor device. (Source: UMass Amherst)

“Normally, loose fitting clothing would be considered a problem because that means we have to deal with a significant amount of noise, which is already a problem for relatively tight fitting devices like fitness bands,” said Ganesan. But the Tribexor device turns this limitation into an advantage, he notes, because loose-fitting clothing can fold, compress and twist more.

The team says the device has 95% accuracy for detecting elbow and knee flexion and extension movements and 85% accuracy for estimating angular velocity of the elbow and knee joints. It also accurately detects a variety of activities of daily living.

“This technology can be particularly useful for monitoring elderly individuals,” noted Trisha Andrew, a professor of chemistry at Amherst. “Current generation wearables, like smartwatches, are not ideal for this population since elderly individuals often forget to consistently wear or are resistant to wearing additional devices, whereas clothing is already a normal part of their daily routine.”

Privacy and photo info services
Researchers at Osaka University suggest a method to improve privacy when people utilize photo-based information services. Such services allow users to take a photo of a landmark to identify it, or a restaurant to see reviews.

While these image recognition services can be useful, they also pose privacy risks.

Along with sharing the user’s current location, such services can also use identifiers from the smartphone to link current results with past results to build a location history that contains even more private information, according to Naoko Nitta, an associate professor at Osaka. “Photos reflect private aspects of their owner, such as interests, preferences, and tendencies, which can be leaked by web-based image recognition services. To address this problem, we developed an encryption-free framework for privacy-preserving image recognition called EnfPire.”

To use the framework, the user extracts a feature from the photo. EnfPire transforms the feature before it is sent to the server. Because the server cannot uniquely identify the transformed image, it returns a set of candidates to the user, who compares them with the original feature using a simple recognizer. “With our framework, the provider of the photo-recognition services is unable to receive enough information for unique image recognition, while the user obtains the correct recognition result and its related service information,” said Kazuaki Nakamura, an assistant professor at Osaka.

Overview of the privacy-preserving framework for image recognition services. (Source: Osaka University)

In real-world experiments, EnfPire degraded the server’s recognition accuracy from 99.8% to 41.4%, while the user’s accuracy was 86.9%.

While EnfPire abstracts location information, it’s not sufficient to protect the user’s history, which could still be approximated from geographical relationships between results. To counter this, the team proposes automatically sending dummy requests from the smartphone to the server, which returns results based on the dummy requests that are automatically removed from the device without the user being aware of the process.