Sensory Overload

There are two new buzzwords—always on and context-aware—that have the potential to transform devices and the way in which we use them. Both are related to smartphones at the moment, but this just the initial place where their impact is being felt.

Both concepts add the potential for smartphones to become part of the Internet of Things (IoT) because they enable meaningful interactions between the phone and other systems that do not require any intervention from the user. This article will be looking at some of the technology necessary to make transition happen and the types of new services that will be enabled. Today, most of these new services are being driven by the possibility of more targeted marketing and sales.

Always-on involves a device having sensors that are constantly monitoring something. As such, they must be ultra-low-power devices so they do not impair battery life. The first device to actually use this approach was the Motorola X, where the handset tracks a user’s every move and physical location keeping the accelerometer, gyroscope and other sensors powered on at all times. Apple’s recently introduced 5s phone also contains a dedicated processor for motion tracking—the M7, which utilizes a solution from NXP.

Why is this important? Consider one of the often-touted uses for the smartphone. You are in a mall and approaching a store in which you have been known to shop. Your phone provides an alert for the sale they are having and offers you a coupon good for the next 15 minutes. The problem today is that inside a mall the GPS doesn’t work, so how does the phone know where you are? The answer is that it tracks your every move and is accurate enough so that after wandering around the mall for hours, it still has you accurately placed. There are many other applications for this technology such as a pedometer or health monitor.

Growth expected in the sensor market. Source: Semico Research Corp. 06/2013

Context-awareness is also an important attribute that must be built into these devices. The device may need to know if you are walking or riding in a car. Maybe it is lunchtime and you want to find somewhere to eat. If you are walking, you need something close that matches your eating preferences and happens to have lunch specials. If you are in a car and needing an evening meal, the options may be very different. The app should probably take into consideration where you are going so that it does not require you to backtrack.

Context is something that has not been common in devices in the past and is giving researchers a few things to think about. As humans we utilize context in many ways and have a lot of social rules for defining context. Without these we would be getting a lot of unwanted or even erroneous information.

So, what does context awareness entail? In a recent blog written by Frank Shemansky, VP of Business Development at Sensor Platforms, a company specializing in context-aware algorithms, he talked about the four stages in the process. The first stage is the selection of the sensors. They use an accelerometer as an example, which they say can be sampled at 50Hz using less than 0.02mWh of energy.

Clearly, having the sensor on all the time is not the issue. The next stage is sensor acquisition. If this is performed on the apps processor, it will be drawing 180mWh of energy and they note that even on a Cortex M3 processor, it would draw 50mWh (This is the processor in the Apple M7). But that is not the end of the story. Even after the data has been acquired from the sensors, processing has to be performed so that a change in context can be detected. This is the only time when it is worth waking the actual application processor.

Quicklogic conducted a user survey to find out how much power draw would be acceptable for sensors and processing. They came back with a figure of 1% to 2% of the battery capacity, and using their estimate of 16 hours between charges, they calculated a maximum acceptable draw of 10.2mW. Finally, the apps processor can decide if the change of context has significance given the current tasks being performed. As Brian Faith, Quicklogic’s director of worldwide sales and marketing, put it in a recent interview, “Waking up a GHz processor to manage Hz sensor data just doesn’t make sense.”

Source: Sensor Platforms showing sensor output. The vertical bars show when a change of context is being detected. Once context is stable no further changes sent to apps processor.

Two companies have released ultra-low-power FPGA devices to tackle this problem in the past month—Quicklogic and Lattice Semiconductor. Not surprisingly, there is a lot of commonality in their stories. They both point to the fact that general-purpose CPUs draw too much power, but fixed-function ASIC devices do not provide the flexibility necessary to keep up with algorithm development in this area. Nor can they be modified based on the current context.

The Quicklogic Flexible Fusion Engine comes in a 2×2.5mm package and sips just 300µW. In the past, Quicklogic has been known for their one-time-programmable logic, which reduces power consumption compared to SRAM-based FPGAs. However, Quicklogic realized that for logic related to I/O, they needed a new type of technology that provided greater levels of reprogrammable. Even with SRAM cells, it consumes 50 µA in standby and can be loaded from externally available flash memory long before the main processor would be ready for any data. Significant parts of the functionality are implemented as hard cores, such as the I2C controller, leaving the full 1k LUTs available for context change algorithms. They also have a simpler device that does not perform context change detection, but can still reduce the power consumption in the main processor by buffering sensor data in a 10 second buffer (at 50Hz sample rate) and delivering it as a burst.

The Flexible Fusion Engine can use context change detection algorithms supplied by Sensor Platforms. For companies that want to develop their own algorithms, Quicklogic provides a full software suite that enables the design, simulation and mapping of those algorithms onto the device.

Lattice is addressing the problem with some even smaller devices. The smallest measures 1.4mm x 1.48mm x 0.45mm and is suitable for an IR subsystem, or a larger package up to 1.71mm x 1.71mm x 0.45mm for the full sensor management system. This larger package comes in a range of logic sizes, from 1K to 4k LUTs. In addition, they have even larger devices for markets other than the smartphone market that would provide additional sensor capabilities. Joy Wrigley, product line manager for ultra-low density solutions at Lattice Semiconductor, says that the third dimension is becoming more important as smart phones are stacking chips to make better usage of the available room. This device sips less than 1mW. They do not quote a standby power because the device is intended to be running all of the time.

In a demonstration of the device, power consumption does not vary much between sensor data changing and remaining constant and came in at .88mW. Gordon Hands, Lattice’s director of marketing for ultra-low density solutions, says that constant power is an artifact of the architecture of the fabric and this has been optimized for typical types of application. In addition to the programmable logic, they also have hardened other functions such as the I2C interface, meaning that the LUTs are all available for sensor management functionality.

Given reports about sensor errors on some phones, you have to wonder about the problem of cumulative errors, although undoubtedly this will improve over time. Also, relying on things such as pressure sensors to tell which floor of a mall you are on, means those sensors have to be very accurate. They also have to account for changing atmospheric weather conditions. It appears as if there may be a way to go before we reach sensor nirvana, but it is heating up to be one of the next battlegrounds of the smart phone market.