Ignoring Anomalies

Everyone has been in this situation at some point in their career—you have a data point that is so far out of the ordinary that you dismiss it as erroneous. You blame the test equipment, or the fact that it is Friday afternoon and happy hour started 10 minutes ago. In most cases it may never happen again and nobody will ever notice that you quietly swept it under the rug.

But in doing so, you may have ignored a very important bug or missed out on the discovery of something that will send your work in a total new direction. Such an event happened to me. It also happened to Rosalind Picard, founder and director of the Affective Computing Research Group at the MIT Media Laboratory, who provided the Thursday keynote at DAC this year, titled “Emotion Technology, Wearables and Surprises.”

My event happened very early in my career as an electronic engineer. Back then we created breadboards of the hardware for debug and early software development. I was working on one channel of a flight control computer for the Airbus program. One of the breadboards would occasionally lock up. My boss asked me to get to the bottom of it. I found that the prototype had a weak power supply, and it would have been easy to stop there and just replace the defective unit.

However, I continued to look at why the voltage droop would cause a lockup. It was impossible to bring the processor out of that state, once entered, except by a complete system reboot. The processor was the Motorola 6800, and what I found was that random data was being placed on the databus during an opcode fetch when the voltage droop happened. If the op-code 0x9D ever happened to be read by the processor, it sent the processor into a test mode that it would never come out of – not even by using a non-maskable interrupt.

The result of finding this was that every avionic and military system in the UK that used the 6800 processor had to be recalled and retrofitted to prevent this from happening. Motorola also changed the design to ensure that it took a sequence of instructions to enter a test mode.

Rosalind Picard at her keynote. Photo: Semiconductor Engineering/Brian Bailey

The anomaly that Picard found was a single datapoint. It led to a very interesting discovery. What follows are excerpts of her talk:

We failed to recognize the anxiety and stress that people on the autism spectrum were feeling. A person with autism may outwardly look calm and disconnected when inwardly they are about to explode. We measured student’s Electro Dermal Activity (EDA) over the course of seven days. Cognitive load and emotional load make the signal go up. Engagement makes it go up and we see huge peaks during study and during labs. To the embarrassment of MIT professors, the low point every day is classroom activity. It was also surprising that for this person, their biggest peak is during sleep.

One day a student came to me and asked to borrow a sensor for his little brother who has autism. I gave him two sensors, and he put one on each of his brother’s wrists. I expected he would use one, and then the other when it broke, because they were hand wired and frequently broke.

On the fourth day of data collection, one of the EDA bands went so high that I thought the sensor must be broken. We have stressed people out at MIT in every way imaginable and never seen a signal go this high. My first impression was that it must have been broken. Furthermore, the other side was not responding at all. Both sensors’ data looked perfectly normal both before and after.

I asked what had happened at the date and time of the spike and he said that it was right before a Grand Mal Seizure. I talked to a brain surgeon at Boston Children’s Hospital and asked if it is possible that a patient could have a huge sympathetic system surge before a seizure? He responded – probably not, but we have had patients whose hair stands on end on one arm before a seizure.

This led us to look at Sudden Unexplained Death in Epilepsy (SUDEP). It is known that the brainwaves at the time of seizure go crazy and before SUDEP the brainwaves in the cortex go flat while the person is still breathing and heart still beating. Then respiration stops and then the heart. This signal on the wrist gets bigger, the longer the brainwaves were suppressed. This is happening while the brain is shutting down. One possible explanation is that when you stimulate the amygdala, you get ipsilateral response. These signals may not show on the cortex. So the person can look brain dead even when there is a huge amount of activity going on deep in the brain. That activity is picked up on the wrist and not on an EEG.

If people are flipped over when they are having a seizure they are less likely to stop breathing or to restart even if they have stopped. The device is already approved in Europe and is going through FDA approval.

What started as a suspected mistake in the data is now saving lives. Since then we have found other mappings between activity in the brain and activity on the skin and we still have a lot to learn.

Don’t ignore anomalies
So the next time you see an outlier in a dataset, don’t ignore it. That outlier may be the indicator of a non-linearity, or a new discovery, or an area of research that may be a lot more important than the original direction you were going in.