Carnegie Mellon University

Sense device

February 24, 2017

Making the smart connection

By Alexandra George

Mike Phillips understands the power of being connected—to devices, other people, and the surrounding environment. An electrical and computer engineering (ECE) alumnus and co-founder and CEO of Sense Labs, Phillips has expertise connecting people to technology through speech recognition. So when he and his colleagues decided they wanted to tackle the energy problem by connecting consumers to their homes, he applied that expertise to develop Sense: a home energy monitor for users to track the energy consumption of their appliances.

“We have a mobile application that lets you know what is going on in your house,” said Phillips. “Think of us as providing the broad-based sensing which is needed to drive intelligence of the eventual smart home.”

The hardware part of Sense works like an electric meter, only rather than taking measurements for a monthly electric bill, it measures incoming power one million times per second. Then, Sense uses signal processing and machine learning to determine the electrical signatures of the various devices in a home. Connecting to a home’s electrical panel and using existing sensors to detect appliances, Sense delivers a report to its smartphone app—identifying what’s on or off, what state each appliance is in, and how much power each appliance uses.

“When we first developed Sense,” explained Phillips, “we had a general notion that if we could know in detail how all the devices in homes use energy and could interact with their owners or inhabitants, we could treat energy as a closed-loop optimization problem—what is the most effective thing the system can do to make the building work well?”

While Phillips currently spends his time putting consumers in touch with their homes, he began his career by pioneering the way to speech recognition software. He first became involved in the area as an undergraduate at Carnegie Mellon University when he joined his advisor, ECE Professor Richard Stern, at the forefront of speech recognition research. 

“It was the very early days of speech recognition technology, and the problem seemed almost impossible,” said Phillips. “But, we knew that since humans could indeed perform speech recognition, then it must be possible.”

Phillips continued researching speech recognition at Carnegie Mellon for a few years before becoming a research scientist at the Massachusetts Institute of Technology (MIT). His group was one of the first to combine speech recognition and natural language processing to create early dialog systems—kind of like Siri, but on a refrigerator-sized computer with a longer response time.

Phillips later transitioned out of academia to found two speech recognition companies: SpeechWorks in 1994 and Vlingo in 2006. SpeechWorks focused on speech recognition for telephone customer service systems, and Vlingo focused on the combination of speech recognition and the natural language processing work he did at MIT, this time running on mobile phones in real systems. When Vlingo was bought in 2012, Phillips and some of his colleagues decided that they wanted to start another company, but this time it wouldn’t be related to speech recognition.

“We got interested in the energy space—realizing that something like 38% of world energy is used in buildings,” said Phillips. “We like consumer-facing applications (based on our Vlingo experience) so we decided to focus first on residential energy consumption.”

In the long-term, Phillips hopes that Sense will drive energy efficiency and perform functions like fault detection. While the focus of Sense is not on speech recognition, the technical aspects are largely similar.

“When we started Sense, it reminded me very clearly of the early days of my work on speech recognition at Carnegie Mellon—a super hard technical problem that seemed almost impossible,” said Phillips. “We are now far enough along that we have a pretty clear path ahead of continuous improvement of the technology—just like what we started at Carnegie Mellon many years ago.”