First Tracking Device using Vibration, AI to track 17 Home appliances

At the present state of affairs, to track each appliance in your home you would need to install a separate tracker for each appliance. Now, what if you had 10, 20 or so appliances? That would be some expense to carry out, not so? But recently, some researchers at Cornell University have developed a single device that is able to track about 17 home appliances at the same time and this device uses vibration with an integrated deep learning network. With this device, you no longer need to worry about forgetting to take wet clothes out of the washing machine, or allowing food to remain in the microwave, or even forgetting to turn off faucets that are dripping. This device promises to make your home smart in a cost-effective way.

 

Vibration analysis has several uses in industry, especially in detecting anomalies in machinery, but this is the first use case for tracking home appliances using vibrations that I have found. This device, called Vibrosense, uses lasers to capture the subtle vibrations that are emitted by walls, floors and ceilings and then incorporates this received vibration with a deep learning network that is used to model the data being processed by the vibrometer in order to create a unique signature for each appliance. I tell you, researchers are getting closer to their dream of making our homes not only smarter, but more efficient and integrated.

But can it detect appliance usage across a house, you may ask? There are so many appliances in a house and the vibrations they emit can intersect. That’s right. The researchers have a solution to that problem. To efficiently detect different appliances in a house and not just in any single room, the researchers divided the task of the tracking device into two categories: First, the tracking device would have to detect all the vibrations in the house generally using the laser Doppler vibrometer, and second differentiate the vibrations from multiple appliances even if they were similar vibrations by identifying the path the vibrations has traveled from room to room.

The deep learning network that is incorporated in the device uses two modes of learning: path signatures and noises. Path signatures for identifying different activities and the distinctive noises that the vibrations make as they travel through the house.

To test its accuracy the tracking device was tested across 5 houses at the same time and it was able to identify the vibrations from 17 different appliances with 96% accuracy. Some of the appliances it could identify were dripping faucets, an exhaust fan, an electric kettle, a refrigerator, and a range hood. Also, when it was trained Vibrosense could be able to identify 5 stages of appliance usage using an accuracy of 97%.

Cheng Zhang, assistant professor of information science at Cornell University and director of Cornell’s SciFi Lab, on speaking about the device, Vibrosense, said that it was recommended for use in single-family houses because when it was installed in buildings, it could pick up the activities that were going on in neighboring houses. A big privacy risk one must say.

A smart device with immense benefits

When computers are able to recognize the activities going on in the home, it makes our dream of the smart home closer to reality. Such computers can ease the interaction between humans and computers, enabling human-computer interfaces that are a win-win for everyone. That is what this tracking device does. One advantage of this device is that it leverages on the use of computers to understand human needs and behaviors. Formerly, we would need separate devices for each appliance or need. But this device has leveraged on that need. “Our system is the first that can monitor devices across different floors, in different rooms, using one single device,” Zhang said.

I feel elated on discovering this device. No more having to wait for my food to be cooked on the microwave. With this device, I could be watching the TV while it watches the food on my behalf. There are a lot of things we could use this for. I think this innovation is very beneficial to the average American.

But one concern about Vibrosense is in the area of privacy. I wouldn’t want my neighbor to know when I am in the bathroom, or have the TV on, or that I was not in the house. But these are the information the device can send out.

When asked on the issue of privacy, Zhang said: “It would definitely require collaboration between researchers, industry practitioners and government to make sure this was used for the right purposes.” I hope that cooperation does come.

The device could even help in enabling sustainability and energy conservation in the home. In so doing, it could help homes to monitor their energy usage and reduce consumption. It could also be used to estimate electricity and water usage rates since the device has the ability to detect both the occurrence of an event and the exact time period that event took place. This is badly sought-for energy-saving advice that home owners need. This is great!

I was thinking about the benefits of a device like this in a typical home and was wowed by its potential benefit that I decided that this innovation needs a place in my solvingit? blog. So, this is a thumbs up to Cheng Zhang and his team at Cornell.

The material for this post was based on the paper: “VibroSense: Recognizing Home Activities by Deep Learning Subtle Vibrations on an Interior Surface of a House from a Single Point Using Laser Doppler Vibrometry.” Cheng Zhang was senior author of the paper. The paper was published in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies and will be presented at the ACM International Joint Conference on Pervasive and Ubiquitous Computing, which will be held virtually Sept. 12-17.