Amazon, Google May Want To Be A Fly On The Wall In Your Home, Patents Suggest

By Jessica Vomiero, GLOBAL NEWS CANADA

April 2, 2018

Amazon, Google may want to be a fly on the wall in your home, patents suggest

It’s expected 75 per cent of US homes will have “smart speakers” by 2020, but what exactly do they do, are they safe and are they worth it? Online security expert David Papp explains.

Amazon and Google are in race to occupy and digitize the space in our homes, but what will they have access to once they get there?

Patent filings by both companies — most of which are still under consideration — indicate that both tech giants want their smart home devices to listen more astutely to what happens in the home.

These patents were first reported by the New York Times last weekend, and were first discovered by the consumer advocacy group Consumer Watchdog in December, 2017.

Amazon and Google are the world’s two leading sellers of smart home devices, and both have claimed that their products only listen to and process audio once users activate them, either by pushing a button or by using the phrases, “Hey, Alexa,” or, “Okay Google.”

Patents propose monitoring “mischief” and “emotional state”

One set of patent applications from Google, involving its subsidiary company, Nest Labs, suggests that audio monitoring could help detect that a child is engaging in “mischief.”

The application states that a device could analyze speech patterns and pitch to identify that a child is present, sense movement while listening for whispers or silence, and give parents the option to program a smart speaker to “provide a verbal warning.” 

The filing gives the example that when children are near a liquor cabinet or are in their parents’ bedroom, the system may “infer that mischief is likely to be occurring…the system may report and/or record the findings for subsequent use,” and could then give the child a warning as a form of “deterrence.”

The patents also outline the possibility for monitoring the “emotional state” of the occupants of a home. For example, “crying may signify a sad emotional state, whereas laughing may signify a happy emotional state.” The document also discusses monitoring body temperature, audio signatures, and facial expressions.

A second set of application documents from Google indicate that “home data” can be used to provide home occupants “interesting information, products and services as well as providing them with targeted advertisements.”

In a set of patent applications filed by Amazon, the company describes a mechanism called a “voice sniffer algorithm,” that could be used to analyze real-time audio and potentially send targeted product advertisements.

This feature could be used on a wide range of devices — from tablets to e-readers. A diagram included with the patent application detailed how a phone call between friends could lead to one party receiving an offer for the San Diego Zoo, and the other seeing an ad for a Wine of the Month membership depending on the words — “love,” “great,” etc. — that each participant used to describe these experiences.

“Orwellian” implications?

Each of the patents described here was filed in 2015, though they were first discovered by the Consumer Watchdog near the end of 2017. In response, the group published a study on the”Orwellian” implications of functions like the ones described in those documents.

“These patent applications show how technology companies use home data to draw disturbing inferences about households, and how the companies might use that data for financial gain,” the study states.

“This isn’t about helping people, it’s about selling people,” said John M. Simpson, Consumer Watchdog’s privacy and technology project director in a statement. “If these patents are implemented, there will be unparalleled surveillance of our private lives.”

In a statement to the Times, Amazon said the company took “privacy seriously” and plainly denied using customers voices for targeted advertising. The company said it had filed “a number of forward-looking patent applications that explore the full possibilities of new technology.”

Amazon added that these patents take multiple years to receive and did not reflect the developments of current products and services.

Google offered the Times a similar response, saying it “did not use raw video to extrapolate moods, medical conditions or demographic information.” The company went on to call Consumer Watchdog’s claims “unfounded” and mirrored Amazon in arguing, “prospective product announcements should not necessarily be inferred from our patent applications.”

Privacy vs. progress

These patents are surfacing just as privacy and surveillance have been thrown into the spotlight. Facebook is currently dealing with backlash from the Cambridge Analytica scandal, followed by the revelation that the company may have kept logs of calls and text messages of users with Android devices running on older software.

Following the news that information from over 50 million Facebook profiles was provided to political consulting firm Cambridge Analytica, who went on to work with several political campaigns in the following years, user anger has torpedoed the company’s stock and shaved over $100 billion off its overall value.

Amazon’s Alexa is randomly laughing at people, and the company is trying to fix it

Furthermore, internet forums are convinced that our devices are already listening to us, and experts are starkly divided on whether this is true. While it is possible to delete voice recordings from Amazon Echo devices, there’s no way to do that on a Google Home speaker just yet.

To stop your Google Home to listening to you, tap the “Mute” button on the back of the device, which prevents Google Assistant from hearing anything at all. To give the Google Assistant verbal commands, you’ll need to un-mute the speaker.

Amazon’s Echo speaker doesn’t let you stop voice recordings altogether, but also it also features a “Mute” button similar to the Google Home. In addition, however, users can delete voice recordings from the Amazon Alexa app in bulk by visiting amazon.com/myx, visiting the “Your Devices” tab, and selecting “Manage voice recordings.”