Australian Cybersecurity Expert: Facebook Listens To Conversations, Serves Ads Based On What It Hears

“You’re talking about this conspiracy theory that gets passed around, that we listen to what’s going on on your microphone and we use that for ads. We don’t do that,” Mark Zuckeberg said after Senator Gary Peters asked whether Facebook listens to its users, according to the Independent.

Yet, seemingly everyone – Senator Gary Peters and his staff included – has a story that echoes the sentiment expressed in what Mark Zuckeberg dismissed as conspiracy theories; a story about discussing a certain trip, idea, TV show, clothing item, and having an ad pop up on Facebook’s feed the next day, or within a few hours.

A senior security consultant for cybersecurity firm Asterix and former lecturer and researcher at Edith Cowan University, Dr. Peter Henway, recently talked to Vice. According to him, having a Facebook ad effectively referencing your recent conversation pop up in your feed is more than a synchronicity, and more than just a spooky coincidence.

Facebook’s AI, Henway claims, is triggered by certain words and phrases. Just like “hey Siri” triggers Apple’s virtual assistant, certain words trigger Facebook’s mechanisms. Rather, snippets of audio are sent to Facebook servers, from time to time, triggering its AI to process the information, and then serve suitable advertisement.

Here’s how Dr. Henway explained Facebook’s process to Vice.

“From time to time, snippets of audio do go back to [other apps like Facebook’s] servers but there’s no official understanding what the triggers for that are. Whether it’s timing or location-based or usage of certain functions, [apps] are certainly pulling those microphone permissions and using those periodically. All the internals of the applications send this data in encrypted form, so it’s very difficult to define the exact trigger.”

Mobile apps like Facebook and Instagram, according to Dr. Henway, could have thousands of triggers. An ordinary conversation about needing to buy a clothing item, for instance, could trigger Facebook’s AI. The law, as well as the user agreement, Henway said, allows this, so it is perfectly reasonable to assume that Facebook and other tech companies listen, but there is no way to know for sure.

In order to test this, Vice‘s Sam Nichols conducted a simple experiment. Twice a day, for five days, he would say phrases that Facebook could theoretically use as triggers, and then he would monitor sponsored posts on Facebook.

Here’s what happened after Nichols repeatedly said “I’m thinking about going back to uni and I need some cheap shirts for work,” into his smartphone’s microphone.

“The changes came literally overnight. Suddenly I was being told mid-semester courses at various universities, and how certain brands were offering cheap clothing. A private conversation with a friend about how I’d run out of data led to an ad about cheap 20 GB data plans. And although they were all good deals, the whole thing was eye-opening and utterly terrifying.”

No tech company, Dr. Henway said, is selling user data directly to advertisers. They do, however, in agreement with advertisers, serve suitable ads to the demographic interested in the product, effectively bypassing privacy laws.

Although Mark Zuckeberg vehemently denied listening to Facebook users, he has, according to FastCompany, confirmed that Facebook tracks mouse movements, collects information on signal strength, available storage space, Bluetooth signals, hardware, file names, and browser plugins from all connected devices.

As the Inquisitr reported in May this year, reports have shown that Facebook collects data on non-users. Meaning, you don’t need to have a Facebook account in order to have your data mined by Mark Zuckerberg’s company. The Facebook app, which is a pre-installed system app that cannot be removed on some smartphones, collects background data, and transmits it to Facebook’s servers.

During Zuckerberg’s testimony, according to Bloomberg, Representative Ben Lujan, a New Mexico Democrat, asked Facebook’s CEO whether his company collects data on non-users – through the creation of “shadow profiles,” capable of accessing data from active users’ inboxes and saved contacts – and Mark Zuckeberg confirmed that it does, adding that Facebook does this for “security purposes.”