> When will it be that you cannot even say a word to a person in-person anymore because it lands in a weakly secured cloud by the microphone inside their smartwatch that runs a weather app that is run by crooks.
I believe it can already be the case with Google’s (and possibly Facebook’s) apps on Android—at least in case of Google I witnessed real-life tests showing how saying something in presence of the phone causes related ads and content to be shown in feeds—but it’s scarier with less scrupulous app maintainers.
Disclaimer: I am not claiming that Android API grants all apps unauthorized access to always-on mic. The device in question was configured to enable continuous listening by the owner. I am not claiming voice recordings are stored or used in nefarious ways.
An owner of a Google-branded Android phone demonstrated it to me last year, it worked remarkably fast. IIRC, the example chosen was Google News.
I can’t see how this is fear mongering. Device was not acting against owner’s will: that person specifically does not tighten relevant privacy settings and enables always-on mic because they see useful content suggestions as a feature.
It is different from my outlook (and I take it yours too), but I think that point of view deserves the right to exist.
The issue is that now when I am talking to someone like that in real life, I know I may be implicitly agreeing that Google would pick up my speech as well. It’s a clash similar to one between a person (e.g., my mom) who grants messenger apps full access to their phone’s address book for convenience, and their contact (e.g., myself) who does not want their information to be shared with Facebook et al.
Addendum: a cursory search confirms this experience.
— Counterpoint: https://www.bbc.co.uk/news/technology-49585682 “phones that secretly listen to us are a myth” (2019). Notably, with all my respect for BBC, this article lacks a lot of context (did they turn off voice activation before testing? what were their privacy settings?), and to test devices listening they played commercials in presence of the phone rather than using voice (I have relatively high confidence that modern devices can easily distinguish between speech nearby and sound from a commercial before recording leaves the phone).
Some sources claim that as of last year Google is changing the way their apps work. Not sure if they stop listening or stop showing relevant content.
(I don’t personally own an Android phone, and even if I did I would have turned off virtual assistants & disabled always-on mic just as I do with my iPhone, so I wouldn’t be able to present first-hand proof.)
What's described in the article is hotword-activated? So not at all what GP is supposing. If you say "Hey Google, tell me about toilet paper" and get ads for toilet paper, that's a fairly understandable cause => effect, but there are persistent anecdotes about conversations manifesting in ads where no hotword activation occurs (typically about Facebook.)
I find this hard to believe because I’d think constant voice recognition would either have a noticeable impact on battery life or it’d have a major impact on data usage. Also, on iOS at least, it’d have to be provided by Apple, to be a constant background thing, and then Apple would already be using it for Siri.
If always-on mic for virtual assistant activation is enabled, it does impact battery life.
Regarding iOS, I hadn’t observed ads obviously based on what I spoke about in presence of my iPhone, but then I don’t use voice-activated Siri and generally tighten up privacy settings.
I’ve never noticed this myself: I’ve always assumed that what is actually going on is that people’s phone usage is more correlated with what they’re thinking/talking about than they realize and ad companies have gotten pretty good at uncovering these latent connections (e.g. the story about Target deducing someone was pregnant from seemingly unrelated shopping patterns).
I remember that story. However, what I observed with Android’s Google (or is it called Google News?) app last year was a tight feedback loop: after talking a little about %SUBJECT% near the phone, and refreshing the feed within the next minute or two, a relevant article from past few days showed up.
(Similar to Vice’s article I linked, but faster.)
Again, the owner of the device saw that as a convenience feature and consciously did not set the phone up to prevent it, which made me feel a little old-fashioned and unnecessarily paranoid.
Also, unlike Vice’s article, in the scenario I have witnessed the recording did not necessarily have to leave the phone: the news app could have kept a large cache of recent articles and locally pick the ones matching the %SUBJECT% that we spoke about.
I am inclined to believe that Google, given their business model and scale, is unlikely to store voice data insecurely or insufficiently de-anonymized, so I’m primarily worried about third-party apps getting access to always-on microphone without visual feedback. (Hopefully it’s not very likely and app stores have tools to detect nefarious uses of relevant APIs at review stage.)
I’d like to see an actual technical write up of this: network logs, tracing of the android device activity etc. My original impression was that the reason why mobile voice assistants have trigger words is that anything more complicated isn’t feasible as an always-on feature. (Although, I do remember stories about the Facebook app using the microphone to suggest that you post a status update about the movie or tv show you’re watching, so maybe it’s more feasible than I imagine).
As a person who's made a living the last few years working in the guts of Android on embedded devices, there are so many holes in this way too common myth that phones are listening all the time.
You don't even need to dive into the technical aspect of it, what on earth is the risk reward here?!
Risk: Forever break the trust people have in your devices, this isn't some grey area intrusive tracking that would just get swept under the rug...
Reward: Get noisy info about people's interests when you literally own the device that contains more information about than their own short term memory does!
It's nonsensical, and there's no way that Google could do this that wouldn't already have been caught.
I mean is the theory that all Google devices do it and somehow no OEM has realized their microphone is getting accessed? (Because even with the lowest level access on the device, modern microphones are not so unsophisticated, there's no universal way to access it in a way a manufacturer wouldn't catch onto sooner or later
Or Google did this but only on phones they own or something?
There is no proof it happens, and no proof it doesn’t happen, because it’s non-trivial to detect based on network activity. The only evidence is observing content relevant to what was being spoken about being suggested across apps.
> Risk: Forever break the trust people have in your devices, this isn't some grey area intrusive tracking that would just get swept under the rug...
Reward: get people to love your services for relevant suggestions. Believe it or not, there are people outside the extra privacy-conscious bubble who do not at all mind their devices listening.
> Or Google did this but only on phones they own or something?
I am pretty sure this depends on software. I have seen this demonstrated on a Google-branded phone with a Google app.
> Reward: get people to love your services for relevant suggestions. Believe it or not, there are people outside the extra privacy-conscious bubble who do not at all mind their devices listening.
Like I already pointed out this is nonsense.
Always on listening even with perfect parsing would INCREDIBLY noisy. There are a million and one reasons for a term to come up in speech. The simplest conversation could surface hundreds of targeting terms.
Meanwhile they literally own the device and the services most people use. They have your search, they have your email, they have social graphs. They can literally make inferences before you even think to talk about them with other people! (and we've seen this happen before with things like disease and pregnancy reveals)
We're at the point where most people's cell phones hold more personal data than they could even recall on demand.
So why on earth would they go and muddy all that easily weighted data with noisy data like everything you say, literally every other form of interaction is already giving them better more concise information about you...
-
> There is no proof it happens, and no proof it doesn’t happen, because it’s non-trivial to detect based on network activity. The only evidence is observing content relevant to what was being spoken about being suggested across apps.
I can't believe people are entertaining this kind of stuff on HN.
You make an unreasonable claim... then act like because you yourself can't prove your unreasonable claim it should be entertained? What?
That's not how that works. You have no actual proof for your unreasonable claim... then that's it. It ends there. The burden doesn't suddenly fall on others to prove the contrary!
Come back with even a modicum of proof. Literally any real proof other than anecdotes where the ad companies who literally have almost all the data in your life anyways are able to come up with topics you're interested in... and maybe someone will entertain this.
And no, talking about something and getting an ad for it after is not proof any more than having a leaf fall on your head while you stand under in a mid-autumn forest right after you whispered "gravity" is proof that the forest is listening to your words.
> I’d like to see an actual technical write up of this: network logs, tracing of the android device activity etc.
FWIW there’s a technical paper[0] that summarizes existing studies as of 2019, and it’s been neither definitively proven nor disproven that it happens. Turns out it’s not at all that trivial to detect.
From the paper:
> Perhaps most importantly, Pan et al. were not able to rule out the scenario of apps transforming audio recordings into less detectable text transcripts or audio fingerprints before sending the information out. This would be a very realistic attack scenario. In fact, various popular apps are known to compress recorded audio in such a way [10, 33]. While all the choices that Pan et al. made regarding their experimental setup and methodology are completely understandable and were communicated transparently, the limitations do limit the significance of their findings. All in all, their approach would only uncover highly unsophisticated eavesdropping attempts. …
> Therefore, the fact that no evidence for large-scale mobile eavesdropping has been found so far should not be interpreted as an all-clear. It could only mean that it is difficult – under current circumstances perhaps even impossible – to detect such attacks effectively.
(Apparently, noticing relevant content being obviously suggested is the only way of detecting it at this time, and of course it comes with its own caveats.)
Well, I'm fairly confident that there'd be a lot of online noise about the iPhone's orange dot being on all the time, the way there was about Clipboard notifications.
I wonder if there is an equivalent of the orange dot on Android.
For sure, it’s an arms race between ecosystem’s root vendor and app developers, but the possibility of vendor itself using some privileged APIs that do not provide visual feedback is also a concern.
To hear "Ok Google" it need to record everything and process everything. Adding "toilett paper" as a processing keyword would not be noticable on battery life.
Which one? Vice’s article and some answers on Quora imply continuous listening without engaging a virtual assistant.
In case of my friend showing me this, this happened a few months ago and I can’t remember exactly how the demonstration went. I’m inclined to believe there was no hotword activation, as I remember myself being quite startled (at that point I disbelieved that a phone can be listening and suggesting relevant content right away), and as you noted with hotword activation it would have been markedly less surprising.
> For your smartphone to actually pay attention and record your conversation, there needs to be a trigger, such as when you say “hey Siri” or “okay Google.” In the absence of these triggers, any data you provide is only processed within your own phone. This might not seem a cause for alarm, but any third party applications you have on your phone—like Facebook for example—still have access to this “non-triggered” data. And whether or not they use this data is really up to them.
Isn’t this pretty much how voice-activated virtual assistants work? Microphones have to be listening in order for devices to respond to “Hey Siri” and “OK Google”, and it does impact battery life.
I think they have some kind of special optimized chip that can listen for only a specific phrase at very low power and wake the rest of the device when it hears it. It seems super unlikely that they can listen constantly to anything anyone says, pick out things that can be advertised for, and show ads for those things the next time the user browses without eating lots of power and data.
The wake phrase is different in different languages, yet they sell the same hardware to everybody. Therefore, obviously, the wake phrase is reprogrammable. It isn't baked into the silicon.
I think the combination of ever-growing lithium-ion battery resource, hardware energy efficiency and performance with 7 and 5nm processes, and improving on-device speech recognition makes it possible with little to no perceived battery life degradation.
If this were true, I think a single person, ever, would have been able to furnish evidence of how this actually works.
Since complex voice-recognition (of other than the activation hotword) is done off-device, you will be able to see network traffic as a result of this occurring. That's quite simple to check for.
> complex voice-recognition (of other than the activation hotword) is done off-device
Wrong. It’s been feasible to do on-device for years[0][1].
> That's quite simple to check for.
It’s also quite simple to check that it happens, if you have a Google phone. It’s been done in front of me last year[2]. It’s been demonstrated to happen by other people than me, so you don’t have to rely on my word here[3].
> a single person, ever, would have been able to furnish evidence of how this actually works.
I don’t think it’s easy to show how this works under the hood, since speech can be recognized on device and devices communicate with remote services very verbosely over HTTPS (probably with certificates pinned to prevent MITM) making it non-trivial to distinguish that traffic from background network activity. Recognized speech data doesn’t have to be communicated in real-time, in fact it would make sense to wait and batch it with other requests for efficiency.
(There’s a technical paper[4] that summarizes research in this direction as of 2019, and turns out it’s not trivial to definitely prove or disprove based on network activity.)
While the technology is now somewhat accessible (as of 2019 in both cases you mention), this conspiracy theory dates back nearly to the introduction of smartphones. I have heard it as early as 2011.
A number of other anecdotal experiments, such as one performed by myself, failed to show this behavior. A more tightly controlled but still informal experiment by a vaguely related security firm failed to find this behavior[1]. An academic effort by researchers at Northwestern failed to find this behavior [2]. This is by far the most thorough academic effort on the topic I have seen.
Facebook has clearly denied it [3]. Google has not issued such a clear statement but has been reasonably open about changes in their policy on voice data [4]. After considering the issue from several angles, P. J. Vogt concluded that no such thing happens [5]. Even the paper you cite notes the total lack of evidence.
Perhaps most notably, almost all of the popular media reporting that bears headlines saying that your phone is listening to your conversations, actually say no such thing when you read the article. Instead they are talking about analytics on voice assistant activations and, frequently, voice memos in Facebook Messenger. Amusingly I've run into two cases where popular press had to issue retractions or corrections after they said that smartphones were always-listening.
The only serious sources I have ever seen assert that this is happening are Vice's Sam Nichols based on Dr. Henway. Henway makes some very specific claims to two different reporters but provides no explanation of how he came to that knowledge. To an almost comical extent, nearly all reporting in favor of this theory (that even claims to have a source) is based around the exact same quote from Henway, who has never published anything formal on the matter or even really elaborated beyond a single paragraph. Nichols only performs a very basic experiment and it is easy to come up with other ways he may have gotten the result he did - in fact, the experiment he performs is nearly identical to the ones performed by others that have failed to show results.
Look, I'm not totally unreceptive to the idea that this is happening, but I don't like people repeating the assertion-as-fact that it is a widespread behavior when major tech companies that have denied it, and no real evidence has ever been amassed to show that it does happen.
Just my opinion, but... well, just all of our opinions. Let's be careful about calling them facts.
I did not claim any facts beyond reporting my observations.
As to facts, I am not a security researcher myself, so I linked to articles when I encountered such a vehement rejection of my personal experience.
If you are calling me out in what I observed, then you are saying I am delusional or lying.
Google is changing their policies and the News app behavior, so I don’t know whether the experiment I witnessed is reproducible anymore. but I am reasonably sure that they would choose to stop showing relevant content as obviously but keep mining data, as it ultimately aligned with their business model.
> A friend and I were sitting at a bar, iPhones in pockets, discussing our recent trips in Japan and how we’d like to go back. The very next day, we both received pop-up ads on Facebook about cheap return flights to Tokyo.
> A private conversation with a friend about how I’d run out of data led to an ad about cheap 20 GB data plans
> Suddenly I was being told [sic] mid-semester courses at various universities
I absolutely belive Facebook can find system information like data caps, or can read notifications (as they explicitly ask for this to auto-fill SMS-based 2FA logins). As far as being in the same location as your buddy you took a trip with, that's a lack of imagination on the advertisers part - you get ads to return to Japan the same way you get ads for the vacuum you just. bought. As far as the writer possibly going back to school? I'd say many writers enjoy writing - so many, in fact, that prices have been depressed for decades. I'd assume many writers have to return to school and change careers.
Is it possible Facebook is listening? I won't dismiss it without at least reading the article. But the linked article reads like the author believes people are unique and, while they are, they're also far more predicatable than we like to pretend.
I believe it can already be the case with Google’s (and possibly Facebook’s) apps on Android—at least in case of Google I witnessed real-life tests showing how saying something in presence of the phone causes related ads and content to be shown in feeds—but it’s scarier with less scrupulous app maintainers.
Disclaimer: I am not claiming that Android API grants all apps unauthorized access to always-on mic. The device in question was configured to enable continuous listening by the owner. I am not claiming voice recordings are stored or used in nefarious ways.