phone camera, Google Lens

[darkoshi]

I let my phone's stock camera app update recently, and the updated Settings screen has a new option for enabling/disabling Google Lens. I had to look up what that is. When it is enabled, the camera UI has a button for selecting Google Lens mode. In that mode, when you point your phone at something and tap the screen, the app will search for and show you info on the item you were pointing at.

At first, the thought of this was quite concerning to me, from a privacy point of view and from a data usage point of view. Does it upload the images to some server on the web? Does the app on the phone determine (on its own) what is in the photo, or is that processing done on a server somewhere else, and the results are sent back to the phone? I wasn't able to find an answer to that yet.

(I've also wondered, when you give an app permission to take photos and record video, etc., is it able to use those permissions even when the app is not in the foreground? What if you swipe the app out of your Recent Apps view - is that a sure way to make sure it isn't still spying on you in the background, or not?)

But the Google Lens functionality also sounded appealing, so I tried it out today. It presented an initial informational screen that said your images and search info would be uploaded/saved to your "Web Activity" unless you turned that option off in your Google Account. So I checked and verified that I have that turned off.

I tested it by pointing the camera at one of our drink coasters, and it came back with a product image and link to where the exact same coasters were for sale. That was pretty neat.

The next few things I tried didn't have as specific results. But it still seems a promising and useful tool. As long as I can be sure that it won't be sending image data somewhere on the internet without my knowledge.

Update: I tested using it with both wifi and data turned off. This way it doesn't give any error message, but doesn't show any results (even when scanning something simple like text). So I guess it must require uploading the images somewhere and then processing them there, not only on the phone.

.

Not long back, I installed a separate non-stock camera app, Open Camera:
https://play.google.com/store/apps/details?id=net.sourceforge.opencamera

My main reason for getting a separate camera app was that I wanted one with a manual focus option. Auto-focus sometimes focuses on the wrong thing, especially when taking close-ups of plants in windy weather.

Comments

[redsixwing]

Good test! Yeah, this has to be server-based. There's no way they're putting (probably machine learning-based, probably very resource intensive) image processing software with that kind of ability on phones (unless I badly misunderstand the capabilities of modern smartphones; mine's not a top of the line model.)

This article has some interesting tidbits. Reading between the lines a bit:

- That data is not only passed to a Google-controlled server, but the images are probably stored, and at least the relevant queries are saved to your Google identity (Edit: and also shared to other Google apps, such as Maps, to facilitate selling you things);

- They are either considering or actually implementing sharing it with other people;

- Google being Google, they may or may not tell anyone before they do that.

[darkoshi]

> but the images are probably stored, and at least the relevant queries are saved to your Google identity

Yes, this was mentioned in the initial message I got after enabling Google Lens in my camera app, but it also said that you could turn off that option under your account's "Web Activity". But I wasn't sure if the images would still be uploaded and saved somewhere else, out of your control, when you turn the option off.

If they are being uploaded (which it now seems they are), I don't trust Google to necessarily delete them afterwards. I agree with your last 2 bullets. Who knows, maybe they'll even add them to Google image search so that they'll come up in other people's search results! (Although I think image search results currently only show images hosted on webpages, so maybe not.)

It still seems very risky to me... one could accidentally click the Google Lens icon on the screen and end up uploading an image somewhere, when you meant to take a private photo, possibly of something sensitive like a document with personal info, bank account numbers; or of someone not completely dressed... not to mention things which are less sensitive but still private, like what the inside of your house looks like, etc.

> this has to be server-based. There's no way they're putting (probably machine learning-based, probably very resource intensive) image processing software with that kind of ability on phones (unless I badly misunderstand the capabilities of modern smartphones; mine's not a top of the line model.)

I suspected that too, but wasn't sure.

[redsixwing]

>But I wasn't sure if the images would still be uploaded and saved somewhere else, out of your control, when you turn the option off.

I think that's a safe assumption.

>Although I think image search results currently only show images hosted on webpages, so maybe not.

I think it's more likely they'll store them privately. It's worth noting they also introduced a face tagging system, with similar opacity about how the tags will be stored and used.

I share your concerns about privacy. This is gross and I don't want it.