By Apu Kapadia
Editor’s note: This article has been updated to clarify that future Facebook augmented reality glasses will not necessarily be in the Ray-Ban Stories product line and that, while the company’s goals include identifying people, the Ego4D research data was not collected using facial recognition technology.
Facebook’s smart glasses ambitions are in the news again. The company has launched a worldwide project dubbed Ego4D to research new uses for smart glasses.
In September, Facebook /zigman2/quotes/205064656/composite FB -2.33% unveiled its Ray-Ban Stories glasses , which have two cameras and three microphones built in. The glasses capture audio and video so wearers can record their experiences and interactions.
The research project aims to add augmented reality features to smart glasses using artificial intelligence technologies that could provide wearers with a wealth of information, including the ability to get answers to questions like “Where did I leave my keys?” Facebook’s vision also includes a future where the glasses can “know who’s saying what when and who’s paying attention to whom”.
Several other technology companies like Google /zigman2/quotes/205453964/composite GOOG -2.67% /zigman2/quotes/202490156/composite GOOGL -2.69% , Microsoft /zigman2/quotes/207732364/composite MSFT -2.44% , Snap /zigman2/quotes/205087158/composite SNAP +0.20% , Vuzix /zigman2/quotes/205105106/composite VUZI -6.51% and Lenovo /zigman2/quotes/205368244/delayed HK:992 -1.54% have also been experimenting with versions of augmented or mixed reality glasses. Augmented reality glasses can display useful information within the lenses, providing an electronically enhanced view of the world. For example, smart glasses could draw a line over the road to show you the next turn or let you see a restaurant’s Yelp rating as you look at its sign.
However, some of the information that augmented reality glasses give their users could include identifying people in the glasses’ field of view and displaying personal information about them. It was not too long ago that Google introduced Google Glass, only to face a public backlash for simply recording people. Compared to being recorded by smartphones in public, being recorded by smart glasses feels to people like a greater invasion of privacy .
As a researcher who studies computer security and privacy , I believe it’s important for technology companies to proceed with caution and consider the security and privacy risks of augmented reality.
Smartphones vs. smart glasses
Even though people are now used to being photographed in public, they also expect the photographer typically to raise their smartphone to compose a photo. Augmented reality glasses fundamentally disrupt or violate this sense of normalcy. The public setting may be the same, but the sheer scale and approach of recording has changed.
Such deviations from the norm have long been recognized by researchers as a violation of privacy. My group’s research has found that people in the neighborhood of nontraditional cameras want a more tangible sense of when their privacy is being compromised because they find it difficult to know whether they are being recorded.
Absent the typical physical gestures of taking a photo, people need better ways to convey whether a camera or microphone is recording people. Facebook has already been warned by the European Union that the LED indicating a pair of Ray-Ban Stories is recording is too small.
In the longer term, however, people might become accustomed to smart glasses as the new normal. Our research found that although young adults worry about others recording their embarrassing moments on smartphones, they have adjusted to the pervasive presence of cameras.
Smart glasses as a memory aid
An important application of smart glasses is as a memory aid. If you could record or “lifelog” your entire day from a first-person point of view, you could simply rewind or scroll through the video at will. You could examine the video to see where you left your keys, or you could replay a conversion to recall a friend’s movie recommendation.
Our research studied volunteers who wore lifelogging cameras for several days. We uncovered several privacy concerns – this time, for the camera wearer . Considering who, or what algorithms, might have access to the camera footage, people may worry about the detailed portrait it paints of them.
Who you meet, what you eat, what you watch and what your living room really looks like without guests are all recorded. We found that people were especially concerned about the places being recorded , as well as their computer and phone screens, which formed a large fraction of their lifelogging history.
Popular media already has its take on what can go horribly wrong with such memory aids. “ The Entire History of You ” episode of the TV series “Black Mirror” shows how even the most casual arguments can lead to people digging through lifelogs for evidence of who said exactly what and when. In such a world, it is difficult to just move on. It’s a lesson in the importance of forgetting.