Where do we come from?

In 1895, the public could discover the first real movie, the entry of a train pulled by a steam locomotive into the train station of La Ciotat, a village on the French Mediterranean coast. The film consists of a continuous 50-second real-time shot. At that time, the train heading towards the audience appeared so realistic that people screamed. This movie was a two-dimensional, black, and white rendering of the reality. Nothing really innovative appeared until 1935, when the film “Becky Sharp” was released. It is generally regarded as the first “color movie” (using the technology of three-colour Technicolour).

This user experience, two dimensional and color rendering of the reality, is basically the one which is still predominant today. In the 50s there was a craze about 3D stereoscopic films (House of Wax, Creature from the Black Lagoon…), requiring audiences to wear inferior, red-and-blue (or red-and-green) glasses. But soon the craze declined, since the technology was complex and expensive, and critics claimed that films were “hard on the eyes”.
At the end of the 20th century, there was a resurgence of 3D (remember the IMAX), which became more mainstream in the 2000s. But today, the bulk of the movie industry revenue is still coming from 2D presentations.
For sure, there have been several concerns and challenges for mass adoption of 3D TV and cinema. A notable one is that viewers can experience sickness which causes symptoms similar to the well-known motion sickness, such as headache, nausea, fatigue, and so on. These are caused by a disconnect between what someone can see and what their body perceives, thanks to the vestibular organs which are our motion detectors.

Augmented and Virtual Reality on the move

Eventually, the 3D experience has been adopted by the digital world, so users can get a more immersive experience. Technologies in Augmented Reality (AR) and Virtual Reality (VR) have emerged. Although similar, there is a major difference between them.
Virtual reality refers to a fully immersive experience in a virtual world, while augmented reality refers to a modified view of the real world. Both require a special device, which can take the shape of a head-mounted display (such as Oculus Rift for VR) or smart glasses (such as the now defunct Google glasses for AR).
The main difference is that AR requires a see-through device to allow the combination of virtual objects with the real world that the user can see through the glasses.
Video game companies started to release virtual reality headsets in the early-mid 1990s. Beginning in the 2010s, next-generation commercial headsets were released by Oculus (Rift), HTC (Vive) and Sony (PlayStation VR), triggering a new wave of applications beyond 3D cinema and gaming, such as VR exposure therapy for treating anxiety disorders, anatomy teaching, astronaut training, flight simulators, architectural design, bridge inspection, visual prototyping and so on. However, these are specific use cases, and mass adoption has yet to take place.

But mass adoption faces challenges

There have been several challenges concerning VR beyond the traditional 3D sickness, which probably explains the slow proliferation of the technology. One crucial problem is the Vergence-Accommodation Conflict (known as VAC), when one’s brain receives conflicting information about the distance between the eye and 3D object (the vergence) and the focusing distance (the accommodation).
For the accommodation, the eyes use a lens (the so-called crystallin) which can change its focus with special muscles that pull or relax the lens. This reflex gives an important depth cue for objects that are close to us (less than 10 meters).
On the other hand, our two eyes can move independently. When we are looking at an object they converge to it, which gives another depth cue. In the real world, our eyes (a true marvel of engineering) have a vergence-accommodation reflex, that is, accommodation and vergence are always in sync. In VR headsets, the virtual image is focused at a fixed depth away from the eyes, while the depth of the virtual objects varies with the content, hence the conflict.
The author of the article has tried several VR applications (for instance flying over the city of Paris) and has systematically experienced these symptoms, despite being experienced in flying real aircraft.
Naturally, the AR headsets are currently subject to the same limitations, and the challenge is even more acute: the virtual images do not blur correctly with the real world which can be seen through the glasses.

Solutions are on the horizon

A first step has been to use more than one plane (two actually) in the headsets, but that is just a partial solution. Promising technologies to solve the problem do exist. One is light field technology, which projects images with real optical depth. Therefore eyes can naturally change focus between virtual objects at different distances, and these objects can be seamlessly fused into the real world.
Simply speaking, when a user focuses on a real person through the glasses, but is wearing a virtual hat, both the person and the hat will appear crisp. If the user focuses on the horizon, both the person and the hat will appear blurred, as it should. CREAL, a Swiss company based at EPFL is currently developing such a technology.

Augmented and Virtual Reality attracts the big players

Despite the current challenges, we see VR adoption accelerating. Over 100 VR gaming titles have broken the $1million milestone in revenue in 2019, growing the total VR software market by 3 times during that year, which marks a major inflection point. Though still small, VR is a growing and sustainable platform for developers. We can easily foresee that when the above-mentioned challenges are addressed, this market is poised for formidable development far beyond the gaming industry and the current use cases.
Not surprisingly, major industry players are eyeing this promising market, and their roadmaps include AR/VR devices to varying degrees.
The activities of Apple in AR/VR are worth following, since the company has been an iconic trendsetter in consumer products which are appealing and easy to use.
Microsoft’s push with HoloLens devices and Facebook’s with Oculus give Apple’s participation in the AR/VR race an air of inevitability. After years of seeing iPhone sales stagnate in a saturated market, smart glasses may appear as the next growth relay. Apple has been exploring this technology for a decade (based on their patent filings), and with the launch of the ARKit framework (which integrates iOS device camera and motion features to produce augmented reality experiences for apps and games), we can suppose that Apple will bring a dedicated AR/VR product in the near future. The rumor is that Apple is working on multiple AR/VR headset prototypes, as engineers search for the “most compelling application” for such a device. An AR headset would be released in 2022, followed by a pair of AR glasses in 2023.
Another prominent technology player is seeing big opportunities in AR/VR.
Last January, Facebook CEO, Mark Zuckerberg, laid out a range of topics on which he and his company will focus over the next decade. By 2030, he says, AR/VR telepresence could allow employees to work remotely from anywhere in the world, alleviating the affordable housing crisis of increasingly populated cities and helping to address inequality by geography. We can reasonably argue that the current pandemic of COVID-19 will make this issue more pressing. Many workers have started using the various cloud-based video conferencing services and are getting used to it. AR/VR appears more than ever as a natural extension.
Mark Zuckerberg made the following comment: “Augmented and virtual reality are about delivering a sense of presence — the feeling that you’re right there with another person or in another place. Instead of having devices that take us away from the people around us, the next platform will help us be more present with each other and will help the technology get out of the way. Even though some of the early devices seem clunky, I think these will be the most human and social technology platforms anyone has built yet.”

And the future looks bright

So, in the coming years, we can expect the release of AR headsets, followed by AR smart glasses, a form-factor which is like the usual glasses we wear nowadays. These smart glasses will be assimilated considerably more to fashionable consumer products than to the AR/VR headsets, which are bulky and not very convenient. In the first phase, most of the computing power to process images will be embedded in smartphones, but miniaturization and optimization will enable a future move of this processing power into smart glasses. The question is more when, rather than if. With the mass adoption of smart glasses, we can expect that a slew of AR/VR apps will be developed, as was the case with smartphone apps. And as with smartphones (and the internet), we probably underestimate the countless possibilities that will be accordingly unlocked.

Conclusion

In conclusion, with the emergence of technologies that can relieve the user from the challenges of AR/VR and the advent of mainstream applications, we can legitimately suspect that we are at last close to a widespread adoption of realistic 3D. This will become ubiquitous more than 120 years after the black and white 2D experience and 80 years after the color 2D experience.
The current COVID-19 crisis may bring an additional boost to the development and adoption of applications that can relieve the user from a physical presence (3D video conferencing, 3D traveling, and so on). Will AR/VR in the future become so realistic as to trigger screams from users as did the first movie in 1895? There are good reasons to believe so.