Will 2024 Be The Year Of Layered Realities?

January is always the month of predictions. Our team has already contributed to Semiconductor Engineering’s 2023 Look Back and 2024 Outlook. My personal tradition has become to combine the outlook with a look back at what industry experts thought of a decade ago. While this involved a trip to my garage to pick up the respective January IEEE Spectrum issue a decade ago, the new reality is that all of this is easy to do online. The topic that stands out to me from my January blogs in 2013 and 2014 and Apple’s impending Vision Pro announcement is that those were the years in which augmented and virtual realities were supposed to have their moments.

Ten years on, are we there yet? Richard Wagner (yes, the composer) would be proud. Read on!

The hotlist of themes – A decade ago and today

Contrasting the issues of IEEE Spectrum from 2013 and 2014 with 2024, one finds related themes:

• In Space, Virgin Galactic space missions, NASA’s hopes for Orion, and discussions about China as “The Next Space Superpower” from a decade ago matched the 2024 outlook on Artemis II. We are still applying a lot of tech to leave Earth. Sustainability back here on Earth is in the 2024 outlook with “Scaling up Solar Powered Kelp Farms.”
• For Brains, this year’s outlook discusses Synchron’s and Neural’s competition to connect to our conscience through brain implants. A decade ago, the “US Brain Initiative Boldly Began.”
• Robotics remains a big topic from “DARPA’s Rescue Robot Showdown” in 2014 to “Humanoid Robots Getting to Work” in 2024. BTW, Elon, why does your Tesla Optimus Robot give me vibes of the humanoids in the 1979 Maximilian Schell classic “The Black Hole.” Ugh. And from the home front, the videos of Optimus folding a shirt received a solid A- for its entertainment value, but a C- for usefulness with comments that we would not even be patient enough to watch him. Clearly not the killer app.

Source: Wikipedia, Yourprops.com

• Semiconductors: 10 years ago, the prediction was that “Chipmakers Push Memory Into the Third Dimension.” Sure, 3D-IC and chiplets will be huge topics in 2024. This year’s outlook talks about how “Chips to Compute With Encrypted Data Are Coming.” Accelerated homomorphic encryption, here we come!
• Consumer products: A decade ago, it was all about “Lighter, Brighter Displays,” Gaming with the Sony PS4, and “OLED TVs Arriving,” which we are now, ten years on, trying to improve with new materials in PHOLEDs. Consumers still are insatiable!
• Communications have moved from discussions a decade ago about how “LTE-Advanced Is the Real 4G” to the industry talking about 5G advanced and “Wi-Fi 7 Signaling the Industry’s New Priority: Stability.” The quest for higher data rates was, is, and probably always will be insatiable.
• No list is complete without mentioning Artificial Intelligence. A decade ago “IBM’s Watson Went to Med School.” This year’s outlook is all about applications of AI and how to control them. “Content Credentials Will Fight Deepfakes in the 2024 Elections” talks about how media organizations combat disinformation with digital manifests, and “A New Olympics Event: Algorithmic Video Surveillance” discusses the related data privacy issues. From a technology perspective, forty years on, we are well ahead of 1984. Let’s work on never getting to its social Orwellian implications!

The big topic: Layered realities and AR/VR takes the stage – again

We watched the trailblazing Google Glasses a decade ago and read how “Google Gets in Your Face” but “Google Glass Features and Apps Still in Flux.” We heard how “Oculus Rift Takes Virtual Reality Mainstream.” This year, I am writing this about a week before Apple rolls out its “Vision Pro.” In its introduction demo video, Apple shows a remarkable approach to maintaining continuity from previous devices and experiences, which previous attempts lacked. Productivity workspaces, 3D camera recording, movies, enhanced Facetime communication, and gaming look “familiar, yet groundbreaking.”

Lack of continuity to previous experiences may have been the fallacy of prior attempts in the VR/AR/XR area. Here is a trip down memory lane of my own experiences:

• 2007: I still have my “Icuiti iWear Video Eyewear Glasses for iPod Glasstron Style,” which seemed a good idea at the time. I used them for watching movies while working out on an Elliptical. The CNET review explains why I stopped using them: “The Icuiti iWear is expensive and uncomfortable, and its video quality is poor. Most users will have extreme difficulty getting the built-in headphones to fit propery.” It was clumsy, hard to use, and uncomfortable. Bad ROI for me at the time.
• 2018: As described in “Embedded World 2018: Security, Safety, And Digital Twins“, I used Microsoft’s Hololens for industrial applications. At the time, I wrote, quite impressed, how I could see through a Microsoft HoloLens, a virtualized, digital model of an industrial example with real-time measurements, and even could operate valves and switches with hand gestures. The digital twin transmitted my actuator commands to the actual control system.
• 2020: During COVID-19 some events were held in what later was dubbed the Metaverse. For one of these, I received an oculusGO headset and attended. It was an extraordinary experience, and some effects describing some of the technology aspects discussed at the event are only possible in VR and were helpful. Zoom carried the user-to-user communication. We are probably some time away from moving that to VR.

Source: https://thewaroftheworldsimmersive.com/the-experience/what-on-earth-is-it/

• 2020/2023: As a theatre fan of pretty much everything live on stage, in 2020, the weekend before the COVID-19 shutdown, I saw Layered Reality’s immersive “War of the Worlds” for the first time. Last year, in 2023 I saw it again and added the immersive “Gunpowder Plot” experience for good measure. I like the term “Layered Reality” in this context. Both shows combine digital technology (Virtual Reality, projection mapping, and volumetric holograms) with live theatre (live actors, movie-scale sets, and special effects) plus real physical sensations (touch, temperature, smell, sound and music, physical movement, and taste). It creates quite an experience. The producer, Layered Reality, significantly improved compared to my first experience in 2020. The new goggles provided a fantastic 360-degree view – I could turn around and see Tom Felton as Guy Fawkes talking to me, see London from the Thames, and, when turning around, see the ferryman operating the boat I was on. Simulated chair movement and immersion into VR tricked my body into holding on as we progressed through the story – like in 2020 for “War of the Worlds.” The 2023 version cut a scene in which users could interact with each other in VR in the 2020 version. I did not miss it, as the interaction with others in VR felt cumbersome to me at the time.
• 2023: While the previous experience layered the realities and did not use Augmented Reality (AR) equipment, true to Richard Wagner’s desire to experience new technology, the Bayreuth Opera Festival last year introduced AR glasses for a unique experience of “Parsifal,” Wagner’s final opera. I was very fortunate to get a ticket (it takes years). But my luck ran out as they had only about 300 AR-enabled seats. The reviews were mixed when talking to fellow patrons who got the glasses. The goggles had to be tuned to the person in advance, some considered them cumbersome, and some felt they impacted the viewing experience in a dark theatre. In one situation, the AR-free audience saw my favorite tenor, Andreas Schager, making a gesture that seemed only to make sense in the AR environment. He was holding the grail. Well, my imagination filled that in just fine.

It is essential to note the difference between consumer adoption and industrial adoption. Following the contenders talked about a decade ago – Oculus Rift, Google, and Vuzix – they all did take different routes. Meta acquired Oculus Rift, Google halted Glasses, and Vuzix focused on the industrial sector, like Microsoft’s Hololens announced in 2015. Key Vusix application domains are healthcare, manufacturing, and warehousing. Consumer adoption and industrial adoption pose very different market questions.

So, will 2024 be the year of AR/VR with Apple’s rollout of the VisionPro? Maybe. The continuity of previous experiences and the intended feeling of “familiar, yet groundbreaking” makes me hopeful. Some of Steve Mann’s thoughts on “Why Smart Glasses Might Not Make You Smarter” remain true today.

Personally, the tech enthusiast in me thinks we will see significant progress in the consumer domain with Apple’s spatial computing efforts. I will report back when I get my hands on a VisionPro. And as somebody working in the semiconductor space, all this is exciting as it will spawn new ecosystems that need to deal with more and more chips that deal with more and more data. Data transport architectures and connecting things up on and between chips is what I am involved in at Arteris, after all, at least when I am not at the theatre.