The convergence of rapid technological advances and the pandemic years have contributed to a quantum leap in interest in, and the development of the metaverse.

Extended Reality (XR), an umbrella term that encapsulates Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR), represents the gateway to metaverse, and to realize tremendous breakthrough transformations in diverse technology areas, including manufacturing, retail, education and healthcare, amongst others.

While the future development of metaverse from here-on will potentially be bereft of a linear technological evolution, and will go through hoops and bounds, one thing is clear. The access points for metaverse will depend on a robust XR devices ecosystem, backed by strong advances in AI, 5G and Cloud.

Qualcomm Technologies Inc. has made a big bet on Metaverse, and is focused on bridging the gap between the digital and physical worlds with its technologies, as well as investments to foster new XR innovations in gaming, healthcare, education and enterprise, amongst others.

I recently spoke with Hugo Swart, Vice President and General Manager of XR, Qualcomm Technologies, Inc., on all things Metaverse, and understand how Qualcomm is enabling the future.

Prabhu: What is the metaverse? What is Qualcomm’s vision of the metaverse?

Hugo: The metaverse is a vision of interconnectivity, it’s the joining of many worlds and potentially their economies. Many call it the second coming of the internet – the spatial internet — complete with personalized digital experiences that span the physical and virtual worlds. 

This will be accomplished with Snapdragon powered VR and AR devices and our innovations such as Snapdragon™ Spaces Developer Platform. Smartphones and PCs will extend into the Metaverse as well, while devices like smartwatches, earbuds, and smarthubs are physical links in the Metaverse chain.  Additionally, the metaverse will spur growth for a whole host of completely new device form factors.

Qualcomm Technologies is uniquely positioned to support the creation of the metaverse’s technological foundations – from key technologies like 5G, AI, immersive camera, graphics and audio to advanced hardware, software and perception algorithms we deliver optimized advanced high performance and low-power options – and we are committed to helping advancements in this space.

Prabhu: What are the building blocks for metaverse?

Hugo: There are multiple building blocks needed to succeed in a persistent spatial internet from Hardware, Software, Perception Algorithms to advanced technologies like 5G, AI, Camera and graphics. To access the metaverse one needs to have a device that will allow them to experience the spatial internet – this is the focus of the XR team at Qualcomm. The widespread adoption of XR headsets is expected to revolutionize human digital experiences and provide an entry point of choice into the metaverse as it evolves. A slew of XR product launches – recent or imminent – by companies such as Microsoft, Meta, Snap, Pico, HTC and Lenovo underline the positive sentiment around the uptake of XR and the belief in the potential disruptive nature of the metaverse.

5G will also play a crucial role in enabling and shaping the metaverse. Realizing the full capabilities of a spatial internet will require truly advanced technologies, connectivity and perception algorithms. Along with AI and Edge Computing technologies, high speed Wi-Fi and 5G will provide the high bandwidth and low latency that is needed for such advancements.

Prabhu: How will Qualcomm enable the entry to metaverse through the devices ecosystem?

Hugo: The initial entry into the metaverse today can be accomplished with Snapdragon powered VR and AR devices. These experiences are brought to life through our innovations such as Snapdragon Spaces which enables developers to create applications that can do things like map out physical spaces so that digital objects can be overlaid on them and enabling hand-tracking so that users may manipulate these digital objects with hand gestures.

We have been working closely with the devices ecosystem for some time now using our mobile and XR Platforms to create AR and VR enhanced experiences. The Lenovo ThinkReality A3 smart glasses and Meta Quest 2 VR headset are good examples of such industry collaborations.

At the beginning of the year, we also announced a strategic partnership with Microsoft to develop custom chips for lightweight AR glasses that consumers and businesses can use for metaverse apps. Devices arising from the collaboration will work with Microsoft Mesh, which allows users to beam a realistic likeness of themselves into the headset of another user so that it feels like the two people are in the same room.

Prabhu: What is Qualcomm’s metaverse strategy?

Hugo: As we head towards the metaverse, Qualcomm Technologies is focusing strongly on connecting the physical and digital worlds and providing both consumers and enterprises with groundbreaking platform technologies and experiences that will enable them to build, work and engage in the metaverse.

Our XR business is structured to address the needs of the industry around 4-key strategic areas:  

1. Platforms: The foundation that powers XR devices with compute, connectivity, AI and more. This includes our Snapdragon XR1 and XR2 platforms that are purpose-built solutions that address the unique needs of OEM’s creating the latest AR, VR and ultimately MR devices
2. Software & Perception Technologies: This includes a robust suite of technologies like tracking, video pass- through and others. Of the course in the last year we have made several strategic acquisitions to augment these capabilities. This allows us to offer turn-key solutions to our customers and help get them to marker faster by removing the need for them to build their own perception technologies in-house.
3. Reference Designs: We have developed a series of stand-along VR and both wired and wireless AR reference designs to help drive the industry forward and help lower the barriers to entry for brands to enter the XR market. We most recently announced our wireless AR smart viewer reference design that will help users to cut the cord for AR glass much in the way we did for VR a few years ago.
4. Ecosystems: Working with key industry customers and developing a thriving developer ecosystem are key to unlocking the full potential of the metaverse. This is why we have developed the Snapdragon Spaces developer platform which empowers developers to unlock the full potential of headword AR using industry-leading technology, a cross-device SDK, and an open ecosystem. Square-Enix is another key partner who we are working with to help bring their existing 2D IP into AR with Snapdragon Spaces.

Another element to our metaverse strategy is to work with our Qualcomm Ventures team on a $100M Snapdragon™ Metaverse Fund. This fund was created to invest in developers and companies that are working on XR technologies. The fund will support the developer ecosystem through a combination of venture investments in leading XR companies by Qualcomm Ventures and a grant program by Qualcomm Technologies for developer ecosystem funding in creating XR experiences in gaming, health and wellness, media, entertainment, education, and for enterprise. Our XR Labs, meanwhile, work on developing breakthrough technologies that will shape the future of XR. These are all important steppingstones toward the development of the metaverse.

Prabhu: What use cases – consumer and enterprise alike – do you see getting enabled in the metaverse?

Hugo: The metaverse technology enables a wide range of use cases across industries. For instance, the gaming industry is seen as one of the early adopters of the metaverse. Games that allow players to interact with other participants in a single virtual interoperable environment have been around for quite some time now in mobile phones, PCs and VR devices. Other consumer focused metaverse experiences include fitness, entertainment as well as social meetups.

Similarly, the metaverse is also set to transform our workplaces. The technology can be used to develop virtual, interactive workplaces. In virtual reality and mixed reality environments digital avatars will be used in holographic telepresence experiences. Holographic telepresence also allows teams to work together across different geographic locations virtually but in a way that feels more in person. Additionally, there are benefits to utilizing XR technologies for corporate training and education.

In the healthcare industry, spatial computing can be leveraged across holographic medical imaging or in enabling interactions between patients and healthcare professionals. The education industry is also set to significantly benefit from the metaverse. For example, during the first year of Covid 19, we assisted with a trial between VictoryXR and Morehouse College. Morehouse held 3 classes in VR.

Qualcomm Technologies continues to explore and develop innovative use cases enabled by 5G for the metaverse. We are currently demoing the following solutions:

• Boundless AR: While 5G-powered AR glasses capable of a direct cloud connection remain a long-term vision for the metaverse, we have created a solution that enables Wi-Fi-powered AR glasses to communicate with the cloud using a 5G phone, puck or laptop.
• 5G cloud gaming: Smooth gaming experiences require zero or minimal network latency. Qualcomm Technologies’ end-to-end latency optimizations reduce latency spikes, while our 5G AI API for gaming reduces buffering. The API allows applications to adapt to changing network conditions, thus providing gamers with better image quality, lower frame drop rate, and an enhanced gaming experience.
• 5G mmWave: Qualcomm Technologies has developed 5G mmWave capabilities that assist with object sensing. The system enables collective sensing – a process in which the network and devices work together to determine the location of a device-free object with no ability to communicate over 5G. For instance, it could be a person, a wall or even a random physical object in our environment.  The system utilizes 5G positioning techniques and RF sensing to determine location. 5G provides a better understanding of the physical world, enabling it to seamlessly merge with the digital and virtual worlds, further making the metaverse a reality.
• Secure services beyond data: We are working to develop secure applications that involve positioning, ranging, and time-resiliency.

Prabhu: When do you see the mainstreaming of metaverse happening?              

Hugo: The convergence of 5G, XR and AI is rapidly transforming the way we interact, work, socialize, play and transact. As these technologies become increasingly pervasive, the metaverse could become a mainstream experience in the next couple of years. While certain aspects of the spatial computing are already in play, other key features will take a longer time to become mainstream.  However, with continued research, innovation and investments such as the Snapdragon Metaverse Fund underway, as well as strong industry collaborations, we can expect the metaverse to become a reality sooner than later.