Will VR and AR finally start to develop independently?

Oct 20, 2021 AI


"When will VR/AR take off?" Amid the Metaverse hype, this question is back in the spotlight.

Five years ago, at the height of the last VR boom, Epic Games founder Tim Sweeney was asked the same question. At the time, Epic Games' main product was its game engine, Unreal Engine, and the team hadn't yet made the $1 billion annual hits Fortnite, and Tim, despite being a big-game guy, hadn't yet had the right to call on Apple to lower the Apple tax.

However, since Unreal Engine is the only professional tool for VR content production (the other is Unity), Tim Sweeney's judgment still carries a lot of weight.

In response to the opening question, Tim Sweeney told a story: after Apple produced the first generation of the iPhone, engineers from traditional phone manufacturers (probably Motorola) took apart the back cover of the iPhone and were amazed - they had never seen it lined up and assembled like that before. How can someone make a phone like this?" It's no wonder that BlackBerry's boss later reassured everyone: "We'll be fine."

What Tim Sweeney wants to say is that to produce a truly innovative product, there must be a supporting core supply chain, and tuning the supply chain to produce core components specifically for itself is one of Apple's tricks, as exemplified by the later success of the iPhone.

Similarly, for VR/AR, to become a truly revolutionary product, there must be a corresponding supply chain that specializes in producing core components for it, so that the metaverse can come sooner.

As the core component, the development of the chip undoubtedly has a decisive role in the rise of VR/AR products. "Benefiting from the maturity of the smartphone industry, VR/AR chips are now trending away from smartphones and towards independence and customization.


Qualcomm XR2 chip abandons low-end market

Before Facebook bought Oculus VR for $2 billion in 2014, the latter had just released its second-generation developer kit model, the Oculus Rift DK2. iFixit.com took the machine apart and discovered that the screen used was that of a Samsung Galaxy Note 3. In addition to praising Oculus for its DIY spirit back in the day, it also illustrates the importance of the smartphone supply chain for VR hardware back in the PC VR era.

Considering the PC VR era of computing power from the PC, so the first task of the VR headset is to fix the display, that is, the OLED screen. Due to the different materials and display principles, OLED screens had the advantage of low latency and no trailing images compared to traditional LCD screens, especially Samsung's AMOLED screens were the first choice for VR products.

In 2018 Oculus and Xiaomi teamed up to launch the Oculus Go (Xiaomi VR All-in-One), a machine that supports 3 degrees of freedom.

This machine uses Qualcomm's flagship chip Snapdragon 821, Oculus and Xiaomi did not use the latest Snapdragon 845. There may be three reasons, one is that the Go is two years ago began to develop, 821 is the flagship chip at the time; two is that the Go's supports three degrees of freedom, 821 chip fully meet the demand; three, from a cost perspective, Oculus Go priced at 1499 yuan, which is more convenient to control the budget.

It is worth mentioning that also in 2018, in addition to the Snapdragon 845, Qualcomm also released the XR 1 (Extended Reality) chip and platform, the giant in the field of mobile chips has seen the future of VR/AR and is starting to make moves. However, while the XR 1 chip supports VR devices from 3 to 6 degrees of freedom, it is primarily aimed at 3-degree-of-freedom viewers like the Oculus Go.

In terms of chip capabilities, the Qualcomm XR 1 is close to the Snapdragon 600 (and some say 760). Qualcomm's official statement is that the XR1 can provide a "high quality" VR experience, while the flagship Snapdragon 845 at the same time can provide the 6 degrees of freedom experience that requires spatial positioning and handle recognition and other high-end features.

That's probably why Oculus is using Qualcomm's Snapdragon 835 chip instead of XR1 when it launches its 6-degree-of-freedom-enabled all-in-one, the Oculus Quest, in 2019.

However, in 2020, with the worldwide sales of the XR2 chip-powered Oculus Quest 2, it shows that Qualcomm is starting to make a real push into VR. The XR 2 chip, said to be modified from the Snapdragon 865 chip, not only supports 8K 360-degree video, but also 7 concurrent cameras for precise motion and gesture tracking. The addition of 5G and AI, meanwhile, provides improvements in network speed and efficiency.

Oculus Quest 2's premium performance is a testament to the success of Qualcomm's XR2 chip in abandoning the low-end market.


AR that focuses on scenes

While Qualcomm started to "prune" the Snapdragon on the smartphone side to create the XR chip, AR hardware companies are using different scenarios and choosing their chips.

At the high-end market, the two pioneers Magic Leap and Microsoft are aiming for the "endgame" from the start. However, because both companies have a high definition of AR (or MR), and because AR itself requires more than VR in terms of rendering, spatial positioning, and integration of reality and reality, it is difficult to find the right chip at the time, at least Qualcomm's Snapdragon or XR could not meet the requirements of AR glasses.

As a result, Magic Leap One used a split design with an NVIDIA Tegra X2 multi-core processor on its compute unit Light Pack, which may not have proven to be a particularly ideal design in hindsight.

People who have used the Magic Leap One headset should know that after using it for a while, the heat and power consumption of the Light Pack is not so well. But the split design is a precedent for the AR glasses that followed.

Microsoft Hololens is an all-in-one form, the first generation of HoloLens headset is said to use the chip of old partner Intel. In addition to Intel's CPU and GPU, one of the special features is the Microsoft team's holographic processor HPU (holographic processing unit), a custom ASIC chip mainly used to process data such as 3D images and gestures.

Microsoft's second-generation HoloLens uses the Qualcomm Snapdragon 850 as the main processor, along with an upgraded HPU 2.0 to achieve more comprehensive hand motion tracking and better visual performance.

However, from Magic Leap One to HoloLens two generations of products, due to the high cost, mainly for the B-side market, and no connection with the C-side market.

On the other hand, in some specific industrial and commercial fields, AR glasses may use more specialized chips to achieve specific functions and keep the product lightweight and endurance because of more vertical usage scenarios.

Liang Xianglong, head of hardware R&D at LLVision, told GeekPark (ID: geekpark) that one of the important scenarios for the company's previous products was to achieve fast face recognition on the glasses side, so the AI image processing requirements were higher. And the team used the Myriad series processors from Intel's Movidius VPU platform, whose specialty is deep neural networks and computer vision applications. Compared to platforms such as Snapdragon in the same class, the Myriad platform is more ideal in both AI performance and power consumption and can be worn for long periods with a split design that keeps the weight of the glasses at 30-40 grams.

Of course, using a relative niche technology platform comes with a corresponding cost. Compared to the Qualcomm platform, Myriad has to write many of its underlying programs, which is time-consuming and challenging for startups. The good thing is, if you can overcome this difficulty, it will form a product moat in a period.


VR, AR finally start to develop independently?

The success of the XR2 has allowed Qualcomm to see the potential of the metaverse. Industry insiders revealed that it was not until the release of XR2 that Qualcomm had a dedicated team design to start designing chips specifically for VR and AR devices, rather than using off-the-shelf Snapdragon chips for modifications.

Liang Xianglong believes that to meet the requirements of future VR/AR devices, the core chip development trend is the high process, low power consumption, high integration. As VR/AR glasses are bound to become thinner and thinner, in the absence of a breakthrough in the development of battery technology, the chip power consumption needs to come down; VR/AR will use functions like spatial positioning gestures to further reduce power consumption and reduce the Development difficulty., Qualcomm's XR series of chip products to be launched next year will most likely have the above features. Liang Xianglong revealed that LLVision is currently trying to move the technology platform to Qualcomm and planning to launch new products next year that goes further in terms of audio and video effects.

On the other hand, industry giants like Apple, Facebook, Magic Leap, and others are likely to continue the direction of self-research and customization. According to foreign media reports, Apple has completed the design of the VR/AR chip in 2020 and commissioned TSMC to carry out a 5nm advanced process production chip. The SoC consists of three chips, the function is wireless data transmission function optimization, compression, and decompression of the film, improving battery efficiency.

Although it is still difficult to predict when the era will really arrive, the giants like Qualcomm and Apple have poured their efforts into the chip area. Therefore, at least we can be sure that the next VR/AR products will usher in a period of accelerated development.