BMW and Meta Working on Offering In-Car AR and VR Experiences for Passengers

Extended reality is a general term that encompasses augmented reality (AR), mixed reality (MR), and virtual reality (VR). Back in January this year, BMW launched the Supplierthon, an initiative that aimed to attract researchers, startups, and pioneers in the global Metaverse community. In December 2022, the Bavarian Group joined the Metaverse Standards Forum, a venue for cooperation to create standards for an open and inclusive metaverse.

One of BMW's latest partnerships is with Meta's Reality Labs research – the brand demonstrated, in an industry-first, the ability to display stable VR and MR content to passengers in a fast-moving car. The content remains stable even when the vehicle turns, accelerates, or goes over speed bumps.

Claud Dorrer, the Head of BMW Group Technology Office USA in Mountain View, explained that it's still early to estimate when the technology will be commercially integrated. However, the company envisions various uses for the XR devices in vehicles. For instance, the driver can use it to find their car in a crowded parking lot, or the system can showcase information regarding the vehicle's condition.

Cameras and motion sensors are required to enable VR headrests to identify the users' exact positions. And that's how the content remains stable when the users move their heads or look around. This works well when you're using VR at home, but things get a bit more complicated in cars.

Due to the device's movement in the car and the passing surrounding when the vehicle is in motion, the technology gets confused. The headset's motion sensors think the device is moving, but the cameras believe it's standing still compared to the car's interior background.

BMW and Meta's researchers set out to solve this issue by incorporating information from a BMW car sensor array into the tracking system of a Meta Quest Pro, the company's most advanced VR headset. This way, the tracking system calculates the device's location relative to the car and the car's location relative to the world. The result is that virtual objects are accurately anchored to a moving vehicle, enabling smooth VR and MR passenger experiences.

There are limitless possibilities regarding the type of passenger experiences in areas such as productivity, entertainment, or even meditation. Some of the following steps could be to allow passengers to see markers for all sorts of locations, such as restaurants, landmarks, and more.

BMW Group is open to finding and working with additional partners to develop industry standards for connecting and using XR consumer electronics devices in vehicles. Of course, the customer's privacy is critical, and the company explains that appropriate steps are taken to ensure optimum protection.