The HoloLens headset tracks head movement using a combination of cameras, accelerometers, and gyroscopes. Inside a moving car, these sensors can get confused, reporting contradictory information. The same happens with humans who get car sick when the information coming from the eyes does not correspond with that coming from the motion sensors inside their inner ear.
Microsoft started to work with Volkswagen to solve this issue in 2018. At the time, the German company was trying to use augmented reality headsets to teach drivers how to get around a racetrack faster. They immediately ran into problems since the camera sensors disagreed with the motion sensors, and this broke the HoloLens.
Eventually, the two companies came up with a prototype that allowed a car to display real-time information on a connected headset without experiencing the “car sick” problem. For that, Volkswagen established a bidirectional data connection between the vehicle and the HoloLens to display and control real-time information from the car. This also led to finding new use cases for the HoloLens AR headset inside a vehicle.
For instance, the headset can be used in autonomous vehicles to visualize car trajectory and even control settings inside the vehicle. This involves displaying holograms within the driver’s forward-facing field of view but also taking into account where the user wearing the glasses is looking. In one of the pictures shared by Microsoft, the HoloLens 2 projects a virtual map onto the dashboard of a car, with navigation arrows popping up at the right time to indicate direction. In a second picture, it alerts the driver to an upcoming pedestrian crossing.
Considering that the HoloLens 2 has a steep price of $3,500, it’s clear that Microsoft targets enterprise users first and foremost. Microsoft does not rule out consumer use in the longer term, although this might take a while. The headset might actually prove a hit with autonomous vehicle users, opening a new type of experience while inside the car.