Search

Android Innovation Labs: Augmented Reality in Dashboards

December 12, 2016


Augmented Reality (AR) in Dashboards/Head Units are an area of significant interest in the connected car space.  BMW, Tesla and several other OEMs have announced their intent to release AR enhanced dashboards in the near future.  

Augmented Reality (AR) enhanced dashboards can enable several innovative use-cases such as:

  1. While using the navigation system, the head unit switches to live camera feed with AR overlays when you “almost” arrive at your destination. This allows for exact destination tracking (as opposed to current navigation systems that announce ‘you have arrived’ but you don’t see your destination)
  2. Using AR overlays whist driving to show speed limits on routes, even when not using navigation
  3. Using smartphone capabilities to enhance dashboards that don’t have specific features, such as notifications via overlays about color-coded distance between your car and the car in front
  4. And many other such use-cases

There are many ways to implement AR in the  dashboard. We believe one of the early deployment technologies will be to leverage the power of the smart phone and its sensors/accessories to augment overlays on traditional head units. Standards such as CarPlay, MirrorLink and Android Auto already exist that allow for phones and dashboards to interact over well defined protocols.

As part of our innovation labs prototyping, we wanted to see how MirrorLink based solution would perform today with AR overlays from the phone. We also wanted to use a stock Android tablet as a HU (head unit) and see how easy (or hard) it was to get mirror link working on a non certified device (which we think will be a common requirement for after market providers). In our demo, we overlay live camera view with location information translated to names of buildings we are driving by. The MirrorLink implementation was provided by RealVNC.

Here is what we did for the technology concept demo: 

  • To enable MirrorLink on a Nexus device, the AOSP Android kernel was recompiled with additional modules and several permissions in SELinux  to build a customized android image. HSC used that customized Android image in a Nexus tablet that served as a HeadUnit running  Android Marshmallow (6.0)
  • HSC  customized the Mirror Link Client provided by Real VNC 
  • A mirrorlink signed app was developed that takes camera feed and displays the feed on the head unit via a MirrorLink connection
  • Performance optimizations and appropriate scaling had to be implemented for display on the HU.
  • GPS location was overlaid on the HU from the phone (again, a realistic scenario where a head unit may not have all the features)

 This demo video is intended to be a technology showcase of integrating Augment Reality tools with MirrorLink and not necessarily a real-life use-case. A real-life use case that leverages such technologies may be one of the use-cases described above or one of many other innovative use-cases.

 

 






No Comments




Add Comment