Android Innovation Labs: Augmented Reality in Dashboards

December 12, 2016

Augmented Reality (AR) in Dashboards/Head Units are an area of significant interest in the connected car space.  BMW, Tesla and several other OEMs have announced their intent to release AR enhanced dashboards in the near future.  

Augmented Reality (AR) enhanced dashboards can enable several innovative use-cases such as:

  1. While using the navigation system, the head unit switches to live camera feed with AR overlays when you “almost” arrive at your destination. This allows for exact destination tracking (as opposed to current navigation systems that announce ‘you have arrived’ but you don’t see your destination)
  2. Using AR overlays whilst driving to show speed limits on routes, even when not using the navigation
  3. Using smartphone capabilities to enhance dashboards that don’t have specific features, such as notifications via overlays about the color-coded distance between your car and the car in front
  4. And many other such use-cases

There are many ways to implement AR in the dashboard. We believe one of the early deployment technologies will be to leverage the power of the smartphone and its sensors/accessories to augment overlays on traditional head units. Standards such as CarPlay, MirrorLink, and Android Auto already exist that allow for phones and dashboards to interact over well-defined protocols.

As part of our innovation labs prototyping, we wanted to see how MirrorLink based solution would perform today with AR overlays from the phone. We also wanted to use a stock Android tablet as an HU (head unit) and see how easy (or hard) it was to get mirror link working on a non-certified device (which we think will be a common requirement for aftermarket providers). In our demo, we overlay live camera view with location information translated to names of buildings we are driving by. The MirrorLink implementation was provided by RealVNC.

Here is what we did for the technology concept demo: 

  • To enable MirrorLink on a Nexus device, the AOSP Android kernel was recompiled with additional modules and several permissions in SELinux  to build a customized android image. HSC used that customized Android image in a Nexus tablet that served as a HeadUnit running  Android Marshmallow (6.0)
  • HSC  customized the Mirror Link Client provided by Real VNC 
  • A mirrorlink signed app was developed that takes camera feed and displays the feed on the head unit via a MirrorLink connection
  • Performance optimizations and appropriate scaling had to be implemented for display on the HU.
  • GPS location was overlaid on the HU from the phone (again, a realistic scenario where a head unit may not have all the features)

 This demo video is intended to be a technology showcase of integrating Augment Reality tools with MirrorLink and not necessarily a real-life use-case. A real-life use case that leverages such technologies may be one of the use-cases described above or one of many other innovative use-cases.



No Comments

Add Comment


We use cookies (including third party cookies) to ensure you get the best experience while visiting our website. Click "Accept All Cookies" to accept the cookie usage. Click "Cookie Settings" to adjust cookie settings.

Mandatory Cookies

These cookies cannot be disabled

These cookies are necessary for the website to function and cannot be switched off.

  • __RequestVerificationToken
  • authentication
  • dnn_IsMobile
  • language
  • LastPageId
  • NADevGDPRCookieConsent_portal_0
  • userBrowsingCookie

Analytics Cookies

These cookies allow us to monitor traffic to our website so we can improve the performance and content of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited or how you navigated around our website.

  • _ga
  • _gat
  • _gid

Functional Cookies

These cookies enable the website to provide enhanced functionality and content. They may be set by the website or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

  • __atuvc
  • euconsent

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.


Not used.