Search

Android Labs: Bluetooth HID device implementation (your phone as a mouse + voice control)

January 28, 2015


Android Labs is an internal R&D activity within the HSC Connected Devices CoE where our amazing engineers implement proof of concept solutions showcasing our depth in the Android and Linux software stacks.

In this edition of "Android Labs", we bring you an innovative implementation of Android HID device specification for Bluetooth that allows your Android phone to be used as a standard BT mouse. We also added keyboard control and voice control to boot.

Remember to take a look at our demo video below! HSC may release the changes as part of its Open Source initiative in the near future. Please keep a tab on our Open Source page.

Why is this different?

There are many applications (like RemoteDroid and others) that have allowed a phone to act as a mouse for a few years. These solutions are typically implemented over WiFi (you need both PC and phone to be connected over WiFi) and comprise of a 'server' process installed on the PC and a client on the smartphone. In other words, this is a client-server model that needs special software on the PC. By implementing the BT HID device profile, your phone actually shows up as a standard BT mouse and requires no additional functionality.

Introduction

The purpose of this activity is to enable an android device to be used as a bluetooth peripheral device such as mouse, keyboard, remote control, gamepad etc. The android device can be then used to control devices like phones, computer, TV, PS3 etc. over bluetooth. To act as a bluetooth HID device, the android bluetooth stack needs to support HID device role of bluetooth HID profile. The current implementation of HID in bluedroid supports HID host role only. This activity implements HID device role in bludroid.

Work done so far

HID device role has been implemented in bluedroid using classic bluetooth. The current usecase allows the android device to act as a mouse and keyboard combo device. To support other use cases like remote control and gamepad, the changes will be required in HID report descriptor. The support of HID device role is being controlled through a compile time flag as of now. Some minor changes have been done in android framework and bluetooth system app to send input reports(mouse positions, key presses etc.) from a third party application. This implementation has been tested on a Nexus5 device connecting with windows as well as linux machines and a few smart phones supporting HID host role. A test application has been developed which provides a touch panel to send mouse cursor positions and two buttons to send left click and right click. For keyboard layout, the soft keyboard has been used which is available to all the android applications. The HID keyboard implementation has also been used to implement voice control of PC from Android device. For this purpose, the speech recognition feature of Android has been used which uses Google engine's voice search functionality. Offline speech recognition is also available however it is not that accurate when compared to online voice recognition. Some of the popular commands are available from voice control for Windows such as lock, unlock, reboot, shutdown, sleep etc. Apart from this, the individual key combinations can also be sent by speaking the actual keys such as “control alt delete” will send CTRL+ALT+DELETE key combination.

Future scope

HID device implementation should be modified in such as way that the HID profile role in bluedroid can be switched at run time based on a configuration. An application containing a master keyboard layout like keyboards for PC(containing CTRL, ALT, ESC and function keys) needs to be developed to provide full control. Mouse implementation can be extended as well to include scroll wheel using some touch gesture. The support for other HID devices can also be included such as gamepad, remote control, touchpad etc. Support for receiving output reports and feature reports needs to be implemented in the bluetooth stack for completeness of HID device role. The implementation can also be extended to support boot protocol mode of HID profile. Moreover, this work can also become the basis for implementing device role in HOGP (HID over GATT) profile over BLE.

(Note: This video has text annotation overlays which may not show up on a mobile device)

(Do you find this article interesting? You may want to check out our Embedded Android pages to read more about what we do in this space.)






9 Comments


  • Aug 26, 2016
    Exe
    I'm really interested in these features!
    Is the code already published?
    Maybe uploading it to GitHub or similar can be very powerful for the community.
    Thanks!

    Reply
  • May 23, 2016
    Guru
    I got to know that Android already supports HID Host so HID Device cant be supported simultaneously. Is it true.
    If true, have u disabled HID host role and enabled HID device role?

    Reply
  • Apr 14, 2015
    Alex
    This would be great, and even better if root would not be required.

    Reply
  • Mar 28, 2015
    Damiano
    Very interesting demo!
    Does this implementation require a rooted device to work though?

    Reply
    • Mar 30, 2015
      HSC Admin
      Hi Damiano, this demo involves implementation of a BT HID profile, so the device where the BT HID profile is implemented requires Android source code access. So yes, you need to be able to compile the Android image on the phone which is acting as a mouse.

      Reply
      • Nov 11, 2015
        z
        There's a new app 'True Mouse/KB' for enabling HID device profile. Let your phone work as real Bluetooth mouse & keyboard combo.
        https://play.google.com/store/apps/details?id=com.z2software.btpluginhidd

        Reply
        • Nov 12, 2015
          HSC Admin
          Right - it's not really an app only solution. If you read the description, its likely extracted the changes as so/jar files. Still requires root to copy the changes to the device system partition

          Reply
          • Jul 24, 2016
            sean t lasco
            So if it's blue toothed to another device can they hear and see what's going on

            Reply
          • Dec 30, 2015
            umar
            how can i access this code

            Reply


Add Comment