Android Labs: Bluetooth HID device implementation (your phone as a mouse + voice control)

Why is this different?

There are many applications (like RemoteDroid and others) that have allowed a phone to act as a mouse for a few years. These solutions are typically implemented over WiFi (you need both PC and phone to be connected over WiFi) and comprise of a ‘server’ process installed on the PC and a client on the smartphone. In other words, this is a client-server model that needs special software on the PC. By implementing the BT HID device profile, your phone actually shows up as a standard BT mouse and requires no additional functionality.

Introduction

The purpose of this activity is to enable an Android device to be used as a Bluetooth peripheral device, such as a mouse, keyboard, remote control, gamepad etc. The android device can then be used to control devices like phones, computers, TV, PS3 etc., over Bluetooth. To act as a Bluetooth HID device, the android Bluetooth stack needs to support HID device role of the Bluetooth HID profile. The current implementation of HID in blue droid supports HID host role only. This activity implements HID device role in bludroid.

Work done so far

HID device role has been implemented in bluedroid using classic bluetooth. The current usecase allows the android device to act as a mouse and keyboard combo device. To support other use cases like remote control and gamepad, the changes will be required in HID report descriptor. The support of HID device role is being controlled through a compile time flag as of now. Some minor changes have been done in android framework and bluetooth system app to send input reports(mouse positions, key presses etc.) from a third party application. This implementation has been tested on a Nexus5 device connecting with windows as well as linux machines and a few smart phones supporting HID host role. A test application has been developed which provides a touch panel to send mouse cursor positions and two buttons to send left click and right click. For keyboard layout, the soft keyboard has been used which is available to all the android applications. The HID keyboard implementation has also been used to implement voice control of PC from Android device. For this purpose, the speech recognition feature of Android has been used which uses Google engine’s voice search functionality. Offline speech recognition is also available however it is not that accurate when compared to online voice recognition. Some of the popular commands are available from voice control for Windows such as lock, unlock, reboot, shutdown, sleep etc. Apart from this, the individual key combinations can also be sent by speaking the actual keys such as “control alt delete” will send CTRL+ALT+DELETE key combination.

Future scope

HID device implementation should be modified in such as way that the HID profile role in bluedroid can be switched at run time based on a configuration. An application containing a master keyboard layout like keyboards for PC(containing CTRL, ALT, ESC and function keys) needs to be developed to provide full control. Mouse implementation can be extended as well to include scroll wheel using some touch gesture. The support for other HID devices can also be included such as gamepad, remote control, touchpad etc. Support for receiving output reports and feature reports needs to be implemented in the bluetooth stack for completeness of HID device role. The implementation can also be extended to support boot protocol mode of HID profile. Moreover, this work can also become the basis for implementing device role in HOGP (HID over GATT) profile over BLE.

(Note: This video has text annotation overlays which may not show up on a mobile device)

(Do you find this article interesting? You may want to check out our Embedded Android pages to read more about what we do in this space.)