The result so far
Here’s a video demo of I was able to achieve with this reverse engineering effort:
Click here if you’re interested in running this yourself.
Background
About a week ago I was having a look at some new input devices to use for my game experiments that were a little more interesting than a simple USB or Bluetooth Gamepad. As I kept a close eye on consumer VR technology in 2016/2017, it occurred to me now that many of the devices released in the past year have been updated and on the market for at least a few months. Google Daydream had been out since November 2016, sold with a packed-in controller, and the Gear VR received a similar 3 degrees of freedom (3DOF) controller in April 2017.
Being the first to market, the Daydream controller was already (1) reverse (2) engineered (3). It wouldn’t take much to use it as an input for my game ideas.
However, there were clear hardware design disadvantages versus the Gear VR controller:
- A non-removable built-in battery with USB-C as the only means of charging it.
- Users reported much shorter battery life than the Gear VR version.
- It lacked a trigger button.
- It was much more prone to sensor drift than its Gear VR counterpart.
Moreover, it was not sold separately, which meant a seller on Amazon, eBay, or other marketplace would be incentivized to sell the headset and the controller together as a whole package rather than split the devices.
The Gear VR Controller used standard AAA batteries and was better designed ergonomically; I found an Amazon Warehouse Deals listing that sold the controller only for £20. Shipped the same day, it arrived in less than a week. Praise modern logistics!
Unboxing and RTFM
After unboxing the device, the first thing I tried doing was pairing it with my laptop. No dice. The device didn’t even show up in the macOS Bluetooth menu in System Preferences. Have to try something else.
I sought out several freeware Bluetooth LE (BTLE) explorer apps and ended up using LightBlue Bluetooth Device Discovery. This app picked it up straight away.
There was an unusual quirk, however, the cause of which I didn’t manage to fully figure out. Occasionally, the controller would break the Bluetooth connection as it entered its sleep mode. Once re-awakened, any attempt to pair with it again would cause it to toggle rapidly between being paired and unpaired.
The only solution I found for this was to manually remove the device profile from the Bluetooth device list every time this happened so it could formally pair again.
I had little prior experience using Bluetooth devices at this level so I had to read up on how the whole BTLE tech stack is meant to work.
Essentially, every BTLE client device talks to a BTLE host device via a GATT (Generic Attributes) profile.
This profile tells the host what the client can do (GATT SERVICES, e.g. read the current temperature) and groups them together (GATT CHARACTERISTICS, e.g. environment sensing > read the current temperature).
For each service:
- You can read its value
- You can write a value to it
- You can subscribe/unsubscribe to notifications for when values change
My goal was simple: use the Gear VR controller outside of the Gear VR apps and without other hardware.
Initial survey
I was able to map out all of the services and characteristics of the controller very quickly. The next step was trying to figure out what they did and how to use them.
Services
Characteristic
180F
2A19
180A
2A29
2A24
2A25
2A27
2A26
2A28
2A50
1879
2A4E
2A4B
2A4A
2A4C
2A4D
2A22
2A32
4F63756C-7573-2054-6872-65656D6F7465
C8C51726-81BC-483B-A052-F7A14EA3D281
C8C51726-81BC-483B-A052-F7A14EA3D282
FEF5
8082CAA8-41A6-4021-91C6-56F9B954CC34
724249F0-5EC3-4B5F-8804-42345AF08651
6C53DB25-47A1-45FE-A022-7C92FB334FD4
9D8489A3-000C-49D8-9183-855B673FDA31
457871E8-D516-4CA1-9116-57D0B17B9CB2
5F78DF94-798C-46F5-990A-B3EB6A065C88
61C8849C-F639-4765-946E-5C3419BEBB2A
64B4E8B5-0DE5-401B-A21D-ACC8DB3B913A
42C3DFDD-77BE-4D9C-8454-8F875267FB3B
B7DE1EEA-823D-43BB-A3AF-C4903DFCE23C
Using LightBlue and a little bit of help from bluetooth.com, we can see that services 180F and 180A are the Battery Service and Device Information Service, respectively.
The other services were non-standard, and a search for those UUIDs (e.g. 4F63756C-7573-2054-6872-65656D6F7465) turned up very little. I did try searching for other reverse engineering efforts on GitHub and found a wiki page indicating that FEF5 was possibly a service used for firmware updates. No doubt a skilled person with more interest in this area will figure it.
This leaves 1879 and 4F63756C-7573-2054-6872-65656D6F7465 the only services still unknown.
Gear VR Input Service APK
Thankfully, retrieving, decompiling, and debugging APK (Android Package Kit) files are all tasks that are well supported and documented within the Android ecosystem. It was sensible to look in that direction next.
Gear VR games make use of the controller’s readings through a service that translates raw Inertial Measurement Unit (IMU) data into something that a game developer is more likely to use: orientation, touchpad position, and which key presses are happening at any given moment. For common input devices like keyboards and mice, this step is handled by the OS, usually using the HID protocol. The Gear VR controller, however, is custom hardware and we can assume the Samsung Gear VR platform developers had to write their own service.
If we peek inside this service, we might be able to figure out how they talk to the Gear VR controller over BTLE and re-implement this mechanism wherever we want! I found the APK for the input service after scanning through some Gear VR threads on forums.xda-developers.com.
The input service package name is com.samsung.android.app.vr.input.service.
Quickly glancing inside
You can open up the APK in a variety of ways:
- Rename the APK to ZIP and use the System Archiver or other unzip tool to extract the contents
- Inflate with apktool.
The resulting output are SMALI files which is Dalvik human-readable bytecode. Code in this format can be manipulated and repacked into an APK with little effort. - Use ClassyShark or other tools to directly open an APK and view the internal structure
You can also extract strings from some of the files inside the APK. This is a great way to scout for particularly helpful debug messages; doing that can quickly tell you if the package has relevant functionality.
For instance, I found the string com.samsung.android.hmt.vrsystem to be particularly interesting:
what does HMT stand for? Thanks to a sharp-eyed fellow Redditor, we now know it is Samsung’s internal codename
for the Gear VR system: Head Mounted Theater.
What was more interesting were strings like the one below, which confirmed this is where we should look for further progress toward the goal.
"Accel: %+05.1f, %+05.1f, %+05.1f \n (%+05.1f, %+05.1f, %+05.1f)"
Next, I extracted the APK as a ZIP (first option) and then used the classes.dex file (contains all the compiled Java classes) within the APK with
dex2jar. Doing this lets allows me to use a Java class decompiler (JD-GUI)
to drill deeper into code that’s a little more human-readable.
This is what it looks like:
This is starting to look more intelligible! We can go through everything to understand what the classes do and how they all fit together.
Identify modules
With some patience and time, a few modules were identified.
1. Software update module
com.samsung.android.app.vr.input.service > package "a"
The first one was the part that handled trusted updated. It was easy to recognize since there were many strings inside this module that made reference to certificates and the names of the hardware manufacturers.
certs/occulus.cert
certs/samsung.cert
Also found inside the decompiled resources were two X.509 certs that contain public keys for verifying that downloaded software updates are coming from either Samsung or Occulus. Out of curiosity, I got the certificate information using:
# DER is the binary format
openssl x509 -inform DER -text -noout -in certs/samsung.crt
openssl x509 -inform DER -text -noout -in certs/oculus.crt
2. Dynamic library loader
com.samsung.android.app.vr.input.service > package "b"
The second module to be identified included code to handle third-party libraries used by the app.
-
com.samsung.android.app.vr.input.service > package “b” > a.class
Dynamic library version query -
com.samsung.android.app.vr.input.service > package “b” > b.class
Dynamic library loader? -
com.samsung.android.app.vr.input.service > package “b” > c.class
Possibly a mechanism to force-close the app on non-Samsung devices
“com.samsung.android.feature.FloatingFeature”
3. Memory mapped device interface
com.samsung.android.app.vr.input.service > package "c"
Its overall purpose may be inferred from “MSM”, which could be “Media State Machine”? It decodes byte arrays received as data from the controller and provides the interface defined by
com.samsung.android.app.vr.input.service/unknown/com/samsung/android/app/vr/input/service/IRemoteInterface.aidl
4. Protocol
com.samsung.android.app.vr.input.service > package "protocol"
Firstly, the little-endian hex strings within package "a" are of
special note since these are the commands sent to the controller.
Next, we find that a.class shows
static
{
e.put("4f63756c-7573-2054-6872-65656d6f7465", "CUSTOM SERVICE");
e.put("c8c51726-81bc-483b-a052-f7a14ea3d282", "COMMAND SEND");
e.put("c8c51726-81bc-483b-a052-f7a14ea3d281", "DATA RECEIVE");
}
This is full annotation of one of the unknown services and along with its characteristics. It tells us we should send commands to
c8c51726-81bc-483b-a052-f7a14ea3d282 and expect to receive data from c8c51726-81bc-483b-a052-f7a14ea3d281. Bingo!
In the meantime, I found another reverse engineering attempt for the Gear VR controller
which used exactly one of these commands
on the COMMAND SEND characteristic to tell the controller to begin sending data. I tried sending these commands with LightBlue and it worked great!
Here are all the commands:
"0000" = Turn all modes off and stop sending data
"0100" = Sensor mode
Send touchpad and buttons data but update at lower rate
protocol > a > o.class
"0200" = ??? initiate firmware upgrade sequence?
"0300" = Calibration mode
protocol > a > n.class
"0400" = Keep-alive command
protocol > a > p.class
"0500" = Setting mode (???)
protocol > a > m.class
"0600" = LPM Enable (LPM = ???)
protocol > a > k.class
"0700" = LPM Disable
protocol > a > j.class
"0800" = VR Mode Enable
high frequency event data update ???
protocol > a > l.class
At this point, I started writing a Web Bluetooth script to make the data inspection easier and more fine-tuned to the information I should be expecting to get back from the device.
Jackpot
com.samsung.android.app.vr.input.service
> package "ui"
> c.class
This class had a lot of debug strings in it, include the previously scouted accelerometer debug string. It contained nearly all the
bit-masks and logic needed to get the correct controller event data. Being cautious of little-endian byte order and the
float, short, and int size differences (and associated typecasts) between Java and Javascript, it was a matter of simple translation to re-implement
the decompiled code in JS using Web Bluetooth to send data and parse the received data.
As I was playing more with the data, I found it helpful to write a histogram for the accumulated entropy of the bytes that were coming back to see where in the byte arrays I should be looking to verify that each byte was being read from the correct index.
As you can see, the troughs in the graph clearly delineate the order and structure of the bytes when the controller is waved around. Having this extra tool while I was playing with the byte arrays made life so much easier. Eventually, I figured out enough of the data to attempt using it in a 3D context.
Sensor fusion and drift
Sensor fusion is all about taking gyroscope (momentary rotation readings), accelerometer (momentary acceleration readings), and in some algorithms, magnetometer (compass readings) readings to reliably output an object’s orientation in 3D space.
Unfortunately, unlike the Google Daydream controller, the Gear VR controller does not seem to do sensor fusion on the device itself, which means we have to do it in software in order to get the actual orientation values (e.g. pitch, yaw, roll or other rotations) that we care about as game developers.
There are two major open-source sensor fusion filter for IMUs that people seem to be using: Mahony and Madgwick. Each has its own biases and characteristics. To my knowledge the key difference is that the Madgwick sensor fusion algorithm will use magnetometer readings to compensate for drift. Therefore it is extra-important to calibrate the magnetometer.
I made a rough attempt at tweaking the values for sensor fusion; it’s still far from perfect. Perhaps I’ll be able to take that further in a later attempt. There is a calibration mode that I still haven’t explored. The answer may yet lie in there.
Takeaways
This project gave me a cursory understanding of how sensor fusion and IMUs work and how the Bluetooth stack works. In regards to pre-existing reverse engineering efforts, I learned how critical it is to always use primary sources (such as the decompiled code) as the first step for figuring something out.
If you have to use shortcuts, verify you understand what the shortcut is doing fully and verify that it’s making all the correct assumptions. Assuming somebody else’s work is correct is an expensive risk time-wise.
Reverse engineering is FUN!
Demo
If you use Chrome and have a Gear VR controller handy, you can try the demo below:
Source code for the project lives here.
Teardown photos
I couldn’t resist a teardown of the hardware.
