Twenty Minutes into the Future - Hacking the Galaxy Gear Smartwatch to run Google Glass firmware

Earlier this year I wrote one of my most popular blog posts “Hacking the Google Smartwatch” about the hacking potential of Motorola’s Moto Actv smartwatch. In this post I’ll turn my attentions to the Samsung Galaxy Gear, which launched in Autumn 2013 and has already been quite comprehensively hacked to vastly expand its capabilities. I’ll conclude by showing you how you can run the Google Glass software to get an early peek at how Glass will work.

It's interesting to see what a couple of years’ difference makes in terms of capabilities, market positioning and most importantly – hacking potential. The Moto Actv originally came out in 2011, and was originally marketed as a sport and fitness accessory, whereas the Galaxy Gear is intended to be a "companion device" to your phone. However, both run Android under the hood and can readily be converted into general purpose computing devices. Let’s take a moment to compare the specifications of the two devices:

Moto Actv: (from the Moto Actv wikispaces site)
  • 600MHz OMAP3 (3630) ARMv7 CPU
  • 256MB of RAM and 8GB of NAND Flash Memory (Or 16 GB)
  • 802.11B/G/N
  • Bluetooth 4.0 (low-power mode)
  • ANT+ for connectivity to fitness sensors (heart rate, etc.)
  • FM Tuner (with Radio Data System for station & song identification)
  • PowerVR GPU
  • 1.6" 220x176 capacitive multitouch LCD display
  • 5 additional hardware buttons on the side (Start, Music, Volume +, Volume -, Standby)
  • 1 additional capacitive button on the front for 'back button'.
  • Ambient light sensor
  • White charging and notification LEDs
Galaxy Gear: (from the Samsung Galaxy Gear Wikipedia page)
  • 800MHz Exynos CPU
  • 1.63-inch Super AMOLED display at 320x320 resolution
  • 1.9MP camera with BSI sensor
  • 720p video recording and playback
  • 2 microphones, 1 speaker
  • Bluetooth 4.0 and LE
  • Accelerometer, Gyroscope
  • 4GB on-board storage
  • 512MB RAM
  • 315mAh battery
I noted previously that the Moto Actv was (at the time) probably the closest proxy to the Google Glass experience that was commercially available at an affordable price. The Galaxy Gear takes us much closer to this point, with its camera, microphones and speaker. And if you missed the spoiler image at the top of this post - yes you can now run the Google Glass software on your watch, with only a small amount of hacking ☺

The Galaxy Gear is available to purchase in its own right, but Samsung have been pursuing a bundling strategy to increase the number of deployed units – e.g. you might well find that you get a free Galaxy Gear if you sign up to a new Galaxy Note on contract. A nice side effect of this is that there is a ready supply of nearly new Galaxy Gears going cheap on eBay, either because people are looking to get cash back on their new phone, or because they didn’t get on with the Gear in the first place. The advertising campaign for the Gear also struck a bit of a bum note, even before that video...

Whilst the Gear runs Android (and hence Linux under the hood), it is not “meant” to be a general purpose computer. Samsung’s vision is very much for it to be a connected accessory, and Android was a convenient ecosystem that already had developer mindshare. However, from a hacker’s perspective Samsung have been very canny about how they positioned the Gear – in particular by leaving the bootloader unlocked, opening the door to custom ROMs (system images), and by giving knowledgeable Gear owners a trivial way to sideload existing Android APKs (software packages).

It’s a little known fact that Android’s “home screen” (launcher) and lock screen are normally user replaceable parts. Unlike Apple iOS devices, if you dislike the default experience your vendor or phone company ships, you can simply change it to one that you are more comfortable with. There is a thriving market for replacement launchers and lock screens, and a number of highly customized alternatives exist – such as the Eyes Free Shell for the visually impaired, and Nova Launcher for smartwatches and other low resolution / small screen devices. Here's a video from High On Android that shows what's involved...

Hacking the Galaxy Gear

It’s become the norm for the Android hacker community to rise to the challenge whenever a new device becomes available, and it was no surprise that shortly after the Galaxy Gear was launched it was thoroughly dissected and an alternative ROM produced by Android developer fOmey. This provides both the Samsung “companion device” tools, and a normal Android user environment using Nova Launcher by default.

fOmey’s “null ROM” has a number of enhancements over and above the stock Samsung ROM, which can be combined with various insights gleaned by the Android hacker community to give us the following additional features that were not part of the original Galaxy Gear product:
  • Bluetooth tethering for Internet access via the peered phone or tablet –US readers should take note that their telco may frown upon tethering or attempt to block it altogether.
  • Direct access to the Google Play Store, so no longer necessary to sideload mainstream APKs.
  • Support for peering any device with the watch, rather than just Samsung phones and tablets.
  • Support for Bluetooth GPS, using a companion app on the peered device to relay the phone/tablet GPS coordinates to the watch.
Shortly after the Gear’s release Samsung also produced an update which permitted notifications from any application to be relayed from the peered phone/tablet to the Gear – the watch initially launched supporting notifications from only a very small set of applications, which was a major criticism from reviewers.

Putting all these things together, the Gear is now fully functional as a device in its own right, with the proviso that it will be relying on the peered phone or tablet for Internet and/or GPS access.

OK, so what can we do with it?

It’s also interesting to consider that Samsung have yet to release their official software development kit (SDK) for the Galaxy Gear, so if you want to write an application for it then for the moment you will need to do so using the normal Android development tools or an add-on such as the Scripting Layer 4 Android (SL4A) software.

SL4A is very interesting for a whole range of reasons, as it lets you run common scripting languages such as Ruby, Python and Perl with access to the full range of Android Application Programming Interfaces (APIs). The Android APIs give the developer access to the device’s microphone, camera, loudspeaker and other sensors, and Internet services if Bluetooth tethering has been enabled.

Regular readers will know that I have a particular interest in assistive technology, and the Gear has great potential here to be the “Seeing Eye Droid” that I described in a recent blog post. With its camera, speaker and microphone it can take a picture, send it to an online service for Optical Character Recognition, and then read the results out aloud.

Whilst this could be done with a regular smartphone, the smartwatch is particularly attractive because the user (who may also have a cane or guide dog) does not have to worry about dropping it, and the Seeing Eye application could be rigged to run as the device’s launcher. The Galaxy Gear can be configured to “wake up” when the owner shakes their wrist, so the whole process could be made a simple as shaking your wrist, then pointing the camera at the writing you want to hear decoded.

Another simple hack is to switch your input device to Swype or the new Google Keyboard, which give you (against all the odds!) a usable keyboard even on the tiny watch screen. This is possible because of the built-in dictionary and error correction that these keyboard applications carry out. With full Android in place (e.g. via the null ROM) you can also pair a Bluetooth keyboard with your watch, which is handy for configuring fiddly applications or long hacking sessions.

Experienced Android developers will also be aware that they can use the “adb” utility from the Android SDK to obtain a Linux shell on their watch, install and remove packages (that sideloading we were talking about earlier) and carry out low level system modifications.

Even without any serious hacking activity on the Galaxy Gear, it is hugely impressive to fire up Google Translate in conversation mode, and then use the watch to translate in real time between two languages. The Star Trek Communicator is really very, very close to being a reality:

It’s no coincidence that in the latest Google Experience Launcher for Android 4.4 (Kit-Kat), the microphone is permanently enabled by default, and your device is always listening out for you to say “Google” and issue a voice command.

Got Glass?

What is even more impressive, though, is to run the Google Glass software on your watch, and then interact entirely with it by voice command. On your phone or tablet you might occasionally carry out a voice search or dictate a message, and feel slightly self-conscious about it. On a watch you have very limited screen real estate to play with, and it quickly becomes second nature to use voice commands to carry out some quite complex tasks. Here’s a short video to whet your appetite:

What we’re doing here is running the “Glass Home” application as our launcher, which is permanently listening out for someone to say “OK Glass”. Then we give it a voice command to carry out a Google search “How tall is the Eiffel Tower?” (and swipe through some further information about the Eiffel Tower), ask it to translate a phrase “Where is the train station?” into Spanish, and obliquely check the weather forecast for London by asking "will I need to take an umbrella?".

No comments:

Post a Comment