The BugSense hybrid app: experiences using Clojure on Google App Engine

Today’s post comes to us from Jon Vlachogiannis and Panos Papadopoulos, founders of BugSense, a mobile error analytics service. We hope you find their insights on using Clojure on Google App Engine informative.

BugSense is a cross-platform error analytics infrastructures for mobile devices. BugSense uses Google App Engine to power its backend, processing more than 1.6 million daily errors, generated by more than 45 million devices around the world. Chances are one of the applications installed on your smart phone (like SoundCloud or Trulia) is already using BugSense.

The Problem
Lots of our clients want to optimize and protect their mobile apps (through code obfuscation) using ProGuard. ProGuard creates more compact code, resulting in faster transfer across networks, faster loading, and smaller memory footprints. On top of that it makes programs and libraries harder to reverse-engineer.

However, because the Android Market doesn't automatically de-obfuscate of stack traces from ProGuard-ed apps, developers who want to analyse errors from their apps must get the stack trace from the Market, format it and use ProGuard locally. The whole process for just a single error could take more than 3 minutes, so we decided to add support for ProGuard to BugSense to make debugging easier and faster.

The Solution: Clojure and Python
The main data-serving portion of our app is written in Python, our language of choice, but ProGuard is an open source project in Java. For easier development, we ported parts of ProGuard to Clojure, a dynamic language belonging to the Lisp family that runs on the JVM. This allows us to “beat the averages” by exploiting all the great features that a LISP language offers (such as macros  and exploratory programming). Using Clojure and having access to a vast number of Java libraries assisted us in tackling the difficult problem of de-obfuscation, with great results.

Once we were done, we deployed using AppEngineMagic and now it's trivial (one click) for our users to de-obfuscate their stacktraces. Now we have the best of two worlds: Python for serving data and Java/Clojure for doing calculations, all in the same Google App Engine application. And it scales automatically and runs even faster than running ProGuard on your laptop!

Practically, that means that we can have a heterogeneous app on Google App Engine so that we can keep programming in our favourite language, Python, but still harness the tremendous wealth of Java libraries using Clojure. Running a hybrid app on App Engine is trivial since they share the same resources task queues, Datastore, and memcache.

However, because our app is implemented in multiple languages, we need to start two different local instances (one for Python and one for Clojure). We use a combination of mocks for both of the instances in order to emulate the hybrid app and their interaction in a local environment for development and testing.

Google App Engine, a success factor
We started as a two-developer startup and our product rapidly became popular across the world. Building on Google App Engine helped us focus on product development and forget about infrastructure and administration, thus enabling us to focus more on our customers' needs. (And sleep tight at night.) Furthermore it helped us to keep costs low and iterate quickly.

To learn more about BugSense, check out our website. If you have comments or questions about this post or just want to reach out directly, you can find us at +jonromero or +bugsense.