Thursday, June 18, 2009

The new Task Queue API on Google App Engine

With release 1.2.3 of the Python SDK, we are psyched to present an exciting new feature - the Task Queue API. You can now perform offline processing on App Engine by scheduling bundles of work (tasks) for automatic execution in the background. You don't need to worry about managing threads or polling - just write the task processing code, queue up some input data, and App Engine handles the rest. If desired, you can even organize and control task execution by defining custom queues. A quick example:

   # for each user, add a task to send a custom email message
   for u in users:
          params=dict(, subject='Hello ' +, body='this is a message!'))
   return # finished now, emails will be sent offline when tasks execute


   # task handler at /work/sendmail, automatically called for each task created above
   class MailWorker(webapp.RequestHandler):
      def post(self):

We're eager to help you learn and experiment with Task Queues. The team recently presented the feature at Google I/O and the video is now available (slides are here). We've also prepared a set of demos to help you get started. And of course, don't miss the feature documentation. The Task Queue API is Python-only for now; we'll have a Java language version available soon.

Please note that the Task Queue API is currently a Labs release - we want to get your feedback on its usability and functionality before finalizing the API. You'll notice that its Python import path currently includes the 'labs' module (google.appengine.api.labs.taskqueue). Before the feature is promoted out of Labs, we may need to:

  • Change the quotas and limits which apply to Task execution (definitely, we hope to raise the number of Tasks you can use per day).
  • Change the API itself if there are usability or functionality issues.
  • Change how we bill for Task Queue usage.

Once we're ready to promote the feature out of Labs, we'll give weeks of notice and provide a transition path for our developers.

Last but not least, the 1.2.3 release is full of other new stuff as well! Stay tuned to the blog for more updates or check the release notes for exciting info on:

  • Asynchronous urlfetch support
  • Django 1.0 support

Visit the Downloads page to get SDK 1.2.3 now!

The Task Queue API is the first milestone of our plan to deliver rich support for offline processing. There's more to come, but we hope the simplicity and power of this first release opens a new range of possibilities for our developers. Try it out and let us know! We'll be watching the Group for your input.

-- The App Engine Team

No comments: