Using the Slack Outgoing Webhook as an example

Webhooks (aka HTTP callbacks) are a fantastic way to connect all the various services we as developers depend on now. Unfortunately, I haven’t see a great walk-thru of how to make a HTTP callback receiver/consumer. So while you’re going to see a lot of code below, you’re not going to see a lot of explanation for why I made the choices I did.

Scenario for Consideration

Let’s say we want to capture Slack messages for some reason. Here’s some documentation on what that outbound webhook looks like: https://api.slack.com/outgoing-webhooks

Setup and Install Dependencies

Go create a new django project, and for your database, let’s use Postgres. We’re going to want to store JSON, so let’s install django-hstore. Also, lets suppose we’re going to do something relatively computationally intensive with the data, so we won’t want to slow down our webhook response just to process the content, so let’s install celery too. (All of this stuff will take you a bit. I’ll wait for you to get back.)

Our Webhook App

Now that you’re all done with that, lets start a new app and name it `slack_messages`. (Be sure to add it to your INSTALLED_APPS in your settings file.)

$ python manage.py startapp slack_messages

Here’s a sample model to hold a bunch of generic information related to the webhook event:

# slack_messages/models.py

from django.db import modelsfrom django.utils import timezonefrom django_hstore import hstoreclass WebhookTransaction(models.Model):    UNPROCESSED = 1    PROCESSED = 2    ERROR = 3    STATUSES = (        (UNPROCESSED, 'Unprocessed'),        (PROCESSED, 'Processed'),        (ERROR, 'Error'),    )    date_generated = models.DateTimeField()    date_received = models.DateTimeField(default=timezone.now)    body = hstore.SerializedDictionaryField()    request_meta = hstore.SerializedDictionaryField()    status = models.CharField(max_length=250, choices=STATUSES, default=UNPROCESSED)    objects = hstore.HStoreManager()    def __unicode__(self):        return u'{0}'.format(self.date_event_generated)

class Message(models.Model):    date_processed = models.DateTimeField(default=timezone.now)    webhook_transaction = models.OneToOneField(WebhookTransaction)    team_id = models.CharField(max_length=250)    team_domain = models.CharField(max_length=250)    channel_id = models.CharField(max_length=250)    channel_name = models.CharField(max_length=250)    user_id = models.CharField(max_length=250)    user_name = models.CharField(max_length=250)    text = models.TextField()    trigger_word = models.CharField(max_length=250)

    def __unicode__(self):        return u'{}'.format(self.user_name)

Here’s a good way to process the incoming webhook and keep a lot of metadata on what happened when for auditing later.

# slack_messages/views.py

import copy, json, datetimefrom django.utils import timezonefrom django.http import HttpResponsefrom django.views.decorators.csrf import csrf_exemptfrom django.views.decorators.http import require_POSTfrom .models import WebhookTransaction@csrf_exempt@require_POSTdef webhook(request):    jsondata = request.body    data = json.loads(jsondata)    meta = copy.copy(request.META)    for k, v in meta.items():        if not isinstance(v, basestring):            del meta[k]    WebhookTransaction.objects.create(        date_event_generated=datetime.datetime.fromtimestamp(            data['timestamp']/1000.0,             tz=timezone.get_current_timezone()        ),        body=data,        request_meta=meta    )    return HttpResponse(status=200)

Be sure to connect it up to your project urls. (Yes, I’m skipping putting it in the slack_messages app urls, but that’s just for brevity.)

# project_name/urls.py

from django.conf.urls import include, url, patternsfrom django.contrib import adminfrom slack_webhook import viewsurlpatterns = [    url(r'^admin/', include(admin.site.urls)),    url(r'^webhook', views.webhook, name='webhook'),]

Finally, let’s make a task to process this webhook. You’ll note that I separated out the selection of get_transactions_to_process and process_trans into their own methods because those are the two areas that will vary. The main method run is relatively reusable, so you may want to make a mixin out of it.

# slack_messages/tasks.py

from celery.task import PeriodicTaskfrom celery.schedules import crontabfrom .models import Message, WebhookTransactionclass ProcessMessages(PeriodicTask):    run_every = crontab()  # this will run once a minute

    def run(self, **kwargs):        unprocessed_trans = self.get_transactions_to_process()        for trans in unprocessed_trans:            try:                self.process_trans(trans)                trans.status = WebhookTransaction.PROCESSED                trans.save()            except Exception:                trans.status = WebhookTransaction.ERROR                trans.save()    def get_transactions_to_process(self):        return WebhookTransaction.objects.filter(            event_name__in=self.event_names,            status=WebhookTransaction.UNPROCESSED        )

    def process_trans(self, trans):        return Message.objects.create(            team_id=trans.body['team_id'],            team_domain=trans.body['team_domain'],            channel_id=trans.body['channel_id'],            user_id=trans.body['user_id'],            user_name=trans.body['user_name'],            text=trans.body['text'],            user_id=trans.body['user_id'],            trigger_word=trans.body['trigger_word'],            webhook_transaction=trans        )

Conclusion

So there you have it. Some moderately generalized boilerplate for a webhook receiver, including async processing and audit history. Hope it helps someone!