Introducing RESTHooks.org

When you've worked with as many APIs as we have (230+), people start to ask what the best APIs look like. Naturally, we point to things like great documentation, endpoints that follow consistent conventions, human readable error messages, and standard authentication methods. But one thing that makes a big difference is webhooks.

Webhooks are something that everyone can get behind, but relatively few services have implemented. Many times because they don't know where to start.

To help people get started, today we are launching RESTHooks.org, an initiative to teach best practices around implementing REST Hooks (a variant of webhooks) and to showcase companies who have great implementations of REST Hooks.

So what are REST Hooks? From the site:

[REST Hooks are] a collection of patterns that treat webhooks like subscriptions. These subscriptions are manipulated via a REST API just like any other resource.

To dig into this, let's walk through a metaphor…

Polling

Imagine you want to know when your favorite store places that 60" TV you have been eyeing on sale. The naive approach is to call them each day and ask if there is a sale going on.

This repeated questioning is in essence what is happening between two services when their communication relies on polling. One service constantly asks the other for data. The way you make this style of interaction "real-time" is to decrease the time between each poll.

Webhooks

You quickly realize that polling (in this case, calling the store each day) is a huge waste. A much better idea is to have the store notify you when they have a sale. So, you look up the address for the store's promo department, write to them letting them know your address, and they start sending you fliers for deals.

This algorithm is the one webhooks utilizes. The end-user goes to the service they want to receive updates from and enters a URL they copied from the second service, thus allowing the first to notify the second when something interesting happens.

This new approach is all fine and dandy, at least until you want to know about more events. Returning to our metaphor, what if you find out that two more stores in town carry the TV? Wanting the best deal, you write each of them, telling them to send you fliers as well. This is starting to be an intensive process. Worse, what happens when you finally buy the TV and want to stop the fliers? You have to write each store and tell them to take you off the list.

REST Hooks

Let's take this example to the extreme. What if, instead of having to mail every store individually, you could just mail the post office and they would handle putting you on the lists and taking you off. You write one letter to one address saying what stores you want to get notifications from, and the post office handles negotiating with the stores for you. This is REST Hooks!

Using this pattern, your users notify your server of their intent to receive notifications, and you do all the setup for them. After that, the process flows just like webhooks.

Why You, The Engineer, Should Want REST Hooks

Polling isn't a solution to make the web real-time, it's a hack! It should make your inner engineer cringe.

For the pragmatic, REST Hooks are better because they are less work. They are less work for you and for your users.

When you first build your API, it will be a little faster to ship polling, but remember, most of the effort of a project is in maintenance. Eventually, you will add a new resource, forcing you to copy-paste code to slap together an endpoint for this new model. Had you built out a good framework for REST Hooks (or used an existing one like django-rest-hooks), it could be a one-line add (seriously, that Django module only requires a change to settings.py to accommodate a new model).

From your users' perspective, REST Hooks are great because it requires almost no effort on their part. Remember that third twist on the mail metaphor, how streamlined that would be? REST Hooks are that easy for your users. They notify you of their intent to subscribe/unsubscribe, and you handle the negotiating for them. They never have to touch the other service.

Why Your Company Should Want REST Hooks

Hopefully you have a rekindled passion for awesomeness alight in your soul. Now what do you say to convince your company that REST Hooks are the way to go? How about a line like, "I know a way to reduce the number of servers we need while simultaneously increasing the number of clients we can support. Oh, it will also drop our bandwidth usage by orders of magnitude."

That may sound like a joke, but it wasn't for us. We spend roughly 66x the resources on polling as we do on webhooks/REST Hooks. Think about it, if you want a client to have a near real-time experience (let's define that as a five-minute delay), and you have 100 clients using your API, that is 28,800 api calls in a day. How many of those will equate to the client finding new data? For us, it was 1.5%. Meanwhile, REST Hooks are instant and guaranteed to deliver new data. Those are big wins.

As for bandwidth, polling requires larger responses. Clients aren't likely to ask for the newest single item, they'll be asking for the newest 20 or 100. Every response, you will provide data on multiple entities, often repeating uselessly. With a REST Hook, you deliver the single entity that changed. No sacrificed bytes.

A final selling point is that your clients will like you more. They have to spend the same amount of resources making requests as you do processing them. Cutting polling out helps both of your services.

Next Steps

Now that you have a better grasp of what REST Hooks are conceptually, it is time to get involved. You can learn more about implementing REST Hooks from the code examples and docs on the site. To provide your wisdom to the community (or proudly announce yourself as an implementer), fork the site repo and issue a pull request. Together, we can change the web!

Visit RESTHooks.org to learn more