Webhooks for Productive Developers
Keeping it simple is usually the best course of action in most things. There are a bunch of problems that can be solved without your development environment. For the many cases that require moving data or alerting when data changes, webhook notifications are a useful pattern to reduce–and sometimes, eliminate–the code you need to write.
Webhooks by Zapier was one of our earliest and most popular apps. Thousands of people of all skill levels use it to instantly move data, make API requests, and send alerts to their own systems.
Many services now support webhooks, allowing developers to automate manual processes and disable cron jobs in favor of real-time updates. This post covers some of the common ways we see Webhooks used at Zapier and some ideas for how you can put them to use in your projects.
What Is a Webhook and How Does It Work?
Webhooks are one way that apps can send automated messages or information to other apps. They're a simple way your online accounts can "speak" to each other and get notified automatically when something new happens. If you’re brand new to the term, our Webhooks Guide provides a great introduction.
From a technical perspective, webhooks are HTTP calls that typically fire off when new data is available and contain a payload that can be acted upon. Webhooks are best described by the alternative, which is to continuously poll a service to see if there’s anything new. Instead, that service would send a webhook when there is something new.
For example, Stripe sends notifications of payments and MailChimp lets you know when there are new subscribers.
Some services support webhooks directly, though not always by that name. You might see "callback URL," "instant notifications," or "HTTP server push" mentioned. Generally, you can think of webhooks as a way to pass messages between different services, and, as a developer, you can get your internal systems in on the fun.
The Webhooks Handshake
Like any HTTP call, there are two sides to a webhook:
- Sender: has some new data and sends it to an expectant party.
- Consumer: wants to receive and likely act upon data as soon as it’s available.
A successful webhook transmission has the sender make an HTTP call to a predefined URL from the consumer. Usually the call uses the POST method, with a JSON object in the body. When the consumer’s server has received the data, it responds with a 200 OK status code, like a programmatic thumbs up to the sender.
If the sender does not receive a 200 OK, typically it will retry the webhook multiple times. After all, that data is important to the consumer, so you want to make sure they get it!
Tracking Your Webhooks
Whether you’re the sender or consumer, you’ll want a way to track the webhooks you’ve sent or received. Further, if you’re a sender, you’ll want both a visual interface to webhooks and an API endpoint for webhook subscriptions—you can follow these webhook design guidelines.
As a consumer, logging may be all you need to track your webhooks. Depending on how much you rely upon them, you may prefer an interface to show statuses.
For example, at Zapier we expose webhook history to end users, who may not even write code for their company. Still, they want to see the data they received, when the webhook arrived, and how they acted upon the data.
Listen for Events from a Service
Webhooks abound. Many SaaS and developer products support webhooks for quickly getting notifications wherever you need them. While Zapier has 1,000+ apps and growing, it doesn’t always support the specific service or use case you need. However, any webhook can be connected to the apps we support (including other webhooks). That makes Zapier a useful tool for wiring together your workflows.
For example, GitHub has granular webhook options to receive any event you want about a repository:
Let’s say you want to know immediately when new forks or branches are created. From your GitHub repository, choose Settings, then Webhooks. By default, GitHub sends just the push event. You want to select your events, so when you choose that option, the list like the image above will appear.
- Uncheck Pushes, and check Branch or tag creation and Forks
- Add a Payload URL to receive the webhook (use your own server, a Zapier webhook, or a tool like RequestBin)
- Click Add Webhook
Now create a fork, branch, or tag for the repository and check for a result at your Webhook URL.
Use the Zap workflow above, which walks you through the process to connect your webhook to Slack. You’ll always know when there’s something new to see in your GitHub repo. Repeat the process with any service that supports webhooks.
Send Yourself the Right Data
Some webhooks are chattier than others. How much data is sent and how often depends on the use case and the way the sender structures subscription options. You may receive a firehose of data, but only need to respond to a small subset.
One way of handling a chatty webhook is to include logic inline: Unless certain conditions are met within the data, ignore it. You’ll still want to return the 200 OK and probably even want to log the data that came through. The inline approach may work for one-off webhooks, but you’ll probably end up creating a queue if you do a few of these.
If you’d rather not over-engineer your solution, another approach is to use Zapier as both the inline logic and log of webhooks consumed.
Instead of pointing your chatty webhook at your own servers, point it at Zapier, then send only the data you need along to your server. You can use a filter step to store your inline logic that tells Zapier whether to send the webhook on to your servers.
Make API Calls Directly
Zapier’s Webhooks app can both consume and send webhooks, and it also makes data from the response of a sent webhook available to the rest of the Zap. That means, we can create one-off API calls, or even a series of API calls in multiple steps of a workflow.
For example, let’s say we want to add a contact to SendGrid. Using its Contact API, I can do that with an API call. Here’s how it would look in Zapier, assuming my data came from an app's trigger or another webhook.
Here I am using a Webhook action step. In this case, it’s a Custom Request, because SendGrid requires the data to come through as an array, even if there’s a single item. So, I need to create the raw data for Zapier to pass along. I am able to include values from previous steps to insert, as I did for the names and email address. For APIs that can accept simple JSON objects, you can provide the data in easier to read key/value fields and Zapier will construct the object.
Not shown in that screenshot is authentication. SendGrid uses API keys in a header, something a Custom Request webhook makes fairly straightforward. In this case, the header key is Authentication and the value is Bearer API_KEY (you can generate keys in the SendGrid settings).
Here’s the result when that Zap step is tested:
If needed, you can add subsequent steps to your Zap that uses the recipient identifier from the response. For example, SendGrid has another API call that adds recipients to specific lists. Here’s how that would look, with another Custom Request:
You can imagine how this series of calls might be useful if, for example, your marketing team has leads coming in from landing pages or online advertising. Using these outbound Webhooks, you’re able to make these API calls without having to write code or route data through your own system.
Fan Out to Multiple Webhooks
Earlier we filtered a webhook to send just the right data. In that case, we were essentially passing along a single webhook. What if we wanted to do that more than once? Yes, you can use Zapier to multiply and amplify your webhooks.
Creating two or more outgoing webhooks from one incoming webhook is especially useful when a service only allows one webhook URL for everything. That’s the case for PayPal and its Instant Payment Notification (IPN) system. Zapier has long been a way to use multiple IPNs with PayPal. You could add purchases to your accounting software, for example, while sending on the raw webhook to an e-commerce plugin or other service that expects to consume an IPN.
Similarly, you can take any webhook in and send it off to multiple webhook consumers. By catching a raw webhooks, where Zapier does not attempt to parse the data, you are able to pass on the exact payload that came with the original Webhook. Alternatively, you can construct just the data you want to pass along with each webhook, giving you granularity with how the webhooks are fanned out.
In the above case, my Zap catches a raw webhook, then in step 2 sends the actual JSON contents to another site. That is repeated in step 3 with yet another webhook.
Additional steps might send that data to one of the 1,000+ apps directly supported, or you could pull out portions of the data to send as API calls where a service doesn’t expect an entire webhook payload. At a minimum, you’ve replaced the infrastructure needed to fan out the webhooks yourself.
Add Webhooks to Your Workflow
We’ve seen a couple examples of using Webhooks, but you can
explore many other webhooks ideas. Incorporate these patterns into your work and you can spend more time solving problems that require original code.
Comments powered by Disqus