Share article:
Best Practices for Webhooks
Webhooks are awesome when you’re running an application that requires updates in real-time. However, you’ll need to be careful when using webhooks, as they can be your best friend but also become your biggest liability. We’ve discovered some common issues and caveats that need to be considered when using webhooks.
Webhook Security
If you’re familiar with webhooks then you’re already aware there is a very obvious issue with them – trust. There is nothing to stop anyone from hitting the webhook’s destination endpoint and sending messages with the same body of the request to potentially manipulate the system. One important thing to consider when you’re creating an API that receives webhooks on your side is how are you going to authenticate and protect any malicious requests.
The easiest way is to look at the address the webhook originates from and filter based on that. Fairly simple solution, however, if your system is hosted on the cloud where your IP address is constantly changing, then it can be painful for the customers to find out where the webhooks will be originating from. Unless you, as a provider, have locked down the range of IP addresses.
Another way to make it secure is to include some sort of a shared secret between your application and the provider that is sending the webhooks to you. Typically, this is done as a query parameter in the callback URL string. A webhook is basically an API request to your platform. So you could potentially add some sort of shared secret to the webhook on the provider side that your application knows and trusts. Now when you do receive a webhook from that provider you can check for that shared secret and if it’s not present, then you can safely assume that the originator of that webhook is trying to do something malicious to your system.
A similar, probably better, way is if you can configure the headers that get sent from the provider. Perhaps, you can configure the headers to include an authorisation stream so when you receive the webhook, you can check for the username and password in the authorisation header and if it matches, then your webhook is good to go.
The fourth and most secure way would be to attach a cryptographic signature to the request and then your application needs to verify that signature. Bear in mind, the ability to do this entirely depends on the provider having implemented the functionality for you to consume. If they don’t, there isn’t anything that you can do to attach a signature to the request.
Scalability
Let’s say that the dental clinic you go to, which has thousands of branches, sends appointment reminders to all of its patients at 9am everyday. Now, if everyone’s checking their mobile phones and replying with a ‘yes’ or ‘no’ quickly, that means there is going to be a huge flood of webhooks going back to the system. If they’re not processed in a timely manner, it can lead to back pressure on the webhook provider. This will eventually lead to timeout errors and unless there is a proper fall-back system in place, this could cause a lot of issues. A simple yet effective pattern to use in such a case would be to separate the receiving logic from the business logic into two separate parts and join them using an asynchronous connection like a queue. This way you’ll keep your receiving logic lightweight and simple by having it only validate the webhooks and once that’s done, just drop it into the queue and let the business logic handle the rest. It’s a wise idea to implement this in the early stages of your system rather than later. If not, it becomes quite challenging to do so especially when you’ve got live web traffic coming to your system.
Be a good HTTP citizen
Since we’re dealing with APIs, the responses that you send back are going to be interpreted in some way by the provider. Just like you’d expect an API to respond appropriately with 200, 400 or 500 response codes, you should be responding with appropriate response codes. Not every provider will have a response code handling mechanism on their hand, but it’s considered good practice to do so. Why? Well if you respond with a 200 response, the provider’s system will know that you’ve successfully consumed the webhook. Maybe something went wrong, and you respond with a 500. The provider’s system may replay that webhook later on because there is some sort of temporal problem with your system and they’ll retry that webhook later on for you to actually process when everything is up and running again.
Be prepared to fail
Try to make your webhooks idempotent. Things might go wrong, and you might face network connection issues between the provider and your system, causing your system to fail to respond in time which could cause the webhook to be retried multiple times. Now if you’ve already processed the webhook and if the same webhook is accidently pushed to your system again, then your system should take some sort of idempotent action where it does not repeat the same action that it took for the initial webhook. The bottom line is to be prepared, for both expected and unexpected failures and having application logic in place to handle those situations in a graceful manner.
The benefits of webhooks in a nutshell – instant, real-time notifications. The downside, as you’ve read, is the lack of security which can be mitigated by implementing certain authentication and verification mechanisms into your system. We have a comprehensive API documentation on how to manage webhooks in your app and SDKs that can be found on our Github repo.
