Skip to content

How to improve webhook management

The manage.gocardless.com admin interface lets you see the webhooks it has sent, and tells you whether they were successfully sent or not. You can also "retry" (send again) webhooks and it will automatically retry ones that failed (presumably according to some policy).

When you have a lot of Direct Debits, there's a lot of information that gets sent by GoCardless through webhooks. This will grow linearly with the number of direct debits you have set up, yet GoCardless only allow a fixed amount of time to process these webhooks. This can lead to either you losing data (webhooks fail, including partial failure) or GoCardless repeatedly sending the same data over and over because it thinks it failed.

Also, it can be nice when troubleshooting to see more of what is going on.

Install the mjwshared extension v1.2+

The mjwshared extension includes various helpers for payment processors. If you use Stripe, you'll have it installed already.

One feature of it is webhook queuing. Once you have this installed alongside GoCardless (v1.12.0) webhooks sent by GoCardless will just be queued, instead of getting processed in real-time. This means we're much more likely to be able to process the request within GoCardless' time limit (currently 10s).

The queued webhooks get processed by a scheduled job.

Beyond installing the mjwshared extension, there is nothing more you need to do for it all to work.

Monitoring

The manage.gocardless.com site's developers page is still useful for checking that the webhooks are getting delivered. But the mjwshared extension also provides a page at Administer » CiviContribute » Payment processor webhooks which lists all the events received through webhooks, alongside a status.

  1. when a webhook is received, the GoCardless extension will validate it and parse it into separate events and put those events on the queue with status 'new'.

  2. when the scheduled job runs (part of mjwshared), the new events are set to status 'processing', and then will be changed to either 'success' or 'error', alongside a message. You can click Details to see the whole event, and the ID of the webhook it was received in.

  3. If you have set a time limit on the scheduled job, and the entire queue could not be processed within that time limit, the unprocessed jobs will be set back to 'new'.

This admin tool also lets you retry or delete a received event, though you really need to be sure in your understanding of when to do that.

The 'message' shows a very simple one liner about the event, but the troubleshooting logs will contain a lot more information - just search on the event ID shown.

screenshot of CiviCRM webhook overview

  • The two times are: time received (queued), and time processed (will be blank if 'new').

  • The processor shows the name of the payment processor (e.g. in this screenshot the org uses 2 GoCardless accounts, so they're a bit different. You will also see Stripe in here if you use that.)

  • Event ID. This is a GoCardless ID for the particular event

  • Message. Summary, for successful events it will show the IDs of Contribution and ContributionRecur entities affected. Note the different types of events, e.g. "OK PaymentsFailed completed" means, OK: we processed it properly, PaymentsFailed is the GoCardless event type meaning somebody could not pay this month (normal), then there's 2 IDs.

  • Clicking Show Details opens up more info. The Identifier field shows the GoCardless webhook ID, but generally Event IDs are more helpful.