Webhooks

Polling is asking "are we there yet?" every five minutes on a road trip. Webhooks are your friend texting you when they've arrived.

That's the entire concept. Instead of your application repeatedly asking a service "has anything changed?", the service tells you the moment something does. You register a URL, and when events occur, the service sends data directly to you.

This seemingly small inversion—who waits for whom—changes everything.

The Webhook Pattern

Three steps:

Registration. You give the service a URL: https://yourdomain.com/webhooks/payment-received. The service stores it.

Event occurrence. Something happens—a payment completes, a user signs up, a file finishes processing.

Delivery. The service immediately POSTs a JSON payload to your URL. You process it and respond with HTTP 200.

That's it. No polling loops. No wasted requests. No artificial delays.

Why This Matters

Polling forces you to choose between responsiveness and efficiency. Poll every second? You'll hammer the API with requests that mostly return nothing. Poll every minute? You'll miss time-sensitive events by up to 59 seconds.

Webhooks eliminate this tradeoff. Events arrive the moment they happen. You only receive requests when there's actually something to process. The service bears the burden of watching for changes—not you.

The cost: you need to host an endpoint that accepts incoming requests, handle delivery failures, and secure that endpoint against abuse.

What Webhooks Look Like

A typical webhook payload:

{
  "event": "payment.completed",
  "timestamp": "2024-01-15T10:30:00Z",
  "id": "evt_abc123",
  "data": {
    "payment_id": "pay_xyz789",
    "amount": 49.99,
    "currency": "USD",
    "customer_id": "cust_123456"
  }
}

The event type tells you what happened. The ID lets you detect duplicates. The timestamp tells you when. The data contains everything relevant to that specific event.

Services send many event types to the same URL—payment.completed, payment.failed, payment.refunded—so your handler routes by event type.

Security: The Unsigned Webhook Problem

Here's the danger: your webhook URL is just a URL. Anyone who discovers it can send fake events. Without verification, an attacker could POST a fake payment.completed event and trick your system into fulfilling an order that was never paid for.

Services solve this with signatures. They sign each webhook payload using a secret key only you and the service know. Your application verifies the signature before trusting the payload:

const crypto = require('crypto');

function verifyWebhookSignature(payload, signature, secret) {
    const expected = crypto
        .createHmac('sha256', secret)
        .update(payload)
        .digest('hex');

    return crypto.timingSafeEqual(
        Buffer.from(signature),
        Buffer.from(expected)
    );
}

Never process unsigned webhooks. Also: use HTTPS (never accept webhooks over plain HTTP), check timestamps to prevent replay attacks, validate payload structure before processing, and implement rate limiting.

The Speed Trap

Webhook handlers have a time limit—typically a few seconds. If your endpoint doesn't respond quickly, the service assumes delivery failed and retries.

The mistake: doing the actual work in the webhook handler. Processing a payment, sending emails, updating databases—all of this takes time. Do it synchronously and you'll timeout.

The pattern: acknowledge immediately, process later.

app.post('/webhooks/payment', async function(request, response) {
    if (!verifySignature(request)) {
        return response.status(401).send('Invalid signature');
    }

    // Queue for background processing
    await queue.add('process-payment-webhook', request.body);

    // Respond immediately
    response.status(200).send('OK');
});

Your endpoint becomes a fast relay: verify, queue, acknowledge. The actual processing happens asynchronously.

The Duplicate Problem

Services retry failed webhooks. Networks hiccup. Sometimes the same webhook arrives twice—or more.

If your handler processes every webhook it receives, you might fulfill the same order twice, send duplicate emails, or corrupt your data.

The solution is idempotency: make your handler safe to call multiple times with the same event.

async function processWebhook(event) {
    const exists = await db.webhookEvents.findOne({ id: event.id });
    if (exists) {
        return; // Already processed
    }

    await handleEvent(event);
    await db.webhookEvents.insert({ id: event.id, processed_at: new Date() });
}

The event ID is your key. Check if you've seen it. If yes, skip. If no, process and record.

Testing Without the Service

Webhooks are awkward to test. They require an external service to trigger them, and that service needs to reach your endpoint.

During development, your localhost isn't reachable from the Internet. Tools like ngrok solve this:

ngrok http 3000

This creates a public URL (like https://abc123.ngrok.io) that tunnels to your local server. Point the service's webhook configuration at this URL, and webhooks flow to your development machine.

For automated tests, mock the webhook:

test('handles payment webhook', async function() {
    const payload = {
        event: 'payment.completed',
        id: 'evt_test_123',
        data: { payment_id: 'pay_test_456' }
    };

    const response = await request(app)
        .post('/webhooks/payment')
        .send(payload);

    expect(response.status).toBe(200);
});

Most services also provide webhook simulators in their dashboards—buttons that send test events to your registered URL.

Where Webhooks Appear

Payment processors (Stripe, PayPal) notify you when payments complete, fail, or refund. You update order status and fulfill purchases immediately.

Code repositories (GitHub, GitLab) notify you about pushes, pull requests, and issues. CI/CD pipelines trigger on these webhooks.

Communication services (Twilio, SendGrid) notify you about message delivery, incoming SMS, email opens and clicks.

File processing services notify you when uploads finish processing, thumbnails generate, or videos finish transcoding.

Monitoring systems push alerts when servers go down, error rates spike, or thresholds are exceeded.

Building Webhooks for Others

If your service offers webhooks:

Sign everything. Use HMAC signatures so receivers can verify authenticity.

Retry with backoff. When delivery fails, retry after increasing intervals—1 minute, 5 minutes, 30 minutes. Give receivers time to recover from temporary issues.

Provide visibility. Show delivery attempts, responses, and failures in a dashboard. Let users trigger test webhooks.

Respect persistent failures. If a URL fails repeatedly, disable it and notify the user rather than hammering a dead endpoint.

Document thoroughly. Payload schemas, event types, retry policy, signature verification code—everything a receiver needs to integrate correctly.

Webhooks vs. Alternatives

Server-Sent Events (SSE) maintain open connections for server-to-client updates. Good for browsers, but requires persistent connections.

WebSockets provide bidirectional real-time communication. More complex, but enables two-way messaging.

Message queues (RabbitMQ, AWS SQS) provide reliable asynchronous messaging. More infrastructure, but more guarantees.

Polling remains appropriate when webhooks aren't available, updates are infrequent, or you can't host an endpoint.

Webhooks persist because they're simple: standard HTTP, stateless, work through firewalls, minimal infrastructure. For most event notification needs, they're the right tool.