The Time I Tried Clerk Webhooks to Persist Users in MongoDB (and What Went Wrong)

Introduction: A Dream outcome faced by challenges

As a developer, I love exploring new tools and methods to solve old problems. Recently, I set out to integrate Clerk — a user authentication service — into one of my projects. The goal? To use their webhooks system to automatically persist user data in MongoDB. On paper, it seemed straightforward. After all, I had worked with similar technologies before, but I was eager to dive into this new approach, confident that this was the future.

What followed was a journey of missteps, outdated documentation, and the realization that sometimes, new isn’t always better. Let me take you through my experience, the mistakes I made, and the valuable lessons I learned.

The Setup: A New Challenge Awaits

I’d used Clerk before, but only through its API. This time, I wanted to go further. I was intrigued by their webhook system. Instead of manually persisting users in the database right after authentication, I thought, “Why not let Clerk handle the heavy lifting?”

The plan was simple:

1. Listen for the user.created webhook event when a new user registered.
2. Persist the user’s details (like email, name, and profile picture) in MongoDB.
3. Profit from the seamless automation.

I grabbed Clerk’s webhook documentation, set everything up, and felt that rush of excitement that comes when you’re about to try something new. This is going to be great, I thought.

Here’s the initial webhook handler code I copy paste from the doc, and modify it a bit:

import { clerkClient } from "@clerk/nextjs/server";
import { WebhookEvent } from "@clerk/nextjs/server";
import { headers } from "next/headers";
import { Webhook } from "svix";
import { createUser } from "@/lib/actions/user.actions";

export async function POST(req: Request) {
    const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET;

    if (!WEBHOOK_SECRET) {
        throw new Error("Please add WEBHOOK_SECRET from Clerk Dashboard to .env or .env.local");
    }

    const headerPayload = headers();
    const svix_id = headerPayload.get("svix-id");
    const svix_timestamp = headerPayload.get("svix-timestamp");
    const svix_signature = headerPayload.get("svix-signature");

    if (!svix_id || !svix_timestamp || !svix_signature) {
        return new Response("Error occurred -- no svix headers", { status: 400 });
    }

    const payload = await req.json();
    const body = JSON.stringify(payload);

    const wh = new Webhook(WEBHOOK_SECRET);

    let evt: WebhookEvent;
    try {
        evt = wh.verify(body, {
            "svix-id": svix_id,
            "svix-timestamp": svix_timestamp,
            "svix-signature": svix_signature,
        }) as WebhookEvent;
    } catch (err) {
        console.error("Error verifying webhook:", err);
        return new Response("Error occurred", { status: 400 });
    }

    const { id } = evt.data;
    const eventType = evt.type;

    if (eventType === "user.created") {
        const { id, email_addresses, first_name, last_name, image_url } = evt.data;

        const user = {
            clerkId: id,
            email: email_addresses[0].email_address,
            firstName: first_name,
            lastName: last_name,
            photo: image_url,
        };

        const newUser = await createUser(user);
        return new Response("New user created", { status: 200 });
    }

    return new Response("", { status: 200 });
}

With everything in place, I deployed the code and eagerly awaited success. Little did I know, I was about to run into a series of unforeseen issues.

The Struggle Begins: A Series of Missteps

504 Gateway Timeout

Almost immediately, I noticed something wrong in the Vercel logs. Every time the webhook was triggered, I got this ominous error message:

[POST] /api/webhooks/clerk status=504

At first, I thought it might be a connection issue, or maybe my database calls were taking too long. I optimized my MongoDB connections and tried again. Same result.

No problem, I thought. I’ve dealt with webhooks before. I’ll figure it out.

Verification Issues

Next, I ran into verification errors. I was sure I had copied the webhook secret correctly, but I kept seeing:

Error verifying webhook: [Error details...]

I spent hours combing through the headers, re-reading Clerk’s documentation, and tweaking my code. At this point, I was more confused than ever. Everything looked right, but nothing was working.

The Realization: Outdated Documentation

After days of debugging and headaches, I finally took a step back. Maybe I had missed something critical along the way? I went back to the Clerk documentation and noticed something small but crucial: an update to the webhook system.

In my haste to try out this new method, I didn’t realize that the version of Clerk I was working with had recently been updated, rendering my entire setup obsolete. The webhook flow I was following was no longer valid.

It was one of those moments where everything just clicked. I had wasted hours — days even(nah kidding) — trying to implement a system that was doomed from the start. If I had just stuck to what I knew, using Clerk’s API directly to persist users, I would have saved myself a lot of trouble.

The Lesson: Sometimes New Isn’t Better

Looking back, I could have avoided all this frustration by sticking with the tried-and-true methods that had worked for me before. But like many developers, I’m always drawn to the shiny new tools, the promise of automation, and the allure of simplifying my workflow. It’s not always wrong to try something new, but there are valuable lessons I learned:

1. Don’t Rush Into New Tools Without Checking for Updates

I should have double-checked the Clerk version I was using and reviewed any recent updates before diving into the new webhook system. Technology moves fast, and it’s easy to miss critical changes if you’re not careful.

2. Stick to What Works (Until You Fully Understand the New System)

While webhooks seemed like the more modern, efficient solution, my previous approach of using Clerk’s API directly was working fine. Next time, I’ll be more cautious about abandoning methods that already work for me.

3. Always Check the Logs and Documentation

In the end, the answers were right in front of me: the 504 errors were a result of Clerk’s outdated documentation, and the verification issues were because of version mismatches. Keeping up with changelogs and documentation is crucial when working with third-party services.

Conclusion:

Stick With What You Know — Until You Don’t

In the end, this experience reminded me of an important lesson in development: new doesn’t always mean better. It’s tempting to jump on the latest technology, but sometimes it’s worth sticking with what works, at least until you fully understand the new method or tool.

If you’re ever in a situation like mine, remember to:

• Check for updates to the tools you’re using.
• Don’t abandon working solutions too quickly.
• Take the time to understand new methods before diving headfirst into them.

Now, I’ll always be more cautious when trying out new things — and I hope this story helps you avoid the same pitfalls I fell into!