Kafka backup strategy: how to build one, by Anatoly Zelenin

Anatoly Zelenin, creator of one of the first open-source Kafka backup tools and co-author of Apache Kafka in Action, on how to build a Kafka backup strategy that actually works: understand the gap, set business-signed RPO/RTO, map failure scenarios, write the playbook, choose storage, test it and only then pick the tool.

Search for "Kafka backup", and one article still shows up near the top of the results, six years after it was published. It was written by Anatoly Zelenin, the engineer who built one of the first open-source Kafka backup tools back in 2019. We sat down with him to talk about why that gap in the Kafka ecosystem still hasn't closed and what it actually takes to build a backup strategy that holds up once you get past the marketing slide.

Anatoly founded the DataFlow Academy and enables organizations to understand how to work with Apache Kafka. He is also the co-author of Apache Kafka in Action. He no longer maintains his own Kafka backup project, but the problem he ran into building it is the same one most teams run into today: Kafka was never designed with backup in mind, and bolting it on afterward is harder than it looks. What follows is roughly the order he works through with teams that call him: understand the risk, set real targets, scope the failure modes, build the playbook around people, decide where the data lives, test it, and only then start comparing tools.

‍

Start understanding why Apache Kafka has a backup gap

Anatoly built his first backup tool because the organizations he worked with stored data in Kafka that didn't exist anywhere else. Losing it wasn't an option. At the time, nobody in the Kafka community was talking seriously about backups.

The reason is a split that has existed in the Kafka world from the start. One camp uses Kafka to move data fast, mostly telemetry and analytics, and doesn't mind if some of it disappears. The other camp uses Kafka as the backbone of event-driven systems, where the data itself matters and can't simply be re-pulled from somewhere else. Our interviewee estimates the second group is the minority, which is part of why backup tooling took this long to mature.

"Replication is not the same as backup. We've known that since RAID systems became available, sometime in the eighties. Crashes happen, people delete things, and then you need a backup."

Anatoly Zelenin

A replication factor of three protects you against losing a broker. It does nothing if someone deletes a topic accidentally, misconfigures retention.ms, or if a bug quietly corrupts data before it gets replicated everywhere. Anatoly has seen all three happen. This is the starting point of any backup strategy: naming the failure modes replication doesn't cover before deciding what to do about them.

‍

Set your Kafka backup RPO and RTO per workload, not per cluster

Ask Anatoly what the bare minimum looks like, and he doesn't start with tooling. He starts with understanding what your applications actually depend on, in what order they need to come back online, and how long each one can wait. A single RPO and RTO for the whole cluster rarely makes sense once you've seen how differently those workloads behave.

"RTO and RPO sound like technical numbers, but the question behind them isn't technical. If recovery takes three days, is that acceptable? I can't answer that, and you can't answer that. The business has to answer it. The default answer is always no. So you put a price on the alternative and let them decide."

Anatoly Zelenin

That price tag rarely comes from the license fee. It comes from getting the organization ready: defining priorities, writing the playbooks, running the drills, paying for the storage tier you actually need. Anatoly is blunt about where a backup tool fits into that picture.

"A Kannika license on its own won't solve your problem. It's the same as Agile. Buying Jira doesn't make you Agile."

Anatoly Zelenin

‍

Map your failure scenarios before you need them

Targets only mean something once you know what you're actually protecting against, and Anatoly is careful not to lump every disaster into one bucket. Losing the entire cluster is one scenario, and usually the simplest to plan for. Someone deleting a single topic is another, and it calls for a narrower, faster restore. A producer pushing bad data into a topic is a third case entirely, and that one usually needs point-in-time recovery rather than a full restore from the last backup.

Each of these has a different blast radius and a different fix. Treating them as one generic "Kafka is down" scenario is how teams end up improvising at the worst possible moment.

‍

Build the playbook, not just the restore script

A successful restore of the Kafka cluster itself is, in Anatoly's words, the easy part. The harder question is what happens next: which application starts first, which ones depend on each other, which can't run in parallel, and who decides whether the call center should brace for a flood of calls. None of that lives in a restore script. It has to be written down and owned by the teams running those applications, not just the platform team operating Kafka.

"If it's three in the morning, you don't want to think. You want to execute."

Anatoly Zelenin

‍

Decide where your backups actually live

Once a team accepts it needs backups, the next question is where those backups live. Same data center as the cluster, a second data center, another cloud region, another cloud provider entirely. Each option trades cost against risk, and jurisdiction adds a layer most teams forget to ask about until a compliance review forces the question.

Adding different storage tiers is common here too. The same pattern you can use on AWS with S3 moving into Glacier after a set period applies just as well to a self-managed setup: keep the last weeks of backups on warm storage for fast recovery and move older data to cold storage where retrieval is slower but cheaper. None of this has a universal right answer.

"There's no perfect solution. It's architecture. It's always trade-offs."

Anatoly Zelenin

‍

Test your Kafka backup strategy before you find out the hard way

A plan that has never been run is still just a plan. A few days before this conversation, Anatoly got an email from a customer whose data center had lost power. The backup generator didn't kick in either. The lesson he draws from stories like that is one most teams already know and few actually act on.

"Nobody cares about backups. Everyone cares about recovery. If you back up every day and never recover, how do you actually know your recovery works?"

Anatoly Zelenin

Banks that run two data centers often schedule an annual drill where they cut over one and watch what breaks. Anatoly argues Kafka backups deserve the same treatment and that the timing matters as much as the test itself. Netflix learned this the hard way when its streaming service went down on Christmas Eve, which is what led to the Chaos Monkey testing approach still associated with the company today.

"When do you want to find out your recovery doesn't work? On a Wednesday afternoon when everyone's at their desk, or on Christmas Eve?"

Anatoly Zelenin

‍

Only then choose your Kafka backup tool

Everything above happens before anyone opens a product page. Functionally, Anatoly's bar for a Kafka backup tool is short: back up the data, restore the data, and get the consumer offsets right. Everything past that is operational fit. Does it run on Kubernetes, bare VMs, or both? Does it expose metrics you can actually alert on? How long does a restore take when the cluster holds several terabytes?

"This is software you need, hopefully, never. But when you need it, it has to work. So the real question is whether you trust it. Is trust observability? Is that how the organization operates? Is it how fast you get a response when something breaks?"

Anatoly Zelenin

That last point is increasingly tied to regulation. Frameworks like DORA and NIS2 don't single out Kafka by name, but once a cluster carries business-critical data, it tends to fall under the same scope as everything else in the regulated estate. That means audits, evidence of tested recovery procedures, and vendor certifications that auditors recognize.

‍

Evaluate the vendor, not just the product

Anatoly's advice for picking a tool isn't a feature checklist. It's closer to due diligence.

"Even if a solution checks every box, even if it's Kannika and everyone around you is using it, you still need to test it in your own environment and see whether it fits."

Anatoly Zelenin

If a vendor advertises 24/7 support, he suggests finding out whether that's actually true before you need it.

"If you want to know whether 24/7 support is real, call them on a Saturday night and ask for help right then."

Anatoly Zelenin

It's a test that goes both ways. A vendor willing to stand behind that claim (and willing to have a customer actually check it) is telling you something a product page can't.

‍

The takeaway

Anatoly's own path through this problem started with kafka-backup, an open-source project he built because no one else had, and eventually stopped maintaining because the problem turned out to be bigger than the tool. What he kept from that experience is the order of operations: understand the gap replication leaves open, set targets the business has actually signed off on, map the failure scenarios before you need them, write the playbook for the people running applications on top of Kafka, decide where the backups live, and test all of it before something goes wrong in production.

The tool comes last. Not because it doesn't matter, but because a tool dropped into an organization that hasn't done the rest of this work won't save anyone at three in the morning. Anatoly now frequently points teams toward Kannika with that same sequence attached as a condition: strategy first, tooling second, and a real recovery test before you call either one done.