Backup and restore a Kafka topic: a step-by-step guide

TL;DR: Restoring a Kafka topic means more than replaying messages. You need the data, the offsets, the schema, and often the consumer group positions to land back in a state your applications can actually resume from. This guide walks through backing up and restoring a single topic step by step, and where most manual approaches fall apart.

‍

Why is restoring a Kafka topic harder than it looks?

A Kafka topic looks like a simple thing to restore. It is a log of messages, so surely you just write the messages back and you are done.

‍

In practice, a topic carries more state than its payload. Every message has an offset that consumers rely on to track their position. If the topic uses Avro or Protobuf, messages reference schema IDs from a schema registry, and those IDs are cluster-specific. Consumer groups store their last committed offset in the __consumer_offsets topic, which has no direct relationship to the data you are restoring. Partition assignment, compaction settings, and retention policies all shape how the data was originally laid out.

‍

Replay the messages without accounting for any of that, and you get a topic that looks fine in a UI but breaks every consumer pointed at it. Deserialization fails because schema IDs no longer resolve. Consumers reprocess millions of records because their offsets no longer map to anything, or they skip data entirely because the new log starts at offset 0. This is also why replication is not a substitute for a real backup. Partition replicas and MirrorMaker copy your data (and your mistakes) in real time, but neither gives you an independent, point-in-time copy you can restore from after a bad deploy or an accidental deletion. If that distinction is new to you, replication is not a backup for Kafka covers why.

‍

This article focuses on the how: backing up and restoring a single Kafka topic with its offsets and schema intact. For the broader picture of what to back up and why, see the Kafka backup pillar guide.

‍

What do you actually need for a complete Kafka topic backup?

A complete topic backup covers four things:

• The message data itself: Keys, values, headers, partition, and offset, for every record in the topic within your retention or compliance window.

• The schema: If you use Confluent Schema Registry or a compatible registry, the schemas referenced by the topic's messages need to be captured alongside the data. Restoring data to a cluster with a different (or empty) schema registry breaks deserialization the moment schema IDs stop matching.

• Consumer offsets: The __consumer_offsets topic tells you where each consumer group was positioned. Without it, or without a strategy for approximating positions, every consumer connecting after a restore either reprocesses everything from the earliest offset or skips ahead to the latest, silently dropping data.

• Topic configuration: Partition count, replication factor, compaction and retention settings. Restoring to a topic with a different partition count changes key-to-partition mapping, which breaks ordering guarantees for anything that depends on partition affinity.

Miss any of these four and the restore "works" in the sense that data lands somewhere, but it does not put your applications back in a state they can resume from cleanly.

‍

How do you back up a Kafka topic step by step?

Using Kafka Connect

There is no built-in kafka-topic-backup command. Teams typically piece this together from a few building blocks.

‍

1. Capture the message data. A Kafka Connect sink connector (S3, GCS, or Azure Blob Storage) is the most common DIY approach. It reads from the topic like any other consumer and writes batches to object storage.

# kafka-connector-s3-sink.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: topic-backup-sink
  labels:
    strimzi.io/cluster: my-connect-cluster
spec:
  class: io.confluent.connect.s3.S3SinkConnector
  tasksMax: 2
  config:
    topics: orders
    s3.bucket.name: kafka-backups
    s3.region: eu-west-1
    flush.size: 10000
    storage.class: io.confluent.connect.s3.storage.S3Storage
    format.class: io.confluent.connect.s3.format.avro.AvroFormat
    partitioner.class: io.confluent.connect.storage.partitioner.TimeBasedPartitioner
    path.format: "'year'=YYYY/'month'=MM/'day'=dd"

This gets the payloads into cold storage. It does not, on its own, capture the schema registry state or the consumer offsets tied to that data. You would need a second job to snapshot the schema registry, and a third mechanism to track offsets, usually a separate consumer subscribed to __consumer_offsets, filtered and decoded manually since it uses an internal binary format.

‍

2. Back up the schema registry. Export the schemas referenced by the topic, keyed by subject and version, so they can be reapplied or mapped to new IDs on restore.

‍

3. Snapshot consumer offsets. Either back up __consumer_offsets directly, or record (group, topic, partition, offset) tuples for the groups you care about using kafka-consumer-groups.sh --describe on a schedule.

kafka-consumer-groups.sh --bootstrap-server broker:9092 \
  --group orders-processor --describe

Using Kannika Armory

If you are running this on Kannika Armory instead of assembling it from Connect and cron jobs, the same outcome takes one resource. A Backup continuously streams a topic's data to storage, tracking each record's original offset internally so later restores can target an exact offset or point in time.

apiVersion: kannika.io/v1alpha
kind: Backup
metadata:
  name: orders-backup
spec:
  source: "my-kafka-cluster"
  sink: "my-storage"
  streams:
    - topic: "orders"

Schemas typically live on a topic too, usually _schemas, so in most setups backing them up is just a matter of adding that topic to the same Backup. If your schema registry runs on Confluent Cloud, where the underlying topic isn't directly reachable, Kannika uses dedicated SchemaRegistryBackup and SchemaRegistryRestore resources that talk to the Schema Registry API instead.

‍

How do you restore a Kafka topic to the right point in time?

Using Kafka Connect

Restoring is where most of the manual work shows up, because you are reversing four separate processes at once instead of one.

‍

Recreate the topic with matching configuration. Same partition count, same replication factor, same compaction settings as the source. If you change partition count, keys will hash to different partitions than before and any ordering guarantees your consumers depend on are gone.

‍

Replay the data through Kafka Connect (or a custom producer). A source connector reading from S3, or a script reading your backup files and producing them back onto the topic, in the original order.

kafka-console-producer.sh --bootstrap-server broker:9092 \
  --topic orders < orders-backup-2026-06-01.jsonl

This restores payloads but assigns new offsets starting wherever the topic currently sits. The original offsets are gone unless you preserved them separately, typically as a header on each restored message, which your consumers then need custom logic to read and act on.

‍

Remap or reload the schema. If you are restoring into the same schema registry, this is a non-issue. If you are restoring into a different one (a new environment, a new cluster, a DR site), schema IDs will not match and every message will fail to deserialize until you build a mapping between old and new IDs.

‍

Reposition consumer groups. This is the step teams most often skip, and the one that causes the most confusion after a restore. Depending on how much offset data you preserved, you are choosing between exact offset restoration, approximating positions from timestamps, or accepting that consumers will reprocess or skip data. We cover the tradeoffs between these approaches in detail in how to handle Kafka consumer group offsets, so we will not repeat it here beyond flagging that it is a required step, not an optional one.

‍

Using Kannika Armory

On Kannika Armory, restoring a topic with its point-in-time state is a Restore resource rather than a sequence of scripts:

apiVersion: kannika.io/v1alpha
kind: Restore
metadata:
  name: orders-restore
spec:
  source: "my-storage"
  sink: "my-kafka-cluster"
  sinkCredentialsFrom:
    credentialsRef:
      name: my-credentials
  enabled: true
  config:
    topics:
      - source: "orders"
        target: "orders"
    filters:
      time:
        absolute:
          toDateTime: "2026-06-01T09:00:00Z"
    legacyOffsetHeader: "__original_offset"

The filters.time.absolute.toDateTime field restores the topic up to that exact timestamp, not just "whatever is in storage," and fromDateTime can be added alongside it to restore a specific window instead of everything up to a cutoff. Schema mapping between source and target registries is handled by generating a mapping file (Kannika's SAME tool does this by fingerprinting schemas on both sides) and referencing it from the Restore via schemaMappingFrom. Setting legacyOffsetHeader writes each message's original offset into a header on the restored message, which Kannika's companion CLI tool, kbridge, then reads to calculate and apply the matching consumer group offsets on the target topic, so consumers resume from the right position instead of the beginning or the end of the topic.

‍

How Kannika solves Kafka topic backup and restore

Kannika Armory backs up topic data and, where needed, schema registry state continuously, as one coordinated process rather than separate pipelines you have to keep in sync yourself. Restores can tag each message with its original offset via a header, and the companion kbridge CLI reads that header to calculate and apply the matching consumer group offsets on the target cluster, so consumers resume exactly where they left off instead of reprocessing everything or skipping ahead.

‍

Restores support point-in-time recovery down to the timestamp, selective topic and partition restoration, and schema ID mapping between registries when you are restoring into a different environment. What normally takes a Connect sink, a cron job against __consumer_offsets, a manual schema export, and a runbook for the day something breaks becomes two Kubernetes resources, a Backup and a Restore, plus kbridge for the offset handoff.

‍

Kannika vs. Kafka Connect sink vs. a custom script for Kafka topic backup

‍

What goes wrong most often during Kafka topic recovery?

• Assuming replication is a backup: Replicas and MirrorMaker faithfully copy corruption and accidental deletions along with everything else. They protect against hardware failure, not human error.

• Forgetting the schema registry: Data restores cleanly, then every consumer throws deserialization errors because schema IDs point at subjects that do not exist in the new registry.

• Losing offsets during migration: Consumers either reprocess the entire topic from the start or connect at the high watermark and silently miss everything that happened before the restore.

• Restoring with a different partition count: Key-based ordering guarantees depend on consistent partition counts. Change it during a restore and messages that used to land on the same partition, in order, no longer do.

• Testing the backup, not the restore: A backup job running successfully for months tells you nothing about whether the restore path works. The only way to know is to run a restore, on a real topic, before you need to.

‍

Restoring a Kafka topic the right way

Restoring a Kafka topic correctly means treating offsets, schema, and consumer state as part of the backup, not an afterthought.

‍

See how Kannika Armory backs up and restores Kafka topics, or start a free trial to test a restore on your own cluster.