A comprehensive guide to preventing duplicate message processing across Kafka, Kafka Streams, RabbitMQ, SQS, and Redis — covering EOS internals, Kafka Streams State Store vs Redis architecture, idempotent consumers, and production deduplication patterns for senior engineers.
A complete guide to Kafka exactly-once semantics — delivery guarantees, idempotent producer, transactions, read_committed consumers, Kafka Streams EOS, zombie producer fencing, two-phase commit internals, and production patterns.
A comprehensive guide to Kafka Streams: from core concepts and internal architecture to stateful processing, failure recovery, exactly-once semantics, windowing, joins, interactive queries, and production system design patterns.
How to replace or offload the RocksDB state store with Redis in Kafka Streams / Kafka consumers, avoid full local state restore, and prevent lost or duplicated messages during batch processing by combining per-partition offset fencing, partition-key validation, epoch fencing and atomic Lua scripts.
