29 docs tagged with "distributed-systems"

An exhaustive architectural study guide synthesizing 60 core system design topics across 7 architectural domains with 60 high-impact scenario breakdowns.

Deep mathematical and architectural guide to distributed consensus, proving the 2f+1 vs 3f+1 quorum thresholds, PBFT three-phase commit, HotStuff linear pipelining, and Byzantine attack defenses.

Core principles of distributed systems architecture including CAP theorem, consistency models, availability, partition tolerance, and key trade-offs every engineer must understand.

Production runbook for blockchain backends - deposit detection with reorg safety, nonce management, transaction replacement, idempotency, key custody with HSM/MPC, RPC resilience, gas estimation, observability, and incident response.

Full-depth guide to CAP theorem and PACELC — the GitHub 2018 incident, why 'choose 2 of 3' is misleading, PACELC's EL trade-off (latency vs consistency during normal operation), database classification matrix, conflict resolution costs, and Brewer's 12-year correction.

Masterclass guide to CockroachDB internals — 5-layer architecture (SQL, Transaction, Distribution, Replication, Storage), Hybrid Logical Clocks (HLC), Multi-Raft consensus, Leaseholders, Write Intents, Parallel Commits, Read Uncertainty, and Pebble storage engine.

Distributed consensus for blockchains - FLP impossibility, Nakamoto consensus, Proof of Stake with Casper FFG, PBFT/Tendermint, Raft vs BFT, fork choice, 51% attacks, long-range attacks, and trade-offs.

A comprehensive deep dive into consistent hashing, addressing modulo scaling bottlenecks, hash rings, virtual nodes, data replication, interview questions, and real-world implementations.

Deep dive into real-time collaborative architectures — State-based vs Operation-based CRDTs, the Join-Semilattice mathematical proof, Sequence CRDTs (RGA, Yjs, Automerge), Operational Transformation (OT in Google Docs), and tombstone garbage collection.

Comprehensive senior-principal architectural comparison of Data-Driven Architecture (DDA) and Event-Driven Architecture (EDA). Explores state vs stream paradigms, ACID vs eventual consistency, temporal coupling, polling tax, dual-write prevention with Transactional Outbox and CDC, and a battle-tested decision framework for production systems.

Senior Principal Engineer guide to database re-sharding, live cluster migration approaches, pros and cons, dual-write choreography, CDC replication, consistent hash expansion, and range splits.

A complete guide to horizontal scaling — sharding strategies, consistent hashing, cross-shard complexities, rebalancing, distributed ID generation, and real-world database comparisons.

Core distributed systems concepts including consensus algorithms, leader election, vector clocks, distributed transactions, fault tolerance, and the fallacies of distributed computing.

Master guide to distributed transactions in microservices — Two-Phase Commit (2PC), Saga Pattern (Choreography vs. Orchestration), Transactional Outbox + CDC, and NewSQL architectures.

A complete guide to Kafka brokers — what they are, how storage works, partition leadership, replication, ISR, KRaft vs ZooKeeper, log compaction, performance internals, and production monitoring. Beginner through senior depth.

A complete guide to Kafka exactly-once semantics — delivery guarantees, idempotent producer, transactions, read_committed consumers, Kafka Streams EOS, zombie producer fencing, two-phase commit internals, and production patterns.

15 rigorous senior and principal staff engineer interview questions on Kafka Streams, covering Stream-Time traps, RocksDB cgroup OOMs, Cooperative Sticky Assignor, EOS V2 transaction internals, sub-topology reorder storms, and The Four Golden Rules.

Comprehensive guide comparing Apache Kafka's legacy ZooKeeper architecture with the modern KRaft (Kafka Raft) metadata mode — covering internal mechanics, failure scenarios, Strimzi Kubernetes deployment, migration strategies, and production deep dives for senior engineers.

Principal engineering guide to payment processing resilience — multi-tier deduplication, request fingerprinting, retriable error classification, the transactional outbox pattern, reverse inquiry polling, and 3-way reconciliation engines.

Deep-dive into Kafka Connect rebalance storm internals — why they happen, the two rebalance protocols, incremental cooperative mechanics, task assignment algorithms, Spring Boot monitoring, and a production-grade rolling restart runbook.

A comprehensive guide to the Raft Consensus Algorithm — covering leader election, log replication, safety guarantees, and how it is implemented in Apache Kafka's KRaft metadata mode.

Senior-level Redis interview questions covering data structures, persistence, clustering, cache patterns, distributed systems, and production failure scenarios.

Master senior-level resilience patterns for mission-critical distributed systems — Idempotency engines, Two-Phase Reservation (TCC) with timeout reapers, Virtual Waiting Rooms, Adaptive Concurrency Limits, Bulkheading, and Fenced Distributed Sagas.

A comprehensive guide to the Saga Pattern — from core concepts for beginners to choreography vs orchestration internals, state machine design, idempotency, compensating transactions, temporal coupling, failure taxonomy, and production observability for senior engineers.

Deep-dive into high write throughput techniques — sharding, partitioning, WAL internals, LSM trees, async pipelines, batching, backpressure, idempotency, and distributed transactions — with production Java/Spring code and failure mode analysis.

Senior Principal deep dive into split-brain syndrome, dual-primary scenarios, multi-leader data divergence, fencing tokens, STONITH, and conflict reconciliation (LWW vs CRDTs vs Version Vectors).

Deep dive into distributed time synchronisation, physical clock drift, TrueTime, Lamport timestamps, Vector Clocks, Hybrid Logical Clocks (HLC), and distributed ID schemes (UUIDv7 RFC 9562 vs Snowflake vs UUIDv4) with B-Tree storage implications.

A complete guide to the Transactional Outbox Pattern — from the Dual-Write problem for beginners to CDC vs polling internals, at-least-once guarantees, ordering semantics, and production monitoring for senior engineers.

A comprehensive guide to Two-Phase Commit (2PC) and Three-Phase Commit (3PC) — from real-world analogies for beginners to WAL internals, failure modes, and XA implementation for senior engineers.