Complete architectural overview of Apache Kafka — origins, the M×N integration problem, the 4 pillars of Kafka's extreme speed (sequential I/O, page cache, zero-copy, batching), message anatomy, and the dumb broker / smart consumer paradigm.
Apache Kafka is a distributed event streaming platform designed for high-throughput, fault-tolerant, and scalable real-time data pipelines and streaming applications.
A comprehensive A–Z reference of terms used in banking and payments. Essential reading for anyone joining the industry.
A bank is made up of many specialised teams. Understanding who does what helps you collaborate effectively, know who to escalate to, and understand where you fit in the payments ecosystem.
These two messages form the **formal payment cancellation (recall) workflow**, allowing a payment to be recalled after it has been sent — even after settlement.
**Payment cards** (debit and credit) are one of the most widely used payment methods globally. They operate on **card schemes** — networks that define rules.
A conflict occurs when two branches have made **different changes to the same line(s)** of the same file, and Git cannot automatically determine which version.
A **consumer group** is a set of consumers that collectively consume a topic's partitions. Each partition is assigned to exactly one consumer within the group.
[Conventional Commits](https://www.conventionalcommits.org) is a lightweight convention for writing commit messages in a machine-readable, human-understandable.
The **Core Banking System (CBS)** is the central software platform that manages a bank's **primary banking operations** — account management, transaction.
Don't let your important business logic classes depend directly on concrete implementations (like a specific database driver, a specific email provider, etc.).
`git add` moves changes from your **working tree** into the **index** (also called the staging area). Think of the index as a draft of your next commit — you.
`git bisect` performs a **binary search through commit history** to efficiently find the exact commit that introduced a bug. Instead of checking commits one by.
A branch in Git is a **lightweight movable pointer** to a commit. Creating a branch costs nothing — it is just a 41-byte file containing a SHA. When you commit.
`git cherry-pick` copies one or more commits from anywhere in the repository and applies them to the current branch as new commits. The original commits remain.
`git commit` takes everything in the **index (staging area)** and creates a permanent, immutable snapshot in the repository. Each commit has:
Git configuration exists at three scopes, each overriding the one above it:
**The golden rule:** Prefer `git fetch` + manual inspect over `git pull` when you want to see what's changed before integrating. Use `git pull --rebase` for.
A **fixup** commit is a special type of squash commit that targets a specific earlier commit for amendment. When you run `git rebase --autosquash`, Git.
**Git Flow** is a branching model designed by Vincent Driessen for projects with scheduled, versioned releases. It defines a strict set of branch types and.
**Git hooks** are scripts that Git automatically executes before or after specific events (commit, push, merge, etc.). They live in `.git/hooks/` and can be.
**Git** is a distributed version control system (VCS) created by Linus Torvalds in 2005. Every developer has a full copy of the repository — including its.
`git log` shows the commit history of the current branch — or any branch, range, file, or author you specify.
`git merge` integrates the history of one branch into another. It finds the **common ancestor** of the two branches and combines their changes, creating a new.
`git push` uploads your local commits to a remote repository, making them available to other team members. It transfers only the objects (commits, trees.
git rebase moves or replays a sequence of commits onto a new base, rewriting history to produce a clean linear commit graph.
The **reflog** (reference log) is a local journal of every place `HEAD` and your branch pointers have pointed to, in chronological order. Every time you.
A **remote** is a named reference to another Git repository — typically hosted on GitHub, GitLab, Bitbucket, or an internal server. A remote stores a URL and a.
**Rule of thumb:** - Use `git reset` on **local, unpushed** changes - Use `git revert` on **pushed or shared** history
**Squashing** combines multiple commits into a single commit. This is used to clean up a messy feature branch before merging — turning a series of `wip`, `fix.
`git stash` temporarily shelves (stashes) your uncommitted changes — both staged and unstaged — so you can switch context without committing half-finished.
`git status` shows the state of your working tree and index relative to the current `HEAD` commit.
A **submodule** is a Git repository embedded inside another Git repository. The parent repository stores a reference to a specific commit of the submodule —.
A **tag** is an immutable pointer to a specific commit — unlike a branch, it never moves. Tags are used to mark release points (`v1.2.0`), milestones, or any.
`git worktree` lets you check out **multiple branches simultaneously**, each in its own directory, all sharing the same `.git` repository. No stashing, no.
Kafka uses a hash of the message key to determine partition assignment. Understanding this mechanism is essential for ordering guarantees, avoiding hot partitions, and designing correct partition keys.
Without idempotence, network retries create duplicate messages on Kafka brokers. Enabling idempotence guarantees exactly-once delivery per producer session.
Interest and fees are the primary ways banks **generate revenue** from accounts and products. Understanding how they work is important for product.
Keep your interfaces **small and focused**. Don't create a "fat" interface that bundles unrelated methods together, forcing classes to implement things they.
**Q1: What are the three layers required for end-to-end exactly-once in Kafka?**
**Q1: Explain Kafka's architecture in 2 minutes.**
**Q1: Walk me through what happens when a producer calls `send()`.**
Welcome! This guide will walk you through the **SOLID principles** — five essential design principles that help you write Java code that is **clean, scalable.
**Q: Explain your current project flow from API request to database. What part of the systems do you own completely? What was the last production bug you fixed.
Kafka Access Control Lists (ACLs) for fine-grained authorization. Covers KRaft ACL storage, resource patterns, OAuth/OPA/RBAC integration, and at-scale management.
Configure Kafka authentication with SASL/PLAIN, SCRAM-SHA-512, GSSAPI (Kerberos), mTLS, and OAuth 2.0. Understand how KRaft mode affects credential management.
**Kafka Connect** is a framework for **reliably moving data between Kafka and external systems** (databases, file systems, cloud services) without writing.
Lightweight per-record transformations in Kafka Connect. Covers built-in SMTs, conditional predicates, custom SMT development, and when to use SMTs vs stream processing frameworks.
A **consumer** reads messages from Kafka topics. Unlike traditional queues (push-based), Kafka consumers **pull** messages at their own pace. This gives.
The six primitives of Kafka data governance — schema policy, topic ownership, access control, encryption/masking, audit/lineage, and data quality. Why brokers don't provide governance and how to build it.
How Kafka log compaction preserves the latest value per key, enabling state stores, CDC changelog topics, and materialized views. Covers cleaner internals, tombstones, tiered storage, and KTable integration.
MirrorMaker 2 cross-cluster replication for disaster recovery, active-active, and hub-and-spoke. Covers architecture, offset translation, failover, Kubernetes deployment, and monitoring.
Deep dive into Kafka partitioning strategies — key-based, round-robin, sticky, custom partitioners, hot key mitigation, partition count sizing, and ordering trade-offs.
End-to-end Kafka performance tuning — producer batching, broker I/O, consumer throughput, JVM and OS tuning, compression selection, tiered storage, and benchmarking methodology.
A **producer** is a client application that publishes (writes) messages to Kafka topics. It is responsible for:
Combined guide to Kafka producers and consumers — internal architecture, delivery semantics, serialization, offset management, error handling, and production patterns.
End-to-end Kafka security guide covering authentication, ACLs, TLS 1.3 encryption, Zero Trust architecture, network isolation, credential management, and monitoring security events.
A topic is a named, durable stream of messages in Kafka — the logical category or feed where producers write and consumers read.
Comprehensive system design comparison of Apache Kafka vs RabbitMQ vs AWS SQS. Covers event streams vs message queues, Claim Check pattern, Web Crawler SQS vs Kafka trade-offs, and single-broker sizing heuristics.
If class `B` extends class `A`, then anywhere you use `A`, you should be able to swap in `B` without anything breaking.
Kafka guarantees **total ordering within a partition**. Messages written to the same partition are always consumed in the exact order they were produced.
Consumer lag is the most important consumer metric:
Your class should be: - **Open for extension** → You can add new behavior - **Closed for modification** → You don't change existing, working code
`pacs.004` is the **interbank payment return message**. It is sent by the **Creditor Bank back to the Debtor Bank** when a previously received `pacs.008`.
**No — `pain.004` is not a defined ISO 20022 message.**
Payment reversals allow **the sending side** (debtor bank or originating customer) to request that a previously submitted payment be reversed — cancelling the.
A partition is an ordered, immutable sequence of records within a topic — the fundamental unit of parallelism, replication, and storage scaling in Kafka.
If you're new to banking, the payment ecosystem can feel overwhelming. This page takes you from **zero to "I understand how a payment works"** using plain.
Kafka guarantees ordering within a partition, but single-threaded processing limits throughput. This guide covers four patterns for achieving high throughput while preserving per-key ordering.
The `acks` configuration controls how many broker acknowledgements the producer requires before considering a send successful, trading off throughput, latency, and durability.
Idempotence protects against duplicates within a session, but it doesn't help when:
A pull request (PR) is a unit of communication as much as it is a unit of code change. A good PR:
The replication factor defines how many copies of each partition exist across the cluster to guarantee fault tolerance and high availability.
Partitions are the unit of parallelism in Kafka. Scaling them is critical for throughput but can break ordering for keyed topics. This guide covers the mechanics, risks, and migration strategies.
Every class should do **exactly one thing** and do it well. If a class is handling multiple unrelated responsibilities, then it has multiple reasons to change.
Congratulations! You've learned all 5 SOLID principles. Here's everything at a glance.
Testing in banking is **high-stakes** — a defect in a payment system can result in customer funds lost, duplicate payments, regulatory breaches, or system.
**Trunk-Based Development (TBD)** is a branching strategy where all developers integrate their changes into a single shared branch (`main` / `trunk`) multiple.
A comprehensive guide to bootstrap.yml in Spring Boot — covering the bootstrap context lifecycle, configuration server integration, spring.config.import evolution in Spring Boot 2.4+, and senior deep dives on pitfalls.
