Decision lies on architect on where the intelligence of system should live: in infrastructure or application logic.
Simple difference - in Kafka, messages live but in RabbitMQ, messages flow through.
| Feature / Aspect | RabbitMQ (The Smart Broker) | Kafka (Distributed Log) |
|---|---|---|
| Broker Responsibility | Routing logic, delivery tracking, retries/DLQs | Sequential storage, serving data by offset |
| Consumer Responsibility | Simple: process and acknowledge | Smart: tracks position (offsets) and state |
| Message Lifecycle | Transient: Deleted on acknowledgement | Durable: retained based on retention policies |
Message Retention and Routing
RabbitMQ — Producer → Broker → Routing Rules → Queue → Consumer. Broker is smart, consumer is dumb. Messages deleted on acknowledgement. Built-in DLQs handle failed deliveries automatically.
Kafka — Producer → Topic → Partition → Consumer. Broker is dumb (just stores), consumer is smart (tracks offsets). Messages retained by policy, enabling replayability — a new service can rewind and backfill from day one.
Ordering and Parallelism
RabbitMQ — global ordering within a queue, but only with a single consumer. Add more consumers → out-of-order.
Kafka — order guaranteed within a partition. Parallelism is limited by partition count, but it's predictable.
Performance
| RabbitMQ | Kafka | |
|---|---|---|
| Model | Push | Pull |
| Latency | Low | Higher |
| Throughput | 4K–10K msg/s | 100K+ msg/s |
| Per-message overhead | High (tracks each) | Low (minimal tracking) |
Delivery Semantics
- At Least Once — both support it; duplicates possible on failure.
- At Most Once — Kafka can drop messages; use when loss is acceptable.
- Exactly Once — Kafka only, but only inside a Kafka→Kafka loop. Cross system boundaries → you're back to at least once. Design idempotent consumers regardless.
Ops Complexity
RabbitMQ — easier to set up, gentler learning curve, lower overhead.
Kafka — steeper curve, historically needed Zookeeper (KRaft mode removes this in newer versions).
Who Uses What
| Company | Tool | Use case |
|---|---|---|
| RabbitMQ | Photo resize / filter jobs | |
| RabbitMQ | Comment threads, karma scoring | |
| Netflix | Kafka | Billing and recommendations |
| Uber | Kafka | Real-time pricing, fraud detection |
| Kafka | Feeds and messaging (original creator) |
When to Use Which
RabbitMQ — discrete tasks (emails, payments), smart routing, sub-5ms latency at moderate scale.
Kafka — multiple independent consumers on the same stream, replayability, millions of events per second.