Message Queue
A message queue is an asynchronous communication mechanism where a producer application sends messages to a buffer (the queue), and a consumer application retrieves and processes them independently, enabling decoupled communication between system components without requiring both parties to be available simultaneously. Message queues implement the producer-consumer pattern and support key enterprise integration patterns such as load leveling (absorbing traffic spikes), fan-out (broadcasting to multiple consumers), and retry logic (requeuing failed messages). Technologies like RabbitMQ, Apache Kafka, and AWS SQS are foundational to event-driven microservices architectures, enabling reliable asynchronous workflows such as order processing, email delivery, and data pipeline ingestion.
Message Queue Technologies Comparison
| Technology | Model | Throughput | Best For |
|---|---|---|---|
| RabbitMQ | Traditional queue (AMQP) | Moderate | Task queues, RPC patterns |
| Apache Kafka | Distributed log | Very high (millions/sec) | Event streaming, data pipelines |
| AWS SQS | Managed cloud queue | High | Serverless AWS architectures |
| Redis Streams | In-memory log | High | Real-time leaderboards, notifications |
| Azure Service Bus | Enterprise messaging | High | Microsoft Azure workloads |
Interactive Tools
Wikimedia Commons, CC BY-SA
Related Terms
Microservices Architecture
Microservices architecture is a software design approach where an application is structured as a collection of small, independently deployable services, each responsible for a specific business capability and communicating over well-defined APIs. Unlike monolithic architectures where all components share a single codebase and runtime, microservices can be developed, deployed, and scaled independently using different programming languages and data stores. This pattern enables large engineering teams to work in parallel, improves fault isolation, and supports continuous delivery pipelines at scale.
Distributed Systems
A distributed system is a collection of independent computers that appear to users as a single coherent system, coordinating their actions by passing messages over a network to achieve a common computational goal. Key challenges in distributed systems include handling partial failures (where some nodes crash but others continue), achieving consensus on shared state, ensuring data consistency across replicas, and tolerating network partitions. Modern distributed systems underpin the internet's most critical infrastructure — from Google's search index and Amazon's shopping cart to financial clearing systems and distributed databases.
REST API
REST (Representational State Transfer) API is an architectural style for designing networked applications that uses HTTP requests to perform CRUD operations on resources identified by URLs. It enforces stateless communication, meaning each request from a client must contain all information needed to process it, with no session state stored on the server. REST APIs are the backbone of modern web services, enabling interoperability between frontend clients, mobile apps, and backend services.
The term "message queue" traces back to early OS inter-process communication (IPC) mechanisms in the 1960s–70s. "Queue" derives from French/Latin "cauda" (tail), referring to the FIFO (first-in, first-out) ordering. Enterprise message queuing was formalized with IBM MQ (then MQSeries) in 1993.