RabbitMQ Fundamentals

RabbitMQ is an open source, multi-protocol message queuing and broker system that supports the pub-sub architecture. RabbitMQ supports:

- AMQP 0-9 – 1.0
- AMQP 1.0
- STOMP 1.0 – 1.2
- MQTT 3.1.1

RabbitMQ has a wide variety of libraries available in different languages like Python, Java, Ruby, PHP, C#, JavaScript, Go, Erlang, Objective-C, Swift and many more to write publishers and producers applications.

RabbitMQ Ecosystem:

In a nutshell, RabbitMQ consists of a broker. A broker is responsible for receiving messages from producers or publishers and routing them to consumers or subscribers.

A publisher or producer is an application that sends messages toward a broker. Each message has a payload and optionally a header which contains some metadata about the message itself. Some of the header data may be used by brokers for routing purposes and some use by end user application. Note that the message payload itself is fully opaque from the brokers’ point of view and is used only by the consumer which receives the message after routing.

A consumer is an application that receives a message from the broker. It is important to note that, producers, brokers, and consumers can reside on different physical or virtual hosts since RabbitMQ works based on a network protocol.

Broker consists of exchanges and queues:

In RabbitMQ, producers send their messages to an exchange which is responsible for processing and routing the messages to different queues. An exchange will route the messages to zero, one, or more than one queues based on bindings and routing keys.

A binding is a logical link which defines the rule of routing between an exchange and a queue. Application developers or admins are responsible for declaring a binding. I will explain how to declare a binding later, but first, let’s get a better understanding of the core concepts and fundamentals of RabbitMQ.

There are multiple types of exchanges in RabbitMQ:

Fanout Exchange
Direct Exchange
Topic Exchange
Headers Exchange

Fanout Exchange:

A fanout exchange simply broadcasts messages to all the queues that are bound to it. Regardless of the information present in the header, a fanout exchange pushes messages to all the queues that are bound to it and ignore the routing key of the message if present in the header of the message.

Direct Exchange:

A direct exchange pushes messages to queues using a string identifier called “Routing Key” which is present in the header of the message. Basically, direct exchanges will look at the routing key of the message and based on that, decides which queue each message belongs to and delivers it. A routing key is an attribute in the header of the message which a publisher needs to insert into the header of the message. Routing key acts as an address for an exchange, therefore, a message goes to the queues which have exact routing key of the message in their binding keys. It means that a direct exchange delivers messages to the queues that have exactly the same binding keys as the routing key of that message. A predefined queue with a binding key is needed before the arrival of the message. The exchange can route the message to that particular queue using that predefined binding.

Topic Exchange:

A topic exchange is very similar to direct exchange except it is more flexible. A topic exchange has the notion of wildcard ‘*’ which makes it way more programmable for routing messages.

A topic exchange performs a wildcard match between the routing key in the header and the routing pattern specified in the binding as opposed to a direct exchange which performs an exact match between the routing key in the header and routing key in binding.

Routing keys in topic exchanges are delimited by a dot ‘.’ , for example, USA.CALIFORNIA.SANTA-CRUZ.TEMPERATURE is a routing key. A binding key can also is delimited by a dot ‘.’ and maybe consist of wildcard ‘*’. For example, USA.CALIFORNIA.*.TEMPERATURE which represents a binding to a queue or any number of queues interested in the temperature of all the cities in California. Another binding example could be USA.*.*.TEMPERATURE which represents a binding to queues interested in the temperature of all the cities in the US. In the above example, if a publisher sends a message with routing the key USA.TEXAS.AUSTIN.TEMPERATURE will be routed to the queue interested in the temperature of all the cities in the US since it matches the binding key consist of the wildcard.

Headers Exchange:

A headers exchange routes the messages based on metadata present in the header of the message. Header exchange ignores the routing key if present in the header and look at other attributes in the header to decide the routing of the message. A headers exchange evaluate the header attributes with the value specified on the bindings. A message will be considered as a match if the value of the header is the same as the value of the binding.

At the time of binding, an admin can bind the queue to the exchange using one or more header. If the binding consist of multiple headers, then the admin needs to specify the “x-match” argument as well. X-match simply dictates an exchange to consider only one or all the headers attributes for the routing. A value of x-match can be either “any” or “all”. If x-match is set to “any”, it means that if only one of the headers is match with any of the headers used in binding attributes, the message will be considered as a match and will be routed to the queue used that binding. If the x-match is set to “all”, then all the headers in binding must be matched with headers in the message. As opposed to a routing key which needs a string value, the value of other headers may be an integer or even a dictionary.

Bindings:

Let’s take a closer look at bindings and how to define them. A binding basically contains the instruction for exchanges to know where to route the messages to. Note that there are multiple ways to define a binding, for example using RabbitMQ web-based user interface, or CLI (command line interface) or in the producer application itself. Here I show you how to declare a binding using “rabbitmqadmin” which is a command line tool for the management and administration of rabbitMQ.

This example, declare a binding for an exchange named “exchange-1” to a queue named “CA-queue” with a routing key “CALIFORNIA”:

sudo rabbitmqadmin declare binding \
    source="exchange-1" \
    destination_type="queue" \
    destination="CA-queue" \
    routing_key="CALIFORNIA"

As you remember, some of the exchanges use routing key attribute and some just ignore it. The above example is a binding for a direct exchange which sends all the messages with routing key “CALIFORNIA” to “CA-queue”.

On my next post, I will cover more advanced topics in rabbitMQ including, virtual hosts (vhost), authentication and authorization in rabbitMQ. Please give me your feedback on this post, I would love to hear your feedback.