RabbitMQ is the most widely deployed open source message broker.
What is RabbitMQ?
- The most widely used message broker
- Open source
- Lightweight and easy to deploy
- Supports different messaging protocols
- Can be deployed to clusters to provide high availability and scalability, necessary in enterprise solutions
- Used by many companies on a large scale including Google, VMWare and Mozilla
Main Features
- Variable level of reliability, generally configuring for increased reliability will reduce the performance so this can be managed as required.
- Complex routing capabilities
- Different configurations to group together brokers for different purposes eg. Clusters, Federation, Shovel models
- Highly available message queues
- Support for multiple protocols eg amqp , mqtt
- Client available in a large number of languages including C#, Java, Go, Erlang etc
Installation on a Mac
Click to install on MAC then to enable the RabbitMQ management plugin run this command rabbitmq-plugins enable rabbitmq_management If RabbitMQ is running and the management plugin is enabled, accessing http://localhost:15672 should load the management UI. The default credentials for the management console are : Username: guest Password: guest Troubleshooting
- Make sure that no other service was already running on port 15672 of your local machine
- Check that RabbitMQ service is started
- Installation may require a restart
RabbitMQ UI
- Connections: is a TCP connection between an application and the rabbitMQ broker.
- Channels: is a virtual connection inside the TCP connection, publishing/consuming is done over a connection rather than a connection and each TCP connection can hold multiple channels.
- Nodes let us see the health of our cluster
- Admin TAB
- We can create a user then also virtual host. A virtual host is simply a way to segregate applications within the same rabbitmq instance so different users can have different right virtual host and queues and exchanges can be configured when created so that they exist only in one particular virtual host
Introduction to Messages, Queues and Exchanges
Messages
- A message is a binary blob of data that is handled by RabbitMQ and can be distributed to queues and exchanges Throughout this material
- we will use the term Producer to refer to a generic program producing messages and sending them to a RabbbitMQ queue/exchange
- and the term Consumer to refer to any generic program receiving messages from RabbitMQ
Queues
- A queue is where messages flowing through RabbitMQ can be stored functioning similarly to a post box
- Can also be seen as a large message buffer
- A queue’s message storing limit is only bound by the host’s memory and disk limits
Exchanges
- An exchange receives messages from producers and pushes them to queues
- An exchange can be set to forward messages to zero or more queues depending on the routing parameters and exchange configurations
- If we relate this to the post office analogy, where the queues are the post boxes then the exchange is the postmen that deliver messages to the post boxes
Messages
Message Acknowledgements
which is a key mechanism in guaranteeing reliable message transfer in RabbitMQ
- If a connection fails between a RabbitMQ server and a client (producer or consumer), messages in transit may not have all been processed correctly and need to be re-sent.
- To detect such instances, message acknowledgements can be used. If the sender does not receive a positive acknowledgement (ack) before the connection fails it will re-queue the message
- It is, therefore, good practice to acknowledge a message after any required operations on the message are performed.
- There are different configurations of message acknowledgements, by enabling automatic acknowledgement mode the message will be considered acknowledged as soon as it is sent – acting in a fire and forget mode.
- This will reduce the safety check that a message has been received successfully but allows for increased throughput.
- Consumers can also send a negative acknowledgement for a message and instruct the message broker to re-queue them.
- Both positive and negative message acknowledgements can be sent in bulk by setting the multiple flags of the acknowledgement command to true.
- Protocol methods for acknowledgements: * basic.ack is used for positive acknowledgements * basic.nack is used for negative acknowledgements * basic.reject is also used for negative acknowledgements but is only capable rejecting one message at a time NOTE: the exact command name may vary slightly between client libraries of different programming languages
Message Ordering
- By default, message ordering in queues in First In First Out (FIFO)
- However, queues can be configured to act as priority queues, in which case messages will be ordered depending on their priority which is set by the sender
Durability
- Durable queues are persisted to disk and thus survive broker restarts. Queues that are not durable are called transient.
- Setting a queue to durable does not make messages that are routed to that queue durable. If the message broker is restarted, a durable queue will be re-declared during broker startup, however, only persistent messages will be recovered.
- persistent , in which case it will be persisted to disk as soon it is received by a durable queue.
- In some cases, non-persistent messages are also written to disk when there is a shortage of memory. However, this will not make them durable
Queues
Temporary Queues
- Queues can be configured to be deleted automatically in 3 ways :
- Exclusive queues can only be used by their declaring connection and will be automatically deleted once this connection is closed.
- An expiry time (also known as a time to live) can be set for the queue. If the queue is left unused for a duration exceeding this period, the broker will automatically delete the queue.
- Auto-delete queues will be automatically deleted once their last consumer has cancelled through the basic.cancel protocol or gone (e.g closed connection)
Exchange
- An exchange receives messages from a producer (sender) and pushes them to queues or rather exchanges.
- An exchange can be set to forward messages to zero or more queues depending on the routing parameters and exchange configurations
- We will now go through each of the following exchanges types and learn about each of their routing capabilities
- Types of exchanges * Fanout * Direct * Topic * Headers
Fanout Exchange
* This is the most simple kind of exchange.
* A Fanout Exchange routes a message to all queues bound to it
* Ignores any routing key provided with the message
You can click the image to enlarge
Direct Exchange
* A direct exchange routes a message with a particular routing key to queues bound to that exchange with that exact routing key.
You can click the image to enlarge
Topic Exchange
- A Topic exchange routes a message with a particular routing key to queues whose routing key matches all or a portion of a routing key
- Messages are published with routing keys containing one or more words separated by a dot eg multi.word.test
- Queues that bind to a topic exchange supply a matching routing key pattern for the server to use when routing the message. These patterns may contain an asterisk (*) to match any word in a specific position of the routing key, or a hash (#) to match zero or more words.
- Examples: a message published with routing key multi.word.test
- will match queues with routing key multi.#, *.word*, multi.word.test, #
- but will not match queues with routing key multi.* , single.# or multi.foo.test
- The # can replace a single or more word while the * will replace a single word which matches the pattern
You can click the image to enlarge
Headers Exchange
- A Headers exchange routes messages based upon a matching of the message’s headers to the expected headers specified by the binding queue.
- It is important to note the difference between the headers exchanges and the topic exchange type :
* The topic exchange type matches on the routing key
* The header exchange matches on the message header
- More than one header criteria can be specified as a filter by the binding queue, in which case the binding queue can specify if the message headers need to contain ‘any’ or ‘all’ of the header criteria
- Message header can be matched in any order
You can click the image to enlarge
ADEYINKA OLUWASEUN DOYIN 