Asynchronous Messaging: RabbitMQ Introduction

RabbitMQ is considered today as a stable open source message broker implementation. It is considered by many as the natural evolution of JMS. What is brings to the table is the interoperability of disparate and heterogeneous parties. Indeed, a client in .Net for example can seamlessly exchange messages with a consumer on Java with minimal changes to any of these. It is worth noting that RabbitMQ is build with Erlang, the leader language in telecommunications systems within builtin support for fault tolerance.

RabbitMQ promotes the usage of AMQP (Advanced messaging queuing protocol) as the wire level protocol or network protocol for exchanging messages. It is a binary protocol that deals with the low level details of encoding and marshaling of message contents.

Architecturally, RabbitMQ provides the following advantages:

• Reliability: Aside from being built with Erlan, RabbitMQ can be configured to persist messages, so that in case of a server crash, all messages can be restored. Additionally, producers and consumers can acknowledge proper reception/delivery of messages.
• Customized routing: RabbitMQ supports different mechanisms for routing through the use of exchanges, it can for example provide point to point communication through direct routing, selective message delivery, similar to JMS message selectors, so that only events carrying a certain "routing key" get delivered to a queue.
• Built in support for clustering and high availability: Many instances of RabbitMQ can be logically grouped under a single cluster in order to provide redundancy and ultimately high availability in the case of crashes.
• Scripting and administration: RabbitMQ provides both a web based console for the purpose of monitoring and administration. In addition, it provides a command line interface to automate its administration through scripts.
• Versatility: there is a plethora of clients for different platforms/technologies. 

Terminology

Since a picture is worth a thousand words, let's start with a high level schematic from RabbitMQ Documentation

• Publisher: The party that is at the origin of the message to be sent.
• Consumer: The destination party that expresses its interest in one or more messages.
• Message Broker: The messaging solution in this case RabbitMQ. It is made of:
• Exchanges: The abstraction that describes an intermediate endpoint/stage on the message broker where all messages are delivered first.
• Queues: The intermediate endpoint where messages are sent from the exchange.
• Route: Provides a routing strategy to define how and when messages on the exchange should be relayed to the queue. This usually takes the form of a routing key, and follows a binding definition.

In the next installment, we'll have a look at the different types of exchanges and various patterns for exchanging information over RabbitMQ.

Microservices: Why Asynchronous communications?

The shift towards Microservices entails embracing and assessing different approaches to each and every aspect. Today, we'll dwell on asynchronous communication between microservices or between client and service at large.

Why asynchronous?

Or why stick to synchronous communications should we ask? It is true that synchronous communications has come to be regarded as the De facto pattern to exchange information between any two endpoints. Think of TCP, HTTP, FTP ... . It does indeed exhibit the following advantages:

1. Simpler reasoning and tracing: Everything starts when an outbound request in made from the client, and we can step through the whole process all the way until the request is processed by the server and a response is sent accordingly.
2. "Instantaneous" feedback:  When a client breaks the contract, or the server deems the current request as invalid, feedback is sent right away to the client. This can take the form of response codes, redirects, or event stack traces.
3. Natural: In the sense that we like to think about thing A happens before thing B, or thing C happens as a consequence of thing B.
4. Translates directly to models: We're all used to sequence diagrams on UML, and very few people can tell from the back of their head how asynchronous calls are modeled. Instead, we like to think of them as a succession of synchronous calls.

I assume the list might be longer, but it fails to address the following concerns:

1. How should a long running process on the server be presented to the client? If the server is experiencing a high peak in demand, or if the resources are scarce, chances are the client will be affected too. Should it just wait? Should it timeout? How should the client interpret this situation?
2. When an upstream dependency (server A calls server B) is unavailable, should the client care? Why should the client be sensitive to our ecosystem. From its standpoint, it reached the server, the rest is all internals.
3. Guaranteed response time: what if I told you that my server will respond in O(1) no matter what. How is that for usability?
4. What if the client does not need a response right away? Assume that I place my order in the context of an ecommerce application. I know what I ordered, I know how much I paid and I know when and where I will be delivered. Why should I hang in there until the server does all its housekeeping (updating product stocks, notifying the warehouse, ....)
5. Better yet, you don't know whom you're talking to but you're sure that your request will be honored. I always feel this way when I need an administrative paper. It seems that I always need to find the right person, and more importantly, at the right time. I always wished I could just submit my request to a "mailbox" and just head home and be sure that it will be fulfilled.
6. My request is more urgent that yours. Now you're in a nuclear power plant, and maintenance staff keeps track of the major events there by posting these events to the monitoring application, business as usual, cooling check... Suddenly, Mr Hero realizes that a major event that jeopardizes the safety of the whole neighborhood is about to take place, and posts an emergency stop payload, now I wouldn't like to be the one who processes a ventilation check while making the emergency stop wait.   

These are some of the pain points that asynchronous communications relieve us from. Long running tasks, unavailable servers, unresponsive servers, server transparency, request reordering and prioritizing... In the next post, we'll see how this can be done.