Consumer Group

Solving the scalability challenge is very important for Arminius, and a core part of how we tackled it is by implementing Consumer Groups over NATS. This document outlines the insides of this central piece of code.

The following example describes the process triggered by the Client Component when one of its connected clients wants to send a message to a room.

graph LR;
    ID1["(1) Client Component"] -- "rooms.{room_id}.{command}" --> ID2["(2) NATS"];
    ID2 -- "rooms.{partition}.{room_id}.{command}" --> ID3["(3) Consumer Client 1"];
    ID2 -- "rooms.{partition}.{room_id}.{command}" --> ID4["(3) Consumer Client 2"];

1. The client component makes a request over NATS with the subject rooms.{room_id}.{command}.
2. Upon reception, the NATS server uses subject mapping (and especially here Deterministic subject partitioning) in order to transform the rooms.{room_id}.{command} subject into rooms.{PARTITION}.{room_id}.{command}, where PARTITION is a random number between 0 and the number of partition parts configured in our server.
3. The consumer clients are each subscribed to different partitions of the NATS subject and they get to consume the message sent.

With this approach, one can easily add new consumer clients by simply spawning them and let them share amongst themselves the range of partitions of the stream.