Its a common question that comes in mind while dealing with JMS.
How to guarantee, there is no message loss?
Also how to secure message delivery, or guarantee message delivery.
How to guarantee, there is no message loss?
Also how to secure message delivery, or guarantee message delivery.
Some of the considerations are below:
Persistent Message( in case of Queue and Topics):
A message can be persistent or non-persistent.
Acc. to JMS specification, when a message is marked as persistent, the JMS provider must “take extra care to insure the message is not lost in transit due to a JMS provider failure”.
If the message is persistent, message is stored in the data structure representing the Queue(it can be table in database or file, etc. ), otherwise message is stored in memory.
When message is sent over Queue, If a consumer is connected the message will be dispatched to it. If there are no consumers connected the message will remain saved on disk until a consumer connects, whereupon it will be dispatched. If message is non persistent message is saved in memory. Which means if the Queue Broker or Queue Manager is restarted, the message would be lost.
A message can be persistent or non-persistent.
Acc. to JMS specification, when a message is marked as persistent, the JMS provider must “take extra care to insure the message is not lost in transit due to a JMS provider failure”.
If the message is persistent, message is stored in the data structure representing the Queue(it can be table in database or file, etc. ), otherwise message is stored in memory.
When message is sent over Queue, If a consumer is connected the message will be dispatched to it. If there are no consumers connected the message will remain saved on disk until a consumer connects, whereupon it will be dispatched. If message is non persistent message is saved in memory. Which means if the Queue Broker or Queue Manager is restarted, the message would be lost.
Durable Subscription(In case of Topics):
A subscription on a topic can either be durable or non-durable. The term durable applies to the subscription, not the topic itself or the messages sent to it. And it applies only to subscriptions on topics, not subscriptions on queues.
If the subscription is open(active), both non-durable and durable behave in same way.
Diff. comes when the subscription is closed. Only durable Client would receives missed messages, when subscription is again open/active. In case of topics, there can be different combinations of persistent/non-persistent messages and durable/non-durable subscription.
If subscription is durable, then message would be saved if the subscriber is inactive, otherwise not.
If message is persistent, it would be saved on disk otherwise in memory.
So, same message would be saved for durable subscribers, and wont be saved for non durable subscriber.
In case of topics there can be diff scenarios:
A subscription on a topic can either be durable or non-durable. The term durable applies to the subscription, not the topic itself or the messages sent to it. And it applies only to subscriptions on topics, not subscriptions on queues.
If the subscription is open(active), both non-durable and durable behave in same way.
Diff. comes when the subscription is closed. Only durable Client would receives missed messages, when subscription is again open/active. In case of topics, there can be different combinations of persistent/non-persistent messages and durable/non-durable subscription.
If subscription is durable, then message would be saved if the subscriber is inactive, otherwise not.
If message is persistent, it would be saved on disk otherwise in memory.
So, same message would be saved for durable subscribers, and wont be saved for non durable subscriber.
In case of topics there can be diff scenarios:
Multi Consumer Queue Multi Consumer Queue is a concept some times confused with Topic. Multi Consumer Queue, is a case when multiple consumers register with a Queue. Message Broker routes the incoming messages to different consumers. Its all the case of load balancing, where one batch of messages is sent to One consumer, and next batch batch is sent to next consumer in line, and similarly batch of messages is sent to different consumer, but same message is not sent to multiple consumers. Messages are routed to active consumers in order they registered with the broker. Idea behind is the message production rate is much higher, and there is a need of load balancing on the message consumer side. We have a special scenarios to consider, which might relate to message loss. When message production rate is slow, broker might dispatch messages in uneven manner, and some consumers may never receive any message. There should be a backup consumer, if the active consumer fails. And to avoid whole batch of messages.
Message is an entity, and once client delivers the message to Message Broker inside Provider, its role is finished. Message are passed to consumer using the store and forward paradigm. To avoid loosing message in case of Provider failure OR consumer failure, message should be persistent message. Message can be in many states –
It might be prepared by Client and not yet delivered to Broker.
Message is routed to Broker by the Message Producer.
Message is at Broker and waiting for Consumer to consume it.
Message has been delivered from Broker to Consumer.
It might be prepared by Client and not yet delivered to Broker.
Message is routed to Broker by the Message Producer.
Message is at Broker and waiting for Consumer to consume it.
Message has been delivered from Broker to Consumer.
When we make message Persistent or when we make Durable Subscription, we guarantee that message wont be lost from Broker. But message can be in a different state also, OR message is carrying some data or information, which should not be lost. If message is delivered from Broker, it must reach Consumer, it should not be lost in network, and if in case it is lost, there should be redelivery.
Guaranteed Delivery – When we make message persistent, we guarantee that message wont be lost from broker. In case of Topics, we take one extra care that we make durable subscriptions. We have to take care that Message is not lost on network, or if it is lost in network, we must know and redeliver. Message Acknowledgment is key to guaranteed messaging and ensures that is message is lost from network while travelling from producer to consumer, it would be redelivered.
Message Acknowledgment - There are several types of Acknowledgement, and acknowledgement is set on session. Message Acknowledgment has different meaning for Broker and Consumer. Acknowledgment means consumer acknowledges that message is received. Acknowledgement is sent from broker, and also from consumer to broker.
Brokers Acknowledgement( Broker to producer) - Broker received message, stored in memory or data store(in case of persistet message), and accepts responsibility of delivering it to consumer.
Consumer Acknowledgement(Consumer to broker) – Acknowledgment is sent according to the Acknowledgement Mode. In case of AUTO, the session automatically acknowledges the client’s receipt of message. In CLIENT, client has to explicitly send acknowledgement by calling message’s acknowledge method. In DUPS_OK, acknowledgment is sent after consuming particular number of messages( lazy acknowledgement). Message can also be received in TRANSACTIONAL_MODE, where message is received in JTA transaction, and acknowledgement is sent only if transaction commits. If broker doesn’t receives an acknowledgement, it would attempt to redeliver the message, with setting a flag JMSRedelivered on message.
There can be a scenario, where consumer consumes a message, and sends ack., but ack. fails and lost in network, and broker would assume that message is lost and not consumed by consumer, and it would try to redeliver, this can lead to duplicate message for consumer. Consumers should always check for JMSRedelivered flag for identifying the duplicate messages.
Remember, Message Re delivery not guaranteed for more than once.
In a complex architecture, several consecutive operations can be combined under single transaction. We can use the same session to receive a message from a queue and send a message to a topic in the same transaction, if required, programatically. We cn make the session transactional, and perform multiple operations into one atomic unit of work if required. Please find below were it was required to use JMS local transaction for guaranteeing delivery.
Scenario – Transfer messages from one queue to another. Assumption – Two Queues – Queue1 and Queue2, we are going to write a Java program to transfer messages.
Remember, Receiving a message form Queue1 and Sending message to Queue2 are two different transactions.
Transaction(T1) – Receive message form Queue 1 Transaction(T2) – Send message to Queue2.
We need to perform both transactions in XA fashion, means if any one fails, it should be rolledback. Means for any particular message, if any transaction(T1 or T2) fails, it should be send back to Queue1. T1 and T1 should be combined into one transaction. We can use XA resources, but here we are doing it manually(to observe more closely). -
Remember, Receiving a message form Queue1 and Sending message to Queue2 are two different transactions.
Transaction(T1) – Receive message form Queue 1 Transaction(T2) – Send message to Queue2.
We need to perform both transactions in XA fashion, means if any one fails, it should be rolledback. Means for any particular message, if any transaction(T1 or T2) fails, it should be send back to Queue1. T1 and T1 should be combined into one transaction. We can use XA resources, but here we are doing it manually(to observe more closely). -
JMSMessageReciver – program to receive message from JMS Queue. MQMessageSender – program to message over MQ.