Hi I have solace dte with 1 topic subscription, I am publishing message on the topic and messages can be duplicate based on some headers and content. For example let’s say I am publishing 50 messages with same productId and product version. Now in consumer I keep these messages in a hashmap and if these message is (id+version) is already in map I ack it. But it seems like message are not removed from broker even after ack
Hi there, I’m Solly AI, I generate responses to your questions using the power of the Solace Agent Mesh:
Please remember that AI technology can make mistakes so ensure that you read our Disclaimer.
AI Generated responses are a work in progress, your help in evaluating my response would be awesome! If my response makes sense, please mark it as a solution or give a 
. If my response doesn’t make sense for your topic, please react to it with a 
.
Looking at the documentation, I can explain why messages might not be getting removed from the event broker after acknowledgment when using DTE with duplicate message handling:
The issue likely stems from how message acknowledgment and spooling work in the event broker:
-
When a message is received by the event broker, it is spooled and matches are created for any subscriptions or queues that match the topic.
-
Messages are only deleted from the spool after:
- All matches have been delivered and acknowledged
- There are no remaining matches associated with the message
-
For DTEs specifically, if a subscription is deleted while a message is spooled for that subscription, the message will still be delivered.
-
Simply acknowledging a message does not guarantee its removal - it must have no remaining matches across all endpoints.
The hashmap approach for handling duplicates based on product ID and version may not be properly handling the acknowledgment process. The documentation recommends using either:
- Looking up Replication Group Message IDs in a database
- Comparing Replication Group Message IDs
To resolve this, you should:
- Use the replication group message ID for duplicate detection instead of product ID/version
- Ensure proper acknowledgment of messages after processing
- Verify there are no other subscriptions or matches keeping the messages in the spool
Here are the relevant documentation links:
Hi @ajay.solace,
Are you acking each message after you process it and add it to your hashmap? It sounds like you are only selectively acking messages that your processing detect as duplicates?
Ok so I have 2 dte with same topic subscription 1 active and another passive, ack from passive happens only if active ack has come except for duplicates. In case of duplicate we simple remove old solace msg from map and ack it. But when we have many duplicate some random solace message are not removed from broker even though ack call happens on duplicate. Also ack can happen from 2 thread. I from listener thread if it’s duplicate or another from active ack thread when same message is processed in active flow.
FYI : we are using jcsmp java api and issue comes only when we have duplicates.
Hi @ajay.solace,
I’m still not 100% following your use case. Are you storing the messages in the queues for long term reference or something like that?
Disregarding the use case there are a few things to keep in mind:
- When you ack a message it will only remove it from the endpoint your consumer is bound to. If you are using 2 endpoints you will need to ack the message from each endpoint. Are you doing that?
- Be careful how you are using threading as
XMLMessageobjects are not safe for use by multiple threads per the docs
Hope that helps!
Ok so we are holding message from passive dte till same message from active flow is ack. Also ack for passive message can happen in listener thread as well as a scheduler thread. What we have noticed is issue comes only when we have duplicate solace message.
Here is code for example.(Only relevant part, apologies for format issue)
Active flow
void handleActive(ByteMessage msg){
activeProcessMap.put(msg.getId(), true);
msg.ack();
}
Passive flow:
void handlePassive(ByteMessage msg){
if(activePocessedMap.contains(msg.getId()){
msg.acknowledge();
} else {
delayMap.put(msg.getId(), msg):
}
}
Inside scheduler thread
delayMap.ketSet().forEach(k→{
if(activePocessedMap.contains(msg.getId()){
delayMap.get(k).acknowledge()
}
});
Hi @ajay.solace,
Okay, I might be wrong, but I think the issue here is that you are storing the messages in the same map when they arrive from different endpoints. When you have a duplicate is it overwriting the message in the map?
Note that I’ll be OOO for several days so hopefully someone else can jump in and help our OR if you have paid support I’d suggest opening a ticket here.
Hope that helps!
On a related note. If you are trying to implement the active/passive pattern you might find it simpler to just use an exclusive queue endpoint. Both consumers can then bind to that endpoint and the broker will only deliver to the active one.
Hi
If my application consumes a message from a Solace queue and sends an ACK, what happens to that message? Will the message be deleted from the queue entirely, or will it only be acknowledged for my application? ( considering some other application also subscribed or in future wants to sync the data from that queue) Additionally, if a message is not deleted, will it still be delivered to my application after sucessfull ack?
Hi @sinchana,
Solace queues are traditional messaging queues. Once the message is acknowledged it is removed from that queue. Depending on the queue type multiple consuming applications can choose what consumption pattern makes sense depending on your requirements.
A few other notes:
-
If you need a separate application to consume the same message that is usually done using hte publish-subscribe pattern that Solace topics enable. The separate application (or it’s own queue) would subscribe to the topic.
-
Solace also offers a message replay capability which can allow an application to replay messages previously acknowledge: Message Replay
If you have a specific requirement or use case you need more input to make happen I would suggest starting a new topic/post on the forum so we can help 
1 Like