Maintaining Strict FIFO Processing with Non-Exclusive Queue in Solace Advanced Event Mesh

Solace Event Broker
1

Hi Team,

I’m looking for guidance on configuring Solace Advanced Event Mesh to achieve strict FIFO processing in our integration scenario.
Current Architecture

Our message flow is as follows:
SAP S/4HANA → Solace Advanced Event Mesh Queue → SAP BTP Integration Suite → Data Lake

*SAP S/4HANA generates business events using Event Enablement Addon.
* The events are published to a SAP Advanced Event Mesh queue.
* SAP BTP Integration Suite consumes the events from the queue, performs the required transformations, and sends them to a downstream Data Lake.

Current Queue Configuration

We are currently using:

*Non-exclusive queue
* No partitioning
* A single queue for all events

Current Scenario

Assume there are 10 messages in the queue.

*Message 1 is processed successfully.
* While processing Message 2, the downstream Data Lake becomes unavailable due to a temporary connectivity issue.
* Since the downstream system is unavailable, Message 2 cannot be delivered successfully.
* However, the Integration Suite continues consuming Messages 3 through 10 from the queue. These messages also fail because the downstream system is still unavailable.

As a result, all subsequent messages are consumed even though the earlier message has not been successfully processed or end up with the same error(which is unnecassary).

Desired Behavior

Our requirement is to maintain strict FIFO processing.

If Message 2 fails because the downstream system is unavailable:

* Message 2 should remain the current message and should be retried once the downstream system becomes available.
* Messages 3 through 10 should remain in the queue and should not be consumed until Message 2 has been successfully processed.
* Once Message 2 succeeds, Message 3 should be processed, followed by Message 4, and so on, preserving strict FIFO order.

In other words, we want a failure on one message to block the processing of all subsequent messages until the failed message has been successfully processed.

Any guidance or recommendations would be greatly appreciated.

Thank you!

2

Hello prerana,

If you need to maintain strict ordering in a non-exclusive queue, you should use partitions. Events of the same type (that is, events with the same partition key) will be assigned to the same partition, where message order is preserved.

However, keep in mind that multiple partition keys can be mapped to the same partition simultaneously, even when other partitions are idle and contain no messages:

Message Distribution with Partitioned Queues

From my perspective, this approach is preferable when you want to enable parallel processing across different event types while still preserving ordering for events that share the same partition key.

Best Regards.
Pedro.

3

Hello prerana,

Since you have mentioned Temporary Connectivity issue which can termed as “Transient Error Handler” should be enabled at your end, If your guaranteed message Delivery is enabled then your message and there is an retry mechanism enabled then your broker just cannot go to the next message to process without processing the message 2. Which means if the retry count fails the message should be there in the Queue until the endpoint system is up and running again to process the message. All the messages should be available in the Queue.

Since the Broker always waits for the acknowledgment from the end point server that the Message was processed successfully then it will discard, But this all features are for Guaranteed Delivery only