Random "failed to retrieve" error in streaming scenario

Hi Jonas,

I cannot determine the source of the problem from the snippets here. The cut and paste appears to have removed a lot of whitespace that would make the code and logs more legible. However if I separate the debug log by timestamps I see this:

13:11:25,779 [WARNING] solace.messaging.core: [_message.py:497] Failed to retrieve 2024-11-23 14:11:25 Caller Description: Message->process_rest_data. Error Info Sub code: [2]. Error: [Bad msg_p pointer '(nil)' in _solClient_msgHeaderMap_getString]. Sub cod e: [SOLCLIENT_SUBCODE_PARAM_NULL_PTR]. Return code: [Fail] 2024-11-23 14:11:25 2024-11-23 13:11:25,780 [ERROR] solace.messaging.receiver: [_persistent_message_receiver.py:348] [[SERVICE: 0xffff80e3beb0] [RECEIVER: 0xffff81f88310]] Failed to retrieve 2024-11-23 14:11:25 Caller Description: Message->process_rest_data. Error Info Sub code: [2]. Error: [Bad msg_p pointer '(nil)' in _solClient_msgHeaderMap_getString]. Sub code: [SOLCLIENT_SUBCODE_PARAM_NULL_PTR]. Return code: [Fail] 2024-11-23 14:11:25 Exception ignored on calling ctypes callback function: <bound method _PersistentMessageReceiver._flow_message_receive_callback_routine of <solace.messaging.receiver._impl._persistent_message_receiver._PersistentMessageReceiver object at 0xffff81f88310>> 2024-11-23 14:11:25 Traceback (most recent call last): 2024-11-23 14:11:25 File "/usr/local/lib/python3.10/dist-packages/solace/messaging/receiver/_impl/_persistent_message_receiver.py", line 349, in _flow_message_receive_callback_routine 2024-11-23 14:11:25 raise PubSubPlusClientError(message=exception) from exception 2024-11-23 13:11:25,781 [WARNING] solace.messaging.core.api: [_inbound_message_utility.py:192] Unable to get message id 2024-11-23 14:11:25 Caller Description: _InboundMessage->get_message_id. Error Info Sub code: [2]. Error: [Bad msg_p pointer '(nil)' in solClient_msg_getMsgId]. Sub code: [SOLCLIENT_SUBCODE_PARAM_NULL_PTR]. Return code: [Fail] 2024-11-23 14:11:25 Error during message settlement: Unable to get message id 2024-11-23 14:11:25 Caller Description: _InboundMessage->get_message_id. Error Info Sub code: [2]. Error: [Bad msg_p pointer '(nil)' in solClient_msg_getMsgId]. Sub code: [SOLCLIENT_SUBCODE_PARAM_NULL_PTR]. Return code: [Fail]

There appear to be duplicate entries, some lines truncated or interrupted, plus some entries off by exactly one hour (timezone setting?) which makes me think there is possibly some multi-processing access occurring. The Solace PubSub+ Messaging API for Python does not support multi-processing access to the same messaging service.

To get to the true source though we will need to understand your threading model, use of executors, and any other parallelism techniques as this appears on the surface to be some sort of concurrency issue.

Regards

Ragnar