add support for receipted_message_id & fix correlation_handler bug

[komuw]

We are currently using sequence_numbers to correlate a request to smsc and a response from smsc:

naz/naz/client.py

Lines 380 to 386 in 881c1fe

	# associate sequence_number with log_id.
	# this will enable us to also associate responses and thus enhancing traceability of all workflows
	try:
	await self.correlation_handler.put(
	sequence_number=sequence_number, log_id=log_id, hook_metadata=""
	)
	except Exception as e:

The SMSC spec has an optional parameter user_message_reference that you can send to SMSC as part of the request(submit_sm) and SMSC will send it back in the response(deliver_sm)

see section 4.4.1 & 4.6.1 of smpp spec.

Two things to note:

1. when you send submit_sm the sequence_number you specify is returned in the submit_sm_resp.
   However, that sequence_number is not available in the deliver_sm request from SMSC.
   This means that our correlation as it currently exists;
   

   naz/naz/client.py

   Lines 380 to 386 in 881c1fe

   	# associate sequence_number with log_id.
   	# this will enable us to also associate responses and thus enhancing traceability of all workflows
   	try:
   	await self.correlation_handler.put(
   	sequence_number=sequence_number, log_id=log_id, hook_metadata=""
   	)
   	except Exception as e:

   
   is broken.
   We need to implement user_message_reference to try and fix that

2. When you send a user_message_reference in submit_sm, it is not returned in submit_sm_resp. it is only returned in deliver_sm.

So way forward:

• continue using our current correlation, but only to correlate submit_sm and submit_sm_resp
• implement receipted_message_id to correlate submit_sm and deliver_sm

One is a bug while the second one is a feature request.

We should at the very least fix the bug

[komuw]

Here's an alternative correlation workflow.

• when sending submit_sm we save the sequence_number

• when we get submit_sm_resp we lookup sequence_number and use it for correlation

• Additionally, submit_sm_resp has a message_id in the body. This is the SMSC message ID of the submitted message.

• We take this message_id and save it.

• when we get a deliver_sm request, it includes a receipted_message_id in the optional body parameters. This is the SMSC message ID of receipted message.

• we either:
  -(a) get that receipted_message_id and use it to lookup from where we had saved the one from submit_sm_resp
  -(b) parse the short_message and get the message_id from there, then and use it to lookup from where we had saved the one from submit_sm_resp(ie part of handle Delivery Receipt(deliver_sm) Format #82)
  option (b) is better since short_message is a mandatory parameter, whereas receipted_message_id is optional
  Unfortunately for option(b); it is vendor specific so we can't rely on it.

  We will use option(a) ie using receipted_message_id even thought it is an optional param that is not always there.

  In other words, all these is just BEST EFFORT; nothing is guaranteed to work whichever way you slice it.
  see:

  • Vendor specific delivery receipt format breaks Deliver_sm opentelecoms-org/jsmpp#62
  • SMPP raises error when trying to parse Safaricom delivery reports. praekeltfoundation/vumi#298
  • The SMPP transport should look for PDU params to detect delivery reports praekeltfoundation/vumi#637