Extract DateTime logic

[mjriley]

Product Description

Purely a backend change.

Technical Summary

The recent CommCare Version Tile made me aware that we're inconsistent in how we're treating deserialization. The full logic is in DateTimeProperty._wrap(), but because it is an instance method, we ended up doing our own manual deserialization with datetime.strptime(date_of_use, ISO_DATETIME_FORMAT). Although this works, it ignores the additional checks that DateTimeProperty does. Since, in both cases, we're handling data from the same source (data that was serialized by DateTimeProperty), I've added a class method to allow us to re-use the same logic.

Safety Assurance

Safety story

I haven't done any testing beyond the unit tests, but I believe those tests are sufficient to prove that the existing behavior has been preserved.

Automated test coverage

Added new tests to verify that the existing behavior has not been disrupted while adding the deserialize method.

QA Plan

No QA

Rollback instructions

• This PR can be reverted after deploy with no further considerations

Labels & Review

• Risk label is set correctly
• The set of people pinged as reviewers is appropriate for the level of risk of the change

[sentry-io]

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: corehq/apps/reports/util.py

Function	Unhandled Issue
get_commcare_version_and_date_from_last_usage	UnboundLocalError: local variable 'version' referenced before assignment /a/{domain}/enterprise... Event Count: 1

---

_{Did you find this useful? React with a 👍 or 👎}

[jingcheng16]

data that was serialized by DateTimeProperty

Can you show where is last_submission.date_of_use and last_device.date_of_use being serialized?

[mjriley]

Can you show where is last_submission.date_of_use and last_device.date_of_use being serialized?

I haven't traced through this, but my understanding is that somewhere along either the Pillow or Couch process, Document models are converted to JSON using the field definitions from jsonobject. Jsonobject serializes and deserializes via its wrap and unwrap methods

[jingcheng16]

@mjriley Without confirming we did _unwrap the DateTimeProperty, I'm not very confident to approve the PR. I searched _unwrap in commcare-hq and cannot find any reference.
@dannyroberts @millerdev (Tagging people who might more familiar with Couch. )

[jingcheng16]

One question ( no need to address it in your PR ):
Can we say all occurrences of datetime.strptime(date_of_use, ISO_DATETIME_FORMAT) can be replaced with your new deserialize function now?

[mjriley]

@mjriley Without confirming we did _unwrap the DateTimeProperty, I'm not very confident to approve the PR. I searched _unwrap in commcare-hq and cannot find any reference. @dannyroberts @millerdev (Tagging people who might more familiar with Couch. )

@esoergel was kind enough to point me to the code (mostly) responsible for this.

The code that inserts it into ElasticSearch is here. which you can see from the method's docstring:

Takes a user dict and applies required transfomation to make it suitable for ES.
:param user: an instance dict which is result of CouchUser.to_json()

The parent class has that to_json call here, where, because ElasticUser's model_cls is CouchUser, we'll be calling to_json on a CouchUser object.

At that point, the details of how to_json ends upcalling unwrap is within Couch's Document class and JSONobject.

Edit: copying his notes on how the internals of JSONobject work:

each declared field a jsonobject instance is a subclass of JsonProperty, added there via the metaclass stuff. Then when you set or modify those fields, it calls instance[self.name] = value, which wraps through JsonObject.setitem, which in turn updates self._obj
and to update self._obj, it calls unwrap on the property

[mjriley]

One question ( no need to address it in your PR ): Can we say all occurrences of datetime.strptime(date_of_use, ISO_DATETIME_FORMAT) can be replaced with your new deserialize function now?

Provided these calls represented data that was serialized by DateTimeProperty, yes.

[jingcheng16]

Nice! Thank you for the explanation I also learned something new from it.