Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tie-up tasks for Cache Data Import/Export using JSON #82

Open
jxblum opened this issue May 13, 2020 · 3 comments
Open

Tie-up tasks for Cache Data Import/Export using JSON #82

jxblum opened this issue May 13, 2020 · 3 comments

Comments

@jxblum
Copy link
Contributor

jxblum commented May 13, 2020

The following things need to be investigated further:

  • DONE - Test with a ClientCache and client LOCAL Region(s) with no connections (i.e. no Pool defined) to a cluster/server using PDX

Currently Throws:

...
Caused by: org.apache.geode.pdx.JSONFormatterException: Could not parse JSON document: [Source: (String)"{"@type":"example.app.model.Customer","id":1,"name":"Jon Doe"}"; line: 1, column: 63]
...
Caused by: org.apache.geode.cache.CacheClosedException: Client pools have been closed so the PDX type registry is not available.

:(

  • DONE - Test using @EnableClusterAware annotation (this is currently not working because the "Cluster Configuration Push" happens in the SmartLifecycle phase of container startup and the "Import" happens in the BeanPostProcessor.postProcessAfterInitialization() callback on the Region, which occurs before the push... #sigh)

  • DONE - Test using @EnableCachingDefinedRegions annotation (i.e. Regions created from Spring's Cache Abstraction annotations or JSR-107, JCache API annotations)

  • Test using @EnableClusterDefinedRegions annotation (i.e. Regions created on the client from the server Region definitions)

  • Test export/import with entity having natural id (e.g. Book.isbn) that would need to include a @identifier JSON object metadata field. Possible for POJO stored in the Region since we can inspect the object class type, not possible when the Region value is a PdxInstance that did not originate from JSON. In that case, the user must manually make sure the non-JSON generated PdxInstance includes a @identifier PDX field before it is exported to JSON.

  • Add more documentation around features & limitations (gotchas):
    ** Java 8 Types, JSR-310 (Time) Types and Jackson 2 vs. 3, etc.
    ** Use of multiple Cache Data Importers and ordering using the @Order annotation or the Ordered interface on Cache Data Importer/Exporter bean definitions.
    ** Explain the (hidden) use of Predicate's to filter Regions on import and export.
    ** Explain the API/SPI for writing extensions, such as Cache Data Importers/Exporters for XML,
    XLS, SQL DML or other data formats.
    ** Explain the types of numeric values: e.g. application domain model class (e.g. Customer) might define an id field of type Long but the value for the corresponding JSON field (i.e. "id") will be a byte as processed by JSONFormatter, therefore Region.get(key) lookups must be customersRegion.get((byte) 1) even if the "Customers" Region is defined as Region<Long, Customer>. Also the Region entry value will be a PdxInstance and not Customer.
    ** ???

  • ???

@jxblum
Copy link
Contributor Author

jxblum commented Jun 16, 2020

Completed handling for client LOCAL Regions. See Issue #91, commit and test class.

@jxblum
Copy link
Contributor Author

jxblum commented Jun 16, 2020

Completed handling for Cache Data Imports/Exports in conjunction with the @EnableClusterAware annotation. See Issue #90, commit and test class, and specifically the configuration.

@jxblum
Copy link
Contributor Author

jxblum commented Jun 17, 2020

Filed the following Apache Geode JIRA tickets related to these problems:

  • GEODE-8235 - "Server should not be required to have an available PDX type registry for ClientCache applications."

  • GEODE-8254 - "JSONFormatter cannot parse JSON Arrays."

  • GEODE-8255 - "JSONFormatter does not properly generate the @type metadata field."

  • GEODE-8256 - "The Jackson ObjectMapper used by a PdxInstance does not properly handle Java 8 Types."

  • GEODE-8257 - "_ The Jackson ObjectMapper used by a PdxInstance cannot handle typed JSON Objects._"

  • GEODE-8258 - "_ Cannot implement PdxInstance and store in a Region._"

@jxblum jxblum removed the in-progress label Aug 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant