Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Common: fix API Request Enrichment output deserialization #374

Closed
oguzhanunlu opened this issue Oct 12, 2020 · 2 comments
Closed

Common: fix API Request Enrichment output deserialization #374

oguzhanunlu opened this issue Oct 12, 2020 · 2 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@oguzhanunlu
Copy link
Member

oguzhanunlu commented Oct 12, 2020

io.circe.jackson.mapper deserializes all integers and numbers to JsonBigDecimal and downstream in the codebase, it is converted to DecimalNode by circeToJackson() in iglu-scala-client, and the validator treats DecimalNode as number which results in a bad row with number found, integer expected error message in case output schema of API Request Enrichment contains integer property

We'll introduce a custom mapper whose only difference compared to the original mapper is how integers are deserialized. Unlike the original, it'll deserialize integers to JsonLong.

Note that json schemas might contain integer values that don't fit into Long and this case will also result in a bad row, but we don't expect to see this case in real life.

The actual diff happens in CirceJsonDeserializer.deserialize where

case JsonTokenId.ID_NUMBER_INT | JsonTokenId.ID_NUMBER_FLOAT =>
  (Some(Json.JNumber(JsonBigDecimal(jp.getDecimalValue))), parserContext)

is replaced by

case JsonTokenId.ID_NUMBER_INT => (Some(Json.JNumber(JsonLong(jp.getLongValue))), parserContext)
case JsonTokenId.ID_NUMBER_FLOAT => (Some(Json.JNumber(JsonBigDecimal(jp.getDecimalValue))), parserContext)
@chuwy
Copy link
Contributor

chuwy commented Oct 12, 2020

The bug manifests itself as a bad row produced by post-enrichment schema-validation. The offending entity is added by API Request enrichment. If its output schema requires integer type, the validator complains that "42 is a number, but integer expected", because in its internal representation, 42 is a number.

@chuwy
Copy link
Contributor

chuwy commented Oct 12, 2020

What about SQL Query enrichment, @oguzhanunlu? It's inputs also seem to be affected, but I don't see any changes regarding to that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants