Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jsonpath support for Ballerina json module #5938

Closed
5 tasks done
SasinduDilshara opened this issue Jan 12, 2024 · 12 comments · Fixed by ballerina-platform/module-ballerina-data.jsondata#1
Closed
5 tasks done
Assignees

Comments

@SasinduDilshara
Copy link
Contributor

SasinduDilshara commented Jan 12, 2024

Currently JsonPath is not supported for Ballerina json types.
The aim of this task is to introduce the Jsonpath support for Ballerina Json type.

Sub tasks

  • Integrate with Java existing JsonPath library library by converting to string (Will refer as Jayway Implementation in the future comments)
  • Integrate with Java existing JsonPath library library by imlementing SPIs. ( WIll refer as SPI Implementation in future comments)
  • Optimize the SPI implementation
  • Load Test the implementations
  • Add unit tests
@SasinduDilshara
Copy link
Contributor Author

SasinduDilshara commented Jan 12, 2024

Currently working on the third subtask.

I compared the output module of the first Jayway Implementation and the SPI Implementation task.
When the complexity of the jsonpath syntax get increased the SPI approach become slow.
But the SPI method is fast for the simple jsonpaths

Currently working a way to optimize

@SasinduDilshara
Copy link
Contributor Author

SasinduDilshara commented Jan 18, 2024

Currently working on the third subtask.

I compared the output module of the first task and the second task. When the complexity of the jsonpath syntax get increased the second approach become slow. But the second method is fast for the simple jsonpaths

Currently working a way to optimize

The reason for this performance issue is, com.jayway.jsonpath library use toString() method inside their function calls.
So currently, the toString runtime implementation for Ballerina maps is 10x slow than the Java json toString implementation.

@SasinduDilshara
Copy link
Contributor Author

SasinduDilshara commented Jan 18, 2024

Currently working on the third subtask.
I compared the output module of the first task and the second task. When the complexity of the jsonpath syntax get increased the second approach become slow. But the second method is fast for the simple jsonpaths
Currently working a way to optimize

The reason for this performance issue is, com.jayway.jsonpath library use toString() method inside their function calls. So currently, the toString runtime implementation for Ballerina maps is 10x slow than the Java json toString implementation.

But if we consider some basic queries and popular queries, the SPI implementation shows good performance.

@SasinduDilshara
Copy link
Contributor Author

While doing the fourth subtask, blocker was found
#5958

@SasinduDilshara
Copy link
Contributor Author

While doing the fourth subtask, blocker was found #5958

Found a workaround for the blocker, currently doing the load testing.

@SasinduDilshara
Copy link
Contributor Author

The load test results for the SPI implementation and the jayway implementation

https://docs.google.com/spreadsheets/d/1wvb6a11Oh7oZkjKAkEeDU9pFpmrCFVpsNyTQu5_-pHk/edit?usp=sharing

According to this the average response time and the throughput of the SPI implementation is better in below ways.
1 Thread - 19% better
2 Thread - 29%
5 Thread - 44%
10 Thread - 25%

I used $.store.book[*].price as the jsonpath expression.

@SasinduDilshara
Copy link
Contributor Author

Important to note that, SPI implementation is 20% slower for jsonpath expressions like $..price.avg(). But this notation is not very much common. For common expressions like field accessing arrays and comparing expressions SPI implementation shows better performance(as mention in the above comment).

@SasinduDilshara
Copy link
Contributor Author

Now, as the next step, I am Implementing unit tests for the SPI implementation.

@prakanth97
Copy link
Contributor

prakanth97 commented Jan 22, 2024

https://docs.google.com/spreadsheets/d/1wvb6a11Oh7oZkjKAkEeDU9pFpmrCFVpsNyTQu5_-pHk/edit?usp=sharing

A small suggestion, instead of sharing the google doc link which is restricted to few people, shall we add screenshot or doc link that can be accessed by everyone?

@SasinduDilshara
Copy link
Contributor Author

https://docs.google.com/spreadsheets/d/1wvb6a11Oh7oZkjKAkEeDU9pFpmrCFVpsNyTQu5_-pHk/edit?usp=sharing

A small suggestion, instead of sharing the google doc link which is restricted to few people, shall we add screenshot or doc link that can be accessed by everyone?

+1, will share a public doc

@SasinduDilshara
Copy link
Contributor Author

PR for the implementation

ballerina-platform/module-ballerina-data.jsondata#1

@SasinduDilshara
Copy link
Contributor Author

Issue found for the concat operations of JsonPath expressions in the library.
Reported in here ballerina-platform/ballerina-lang#42064

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
2 participants