modified prov-info and <id>/prov-info to use dataset_type instead of … #596
Conversation
…data types. samples/prov-info didn't appear to include any references. Also applied an identical change to what was done in the data-sankey branch to make validate_organ_code accept an optional argument that includes the organ dictionary so that ontology api doesn't need called thousands of times.
| @@ -2721,7 +2719,7 @@ def get_prov_info(): | |||
| # Token is not required, but if an invalid token is provided, | |||
| # we need to tell the client with a 401 error | |||
| validate_token_if_auth_header_exists(request) | |||
|
|
|||
| organ_types_dict = schema_manager.get_organ_types() | |||
There was a problem hiding this comment.
Looks like the organ_types_dict is assigned twice with the same call:
Lines 2722 to 2728 in 95b243e
Just keep one.
| @@ -3016,7 +2992,7 @@ def get_prov_info_for_dataset(id): | |||
| # Token is not required, but if an invalid token provided, | |||
| # we need to tell the client with a 401 error | |||
| validate_token_if_auth_header_exists(request) | |||
|
|
|||
| organ_types_dict = schema_manager.get_organ_types() | |||
There was a problem hiding this comment.
Found duplicate in /datasets/<id>/prov-info too
There was a problem hiding this comment.
@yuanzhou I wonder if this duplication was caused because I applied the same change in 2 different branches. I'll fix this real quick. So what happens is, whenever that validate organ code function got added, it made an external call to ontology api (this was probably added while converting yaml references to calls to ontology) but since validate organ code is called once per dataset, it ended up making the prov info endpoints take several minutes to run. I saw that this validate organ code function gets called multiple times throughout entity api and I wanted to createa fix that required as little reworking and re-testing of those other endpoints as possible, so I just added an optional argument to that function where that dict can be passed in so in situations where that function is called inside a loop, its not making constant calls to ontology. The thing is, I had to make this change in 2 different branches in entity-api, so I was curious if there would be any sort of duplication. Will push fix in a couple minutes
There was a problem hiding this comment.
I don't see the duplicates in the sankey data PR, could just be in this prov-info branch.
That's good performance optimization you did even though that organ call internally is being cached.
There was a problem hiding this comment.
Yes my concern was less about the speed of the prov-info endpoints (since as you mentioned, its cached) but rather clogging up ontology with thousands of requests.
|
@yuanzhou those duplicates have been removed |
modified prov-info and /prov-info to use dataset_type instead of data
types. samples/prov-info didn't appear to include any references. Also
applied an identical change to what was done in the data-sankey branch to
make validate_organ_code accept an optional argument that includes the organ
dictionary so that ontology api doesn't need called thousands of times.