Derek furst/multiple components datasets #545
Conversation
…d. For now, returns uuids of the entities created.
containing both datasets. Still need to apply the same modifications and read triggers done with create_entity. Creation action field is not currently supported in the triggers. Need to merge in changes from creation_action branch
…has its support merged in. Normalized output. Added reindex call like is done on a normal create entity request.
…eation action from the dataset since this is set manually act activity creation time. Replaced json_data_dict with dataset where it was put mistakenly during validation.
There was a problem hiding this comment.
@DerekFurstPitt the way you handle this specialized endpoint with mixed use of schema components can be simplified and the SenNet project will appreciate this effort too. I'll describe the steps:
-
First check the required
X-Hubmap-Applicationheader. And group all the input json validations at the beginningwithout using any. The purpose is to ensure data integrity by checking the required fields and their data values in terms of existence, size, and type...schema_manager.validate_json_data_against_schema() -
Similar to
create_multiple_samples_details(), we create two set of IDs (which also validates thegroup_uuidcomprehensively) via uuid-api usingschema_manager.create_hubmap_ids()for the two component datasets. Then build two merged dicts and callschema_manager.generate_triggered_data()on each to generate all the dataset node properties via the schema. -
As what you've already done, generate one Activity node to be linked to the direct ancestor dataset and the two new component datasets. It's also the time to update the
Activity.creation_actionwith the value of "Multi-Assay Split" as you did. -
Create a similar app neo4j query as
app_neo4j_queries.create_multiple_samples()and get everything (nodes and relationships) into Neo4j, without using the schema. -
Add the new datasets to indices via reindex. And return the desired data structure back to ingest-api. Just like what you did.
By following the above approach, we'll have one specialized endpoint that handles the custom json input with custom validations, and one app-specific neo4j query.
…will parse the entity-api-spec.yaml file
…ormat Fixed so that www.yamllint.com and more importantly yaml.safe_load() …
multiple samples endpoint.
There was a problem hiding this comment.
I think this addresses the major points that we all discussed. I believe you also addressed all the issues @yuanzhou mentioned in the card but I will leave that to him to confirm
There was a problem hiding this comment.
@DerekFurstPitt the rework looks great! In addition to updating the comments to indicate the correct return results, can you also sync the latest main into your branch? When I tested locally with your branch, I noticed the dependency issue that had been addressed in main.
…panying details function. Merged in changes from main.
Implemented datasets/components endpoint to register multiple components from a mult-assay ancestor. These entities are not
created as part of a typical entity creation process. We can't use the normal triggers because those are configured to work with a single ancestor per child dataset. Instead, all of the validation is handled within this endpoint. For activity creation and linkages to the direct ancestor, new functions were created. While, for now, we are only allowing a single parent ancestor, the functions were created such that they would accommodate multiple ancestors. The main difference between this schema function and the existing ones is that it accepts multiple child entities rather than just one. The activity created is of the type Multi-Assay Split