Merge two records into a single record

[george-neha]

As a user/Social Worker (SW), I want to merge two similar records in an entity into one to avoid duplicates and ensure consistency and accuracy.

Objective
In the system, two users/SWs may record notes/tasks/schools/children twice, which can lead to duplication of work and waste of their time. To avoid this, there should be an option to merge two records of a single record. After merging the two records in the system, there should be only one record with the details captured from both records to ensure no loss of unique data.

Proposed Solutions & Alternatives
Acceptance Criteria:

1. I should be able to select or search two records (even archived/inactive ones) from the list I want to merge. If the toggle for archive functionality is ON then user should be able to search an archived record for merge as well.
2. I should be able to select data from either or both records. (see exceptions below)
3. I should be able to preview the data fields of both records simultaneously and select the data I want to merge.
4. Merging of records should be a one-time activity. Once the records are merged into one, the deleted data should not be available.
5. After merging two records, the system should notify the SW/user that “ assigned to you has been merged.”

Exceptions:

1. If there is a date field, I should be able to select only one date (field-level logic)
2. For fields that have only single-select dropdown values (e.g., gender, status, category, address), I should be able to select only one field.

Out of scope

1. Merging more than two records. After getting user feedback, we can scale it up to 5 records.
2. Automatic detection of duplicate records and merging them
3. Audit trail of the records that are merged.

Low fidelity skteches for merging two records

[Abhinegi2]

To implement the merge functionality this is my approach:

• Add a "Merge" button in the bulk actions item.

• Manage Selection Validation:
  If more than two rows are selected and the user clicks on "Merge," show a warning: "You can only select 2 rows for merging right now."

• Create a new MergePopupComponent:

• Display a popup showing:

  - Child A and Child B are side by side for comparison.
  - Merge Preview to show the resulting merged data.
  

• Allow users to:

  • Select data from either or both records.
  • Choose only one option for date fields and single-select dropdowns.
  • Include action buttons: Cancel and Merge.
    - Show a confirmation dialog before merging.

• After merging, show a success message: "Records merged successfully."

Let me know if it looks good to you and if you have any suggestions

[Abhinegi2]

@george-neha, I've created the first draft of the merge screen. You can check it out (https://pr-2879.aam-digital.net/) by navigating to Bulk Action -> Merge.

I'm also attaching a screenshot of the current UI. Let me know if you have any feedback or suggestions. In the meantime, I'll continue working on the functionality.

[Abhinegi2]

@sleidig
I have a question about the merge feature. Currently, I am merging all the updated details into the first record, updating that entity, and then deleting the second record. Is this the best approach, or should I create a new entity record(with updated details) and delete both selected records instead?

[sleidig]

That's a good question, @Abhinegi2 ...

I think updating one of the existing records is good. It means we have to make less changes to related records (other entities that reference this only have to be changed if they point to the second, deleted ID - although this is only part of a second step #2858 ). At the moment I can't think of a disadvantage of this compared to creating a completely new record. Do you see any potential issues?

[Abhinegi2]

Currently, the only disadvantage I see with creating a completely new record is that the related records from any of the selected entities do not merge. However, when merging into an existing record, we at least retain the references from the selected record(which we are going to cover in 2858).

[george-neha]

1. For fields other than the exceptions (mentioned above), as a user, I should be allowed to select values from records A & B for merging.

2. Bug- sometimes, on selecting a field, both fields get selected

3. Address is not displayed

4. After merging, only one record should exist. currently, after merging till the page is loaded, both records are visible in the list.

[george-neha]

@Abhinegi2 In notes, when merging two records, the system displays an internal student ID instead of the child's name.

[sleidig]

In notes, when merging two records, the system displays an internal student ID instead of the child's name.

@Abhinegi2 for this as well as some similar remarks: I would suggest to use EntityFieldViewComponents and/or EntityFieldEditComponents here, so that we always use the dataType specific UI

[Abhinegi2]

I have integrated the EnfityFieldViewComponent for now, and it seems to be working fine. @george-neha please check now also currently after merging the Notes record related entities not working as expected which I will covering in #2858

[george-neha]

1. During merge, can the fields listed in an order such that fields of at least one tab are together? Currently, during the merge of 2 children's records, fields from the dropout and health tabs come in between the fields of general information.
2. Before the records are merged, display a warning message to alert the user that this is one time activity and cannot be undone.

[sleidig]

1. During merge, can the fields listed in an order such that fields of at least one tab are together? Currently, during the merge of 2 children's records, fields from the dropout and health tabs come in between the fields of general information.

This is not trivial (e.g. the same field can be displayed under multiple tabs; some fields may not be shown in any tab but still contain old data that has to be merged to avoid data loss; etc.). Let's discuss this separately as a sub-issue to the overall feature

[Abhinegi2]

@george-neha I have added a confirmation dialog similar to other bulk actions, Please check now

[sleidig]

Looks good so far!

I have a few extra points on my UX wishlist still:

• I would like to manually edit the values in the "Merge Preview" column, using the normal form fields (EditComponent). (e.g. instead of using the exact name from record A or B, I want to type a new name that may be a custom combination of the two previous names)
• I would like to have "smart" pre-selection of merged values:
  • if the values are identical -> select the value
  • if only one record has a value and the other's value is empty -> select the one with a value
  • ...?
  • for other fields, by default select the one from record A, so that I don't have to click through all the fields

@george-neha , are these part of any of the other sub-issues already?
Maybe at least the second point should become its own sub-issue, to keep this manageable here.

[george-neha]

The first point is covered in #2857, where the user can manually select the children/groups/users to a particular record and add additional children/groups/users to the record.

I'll create a sub-issue for the second point you mentioned. #2892

[Abhinegi2]

@sleidig and @george-neha, I’ve implemented the first point—users can now edit the columns. Please have a look now (https://pr-2879.aam-digital.net/). Meanwhile, I’m working on code cleanups.

Also, I’m thinking of hiding the labels in the merge preview columns. What do you think?

[sleidig]

Nice! Although the label is also shown on the left, for me it doesn't distract having it on the merge preview (and actually makes it a bit easier, not having to look across the screen to see the context). I would leave it as it is

[george-neha]

@Abhinegi2 In notes/tasks, I'm only able to select only one of the options from children/groups/users.

[sleidig]

@Abhinegi2 In notes/tasks, I'm only able to select only one of the options from children/groups/users.

Isn't that a requirement only covered in #2857 , @george-neha ?

[george-neha]

This was part of the original requirement to allow users to select data from both records.

2. I should be able to select data from either or both records. (see exceptions below)

I can update the ticket accordingly if we all feel it's better to be covered under #2857.

[Abhinegi2]

For the first draft, I’m thinking of releasing this initial version (current first draft) to users. It now includes the merge functionality for all entities, including notes. I’ve updated the logic so that the entityType dropdown now supports multi-select.
Please test it and share your feedback. I feel it would be best to roll out this first version independently rather than wait for the integration branch to include all features. We can cover the remaining areas in #2857.

Let me know your thoughts!

[aam-digital-ci]

🎉 This issue has been resolved in version 3.48.0-master.4 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[aam-digital-ci]

🎉 This issue has been resolved in version 3.48.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀