Fix to normalization. Global normaliztion was being applied, when ele… #324
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR refines the normalization process for embedding arrays in the Adalflow library. The original implementation applied global normalization using the
normalize_np_array
function, which had limitations in handling both 1D and 2D arrays efficiently. The updated function,normalize_embeddings
, addresses these limitations by:For a detailed comparison between the old and new approaches, including example scenarios and output, please refer to this gist.
Files Changed:
normalize_embeddings
to enhance reliability and flexibility in embedding normalization.normalize_np_array
withnormalize_embeddings
Fixes #323
Breaking Changes:
None.
Was this discussed/agreed via a GitHub issue?
Yes
Did you read the contributor guideline?
Yes
Did you make sure your PR does only one thing, instead of bundling different changes together?
Yes.
Did you make sure to update the documentation with your changes?
Not applicable
Did you write any new necessary tests?
Not exactly. Tests were conducted to verify the function handles both 1D and 2D arrays correctly at the gist shown here https://gist.github.com/Unobtainiumrock/f1f6b993a038ac7641fd6096bb78250f
Did you verify new and existing tests pass locally with your changes?
Yes
Did you list all the breaking changes introduced by this pull request?
No breaking changes were introduced.
Additional Notes:
The improved normalization function ensures the library handles embeddings robustly, minimizing potential errors during preprocessing. This change is essential for developers relying on accurate and reliable embedding normalization. The linked gist provides a detailed breakdown of the differences between the global normalization and row-wise normalization approaches, with illustrative examples.
Had fun working on this! 🙃