Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use DictTuple(tuple) for Dataset.assets and SyftClient.datasets instead of TupleDict(OrderedDict) #8164

Merged
merged 32 commits into from
Oct 30, 2023

Conversation

kiendang
Copy link
Contributor

Description

Please include a summary of the change, the motivation, and any additional context that will help others understand your PR. If it closes one or more open issues, please tag them as described here.

Affected Dependencies

List any dependencies that are required for this change.

How has this been tested?

  • Describe the tests that you ran to verify your changes.
  • Provide instructions so we can reproduce.
  • List any relevant details for your test configuration.

Checklist

@madhavajay
Copy link
Collaborator

@kiendang interesting. Lets discuss in the Eng channel in slack.

@kiendang kiendang force-pushed the dataset-index-2 branch 3 times, most recently from 8bab166 to ce69f4c Compare October 21, 2023 02:49
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@kiendang kiendang force-pushed the dataset-index-2 branch 3 times, most recently from 538be69 to c9f8d7d Compare October 21, 2023 08:16
@kiendang kiendang marked this pull request as ready for review October 21, 2023 09:54
@kiendang kiendang force-pushed the dataset-index-2 branch 3 times, most recently from bd3c9cb to 06c2a0a Compare October 22, 2023 06:53
@madhavajay
Copy link
Collaborator

@kiendang great work, I had a play and found a possible bug.

It seems if the keys are ints and non 0 sequential it freaks out on casting to a dict.
Screenshot 2023-10-23 at 3 46 33 pm

@madhavajay madhavajay self-requested a review October 23, 2023 05:48
Copy link
Collaborator

@madhavajay madhavajay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work. The feel of using this is nice when indexing with both string or int keys.

Its a little complex to understand at first read and this kind of code often can hide bugs because its so complicated but the tests definitely help. Can you fix the non zero sequential int keys issue I found and then I think we can merge it. 🙌

@kiendang
Copy link
Contributor Author

kiendang commented Oct 23, 2023

@madhavajay Yah keys should not be int. If you have a d: DictTuple with int keys, now d[2] is ambiguous since it means both "getting the 3rd element" and "getting the element with key 2". I'll just put in a check, raising an error on attempting to create a DictTuple with int key.

You would encounter similar error with the current TupleDict too.

p = TupleDict({1: "z", 2: "b"})
p[2]

File "/Users/kien/pysyft/packages/syft/src/syft/types/tupledict.py", line 14, in __getitem__
    return list(self.values())[key]
           ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: list index out of range

the tests definitely help

I have a more comprehensive test suite for this but is currently blocked by #8188 (without #8188 it is very slow). Will push that once it is resolved.

@kiendang kiendang force-pushed the dataset-index-2 branch 3 times, most recently from bbee1bb to 573a6f9 Compare October 27, 2023 05:10
@madhavajay madhavajay merged commit 348385b into OpenMined:dev Oct 30, 2023
26 of 27 checks passed
@kiendang kiendang deleted the dataset-index-2 branch October 30, 2023 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants