Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

foreach statement to iterate in parallel over arrays #997

Open
matteo-cristino opened this issue Jan 8, 2025 · 7 comments · May be fixed by #1002
Open

foreach statement to iterate in parallel over arrays #997

matteo-cristino opened this issue Jan 8, 2025 · 7 comments · May be fixed by #1002

Comments

@matteo-cristino
Copy link
Collaborator

Since till v4 nested foreach ran in parallel over the arrays we should introduce a new statement to do it also in v5, so that old scripts that used it can be updated to use v5 (anyway I do not think that this feature was used much, but it can turn usefull in the future maybe). Possible idea:

Foreach 'touple' in zip 'list_of_array'

where list_of_array is a list containing the names of the arrays that are in input to the zip and touple will be an array containg the values of the arrays in the same position.

Moreover the loop execution should stop as soon as the on of the arrays ends.

Open question: maybe zip is too much of a programmer's term?

P.S. this is not urgent since a workaround for this can already be done using zencode (find the length of the shorter array, loop form 1 to that length and extract each element in that position from all the arrays), this is simply a utility that I think we can easily implement.

@jaromil
Copy link
Member

jaromil commented Jan 8, 2025

Zip is nice for us programmers, but confusing for anyone else. However here helps to define the function:

ZipWith is when you take two input sequences, and produce an output sequence in which every two elements from input sequences at the same position are combined using some function. An example input zipwith (+) [1, 2, 3] [4, 5, 6] and output [5, 7, 9]

ZipWith applies a function to two arguments on each iteration and inserts the output into a new array.

OTOH zip specifically refers to combining elements as tuples (could be used to create dictionaries from arrays).

If I understand well the problem here, we define the function inside the foreach branch (a series of zencode statements) and it will be always passed a "touple" so two arguments from two distinct arrays. I think then two is a low enough number to enumerate these two arrays in the zencode statement rather than inside a pointer-array.

1. Foreach 'values' at same position in arrays '' and ''
2. Foreach 'values' at same position in arrays named in ''

in which 1. operates on two input arrays only (classic zip) and 2. allows a variable number of arrays.

The branch-scoped dictionary values will be then a dictionary of this form:
values.$array_name = $array_value_at_pos
updated at each iteration.

Does that sounds OK?

@matteo-cristino
Copy link
Collaborator Author

Trying to implement this I found a problem. If we save the array values in values array under the array name I have no way to extract them:

  • using pickup from path 'values.$array_name' the name of new variable collide with the array name
  • using copy '$array_name' from 'values' to 'new_var' the $copt_name is parsed with a mayhave thus zenroom correctly error saying it is not a string.

What about values being an array in which the order follow the order of arrays in input?

@matteo-cristino
Copy link
Collaborator Author

This thing is not so easy after all:

  • use of CACHE to store 'values' array can not be done since a lot of when statement (like copy ones) use also the CODEC of the element (that cache does not have)
  • as a workaround I tried to create a schema on the fly to create the right CODEC for values, but I'm not sure this can be the right solution, I have not found still an example, but simple objects extracted from 'values' (like with a copy statement) will lose their original CODEC (will be the complex one with the schema created for values) and this can lead to maybe some erroneuos stuff

Moreover this approach will requires a lot of statements to handle the copy to an object and the remove of it, e.g.

Foreach 'values' at the same position in arrays named in 'arrays'
    When I copy '1' from 'values' to 'sign'
    When I copy '2' from 'values' to 'msg'
    If I verify 'msg' has a ecdh signature in 'sign' by 'pk'
        When I copy '3' from 'values' to 'temp'
        and I move 'temp' in 'res'
    EndIf
    When I remove 'sign'
    When I remove 'msg'
EndForeach

After this I got an idea, what if we take a different approach on zip? I mean since we do not have tuples, we can save all parallel values in variables which name can start with a name that you set in the sentence and then a suffix that is the name of the array, in this way original CODEC can be mantained and the creation and deletion of this elements can be done internally in lua and the user will have less to worry about. For example the above example can be

Foreach values at the same position in arrays named in 'arrays' saved with prefix 'prefix'
    If I verify 'prefixmsg' has a ecdh signature in 'prefixsign' by 'pk'
        When I move 'prefixres' in 'res'
    EndIf
EndForeach

But the question is always the same, is this readable? 📎

@jaromil
Copy link
Member

jaromil commented Jan 12, 2025

I think yes if we move prefix near 'create' and remove 'named in' which is implicit without 'and'

Foreach values prefix '' at same position in ''
    If I verify 'prefixmsg' has a ecdh signature in 'prefixsign' by 'pk'
        When I move 'prefixres' in 'res'
    EndIf
EndForeach

also I tried to remove array or dictionary here wondering if we can elide its need and autodetect.

@jaromil
Copy link
Member

jaromil commented Jan 12, 2025

Additional idea connected to this PR and also generally useful: implement a simple parser for polynomial math using the values in array, will make it very handy to derive columns like in a spreadsheet.

@matteo-cristino
Copy link
Collaborator Author

Regarding

also I tried to remove array or dictionary here wondering if we can elide its need and autodetect.

at the moment we do not support for loop over dictionaries, only arrays.

Instead with this

Additional idea connected to this PR and also generally useful: implement a simple parser for polynomial math using the values in array, will make it very handy to derive columns like in a spreadsheet.

did you meant something like https://github.com/dyne/Zenroom/blob/master/src/lua/zencode_when.lua#L723 but that takes in input an array? Can I propose to also create the file zencode_math.lua to include all math operationss?

@matteo-cristino matteo-cristino linked a pull request Jan 13, 2025 that will close this issue
@jaromil
Copy link
Member

jaromil commented Jan 16, 2025

OK. no I did not meant that,but anyway lets merge this in person next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants