Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add output lengths for layers #491

Merged
merged 4 commits into from
Feb 23, 2024
Merged

Add output lengths for layers #491

merged 4 commits into from
Feb 23, 2024

Conversation

SebastianM-C
Copy link
Contributor

@SebastianM-C SebastianM-C commented Feb 15, 2024

This PR implements the inputsize and outputsize interface defined in LuxCore in LuxDL/LuxCore.jl#19

I'm not sure if defined this correctly for ReshapeLayer and Scale.

Also, I'm not sure if the functions can be defined for some of the layers.

src/layers/basic.jl Outdated Show resolved Hide resolved
@avik-pal
Copy link
Member

Can you also add them to the LuxCore API docs?

@avik-pal
Copy link
Member

  • Bump the lowerbound for LuxCore

Copy link

codecov bot commented Feb 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.85%. Comparing base (42aa7db) to head (7279bdd).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #491      +/-   ##
==========================================
- Coverage   88.61%   87.85%   -0.77%     
==========================================
  Files          29       29              
  Lines        1625     1630       +5     
==========================================
- Hits         1440     1432       -8     
- Misses        185      198      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines 191 to 185
@test LuxCore.inputsize(layer) == (10, 5)
@test LuxCore.outputsize(layer) == (10, 5)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@avik-pal is this ok? It seems a bit inconsistent with the size checks above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think outputsize is fine. Do we really need to be able to define inputsize for Symbolics? I feel we should return a set of possible sizes for the input.

Copy link
Contributor Author

@SebastianM-C SebastianM-C Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. I defined it only for consistency, but I'm not sure where it would be useful.

Would that be a tuple of tuples?

@@ -9,6 +9,7 @@
x = randn(rng, 6, 3) |> aType

@test size(layer(x, ps, st)[1]) == (2, 3, 3)
@test Lux.outputsize(layer) == (2, 3)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure if the definition of outputsize makes sense for ReshapeLayer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about the fact that the size of the output in the test above (and also in general) has a different size compared to outputsize because the input also matters and we don't know it from the layer definition.

@SebastianM-C
Copy link
Contributor Author

Should I just implement outputsize in this PR? I think inputsize is harder to define and I'm not sure if it's even a good idea in the first place. If we think about it in terms of all possible combinations, I'm afraid it could lead to combinatorial explosion (for ReshapeLayer you'd basically have to invert reshape for arbitrary inputs). But the main thing is that it's not needed for anything (that I know of).

@avik-pal
Copy link
Member

avik-pal commented Feb 20, 2024

Yes let's restrict the current PR to inputsize outputsize

@SebastianM-C
Copy link
Contributor Author

Yes let's restrict the current PR to inputsize

You mean outputsize?

@SebastianM-C SebastianM-C changed the title Add input and output lengths for layers Add output lengths for layers Feb 20, 2024
@SebastianM-C
Copy link
Contributor Author

@avik-pal can you approve ci?

@SebastianM-C
Copy link
Contributor Author

I rebased on master and fixed formatting. I'm not sure why codecov failed.

@SebastianM-C
Copy link
Contributor Author

@avik-pal Looks like CI passes, the codecov part at the end of the tests is unrelated and seems to randomly fail.
Is there anything left to address here?

@avik-pal avik-pal merged commit 5f40fe9 into LuxDL:main Feb 23, 2024
10 of 12 checks passed
@SebastianM-C SebastianM-C deleted the io_length branch February 23, 2024 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants