Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement EfficientNet #171

Merged
merged 16 commits into from
Jun 25, 2022
Merged

Implement EfficientNet #171

merged 16 commits into from
Jun 25, 2022

Conversation

darsnack
Copy link
Member

Follow-up from #106 (closes #106).

@darsnack darsnack mentioned this pull request Jun 19, 2022
46 tasks
@darsnack darsnack requested a review from theabhirath June 19, 2022 03:42
@theabhirath
Copy link
Member

Is there also a way we can port the weights functionality from #113? Would be a good complement to #164

@darsnack
Copy link
Member Author

Probably using the same technique as #164. I think any model that is plain convolutions, batch norm, and fully connected can be done in this way. There a gist linked on the HF repo model cards if you want to tackle adding this for other models in the repo.

@darsnack darsnack closed this Jun 19, 2022
@darsnack darsnack reopened this Jun 19, 2022
@darsnack darsnack requested a review from theabhirath June 22, 2022 16:05
@darsnack
Copy link
Member Author

Test errors are happening later for the same memory issues as always. I've already limited the tests to the smallest subset that seems necessary.

@theabhirath
Copy link
Member

Seems all good, but I'm worried that EfficientNets, which are the least heavy of the models, are causing this....or maybe it's a cumulative effect from the other models (Is GC.gc() perfect?)

@darsnack
Copy link
Member Author

The tests get passed EfficientNet and error around ConvNeXt. Maybe ConvNeXt is right on the threshold for the CI?

@theabhirath
Copy link
Member

Might be the MLPMixers, since those are quite large (and we are testing for the :large models there). But just a hunch

@theabhirath
Copy link
Member

Seems like it was the MLPMixers after all, nice!

test/convnets.jl Outdated Show resolved Hide resolved
@theabhirath
Copy link
Member

Could maybe try not testing for the ConvNeXt and ConvMixer large models, but I'm not sure if those are the only pain points for memory here - Xception and InceptionResNetv2 were pretty bad as well

@darsnack darsnack requested a review from ToucheSir June 24, 2022 10:08
@darsnack darsnack merged commit 099c1a5 into FluxML:master Jun 25, 2022
@darsnack darsnack deleted the efficient-net branch June 25, 2022 08:58
This was referenced Jun 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add EfficientNet with pretrained weights
3 participants