Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix data oob notebook #636

Merged
merged 43 commits into from
Jan 15, 2025
Merged

Fix data oob notebook #636

merged 43 commits into from
Jan 15, 2025

Conversation

mdbenito
Copy link
Collaborator

@mdbenito mdbenito commented Jan 15, 2025 •
edited
Loading

Description

This PR closes #619 and includes a bunch of fixes found along the way

Changes

  • Fixes DataOOBValuation to only accept bagged models
  • Corrects the analysis in data_oob.ipynb to account for data imbalance.
  • Adds documentation about Data-OOB
  • Includes a method run_removal_experiment to quickly run this experiment
  • Fixes some types
  • Includes variances and counts in valuation results
  • Fixes a warning message in valuation result about data conversion

Checklist

  • Wrote Unit tests (if necessary)
  • Updated Documentation (if necessary)
  • Updated Changelog
  • If notebooks were added/changed, added boilerplate cells are tagged with "tags": ["hide"] or "tags": ["hide-input"]

schroedk and others added 30 commits September 16, 2024 12:11
@schroedk
WIP: data-oob adaptation to pydvl.valuation
aa6f4fa
@mdbenito
Fix warning message
fc0f16d
@mdbenito
Missing type
d76dd02
@mdbenito
OOB notebook: fix text, update code to new api, use joblib
37dba9d
@mdbenito
Move legacy code outside core dist
aae6b3b
@mdbenito
Add deletion note
ce42b0c
@mdbenito
Minor text tweaks to nb
6ae19eb
@mdbenito
Fix warning message
3555960
@mdbenito
Export both variances and counts to dataframe
7266c03
@mdbenito
Add BaggingModel type and predicate
c037e5a
@mdbenito
Revert typo leading to bogus stderr computation
2bdbea2
@mdbenito
Simplify type check and improve tests
9d0ff80
@mdbenito
Remove support for implicit training of bagging models in data_oob
b754227
@mdbenito
Missing type
2d9ce58
@mdbenito
Fix plotting for oob
d49f6a3
@mdbenito
Fix naming of OOB scoring function
01628ff
@mdbenito
Ensure sorting of values prior to plotting
1b3794f
@mdbenito
Require sklearn >= 1.3 for TargetEncoder
611ae47
@mdbenito
Update tests for ValuationResult
cfcd148
@mdbenito
WIP
f53e8a8
@mdbenito
WIP2
df5023e
@mdbenito
More requirement to correct file
6a0fd35
@mdbenito
Merge develop into feature/619-oob-valuation
fcf504c
@mdbenito
Fixes for new dataset interface
cda1f99
@mdbenito
Merge branch 'feature/619-oob-valuation' into feature/619/temp
9f82e8e 
# Conflicts:
#	notebooks/data_oob.ipynb
#	requirements-notebooks.txt
@mdbenito
Fixes for new dataset interface
59ddcc6
@mdbenito
Import future annotations
fa0db1b
@mdbenito
Fix dtype of names in results
f8994f3
@mdbenito
Comments
5ec3a76
@mdbenito
Docstrings
c6648f3
@mdbenito
Copy link
Collaborator Author

@schroedk Since you probably don't have time for this, I'll merge right away and tag you here for reference, in case you want to look at it.

@mdbenito mdbenito merged commit bf380bd into develop Jan 15, 2025
39 checks passed
@mdbenito mdbenito deleted the feature/619-oob-valuation branch January 15, 2025 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Adapt data_oob.ipynb to use new package pydvl.valuation
2 participants
@mdbenito @schroedk