Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change data selection method #100

Open
guidocioni opened this issue Mar 22, 2024 · 1 comment
Open

Change data selection method #100

guidocioni opened this issue Mar 22, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@guidocioni
Copy link
Owner

When we have multiple columns with the same name, e.g. precipitation, precipitation_member1 for ensemble or precipitation, precipitation_icon for deterministic, we always use the logic data.loc[:, data.columns.str.contains('precipitation')] to select the relevant columns.

However this is prone to errors in case there are other columns that contain that same string. I think we should change this to something more robust like maybe a regex expression on the columns names?

@guidocioni guidocioni added the bug Something isn't working label Mar 22, 2024
@guidocioni guidocioni self-assigned this Mar 22, 2024
@guidocioni
Copy link
Owner Author

guidocioni commented Aug 19, 2024

For the ensemble now I'm using a more stringent regex selection method, as this contains method was causing some issues as shown in #157.

The fix is in a56dab6

For example if var='rain', then we can write

columns_regex = rf'{var}$|{var}_member(0[1-9]|[1-9][0-9])$'

and filter the columns from the dataframe as

df.loc[:, df.columns.str.match(columns_regex)]

This should not cause issues but is a little bit verbose... Would be cool to maybe pack it into a function?

Also we need to implement a similar logic for the other pages that use contains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant