Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expand_households fails if zone IDs are floats #79

Open
bstabler opened this issue Jul 11, 2019 · 8 comments
Open

expand_households fails if zone IDs are floats #79

bstabler opened this issue Jul 11, 2019 · 8 comments

Comments

@bstabler
Copy link
Contributor

bstabler commented Jul 11, 2019

In the example below, the zone IDs in the control data file are stored as floats even though they are whole numbers (i.e they are 5.0 instead of 5 for example). This is fine for some of the steps, but is a problem for expand_households, specifically this line. We should fix this by maybe checking for this on input and/or cast the zone IDs to int(). I confirmed expand_households works if I remove all ".0" from the file.

image

@toliwaga
Copy link
Contributor

How did this csv file end up with floats in it - was it created by populationsim? (If so, then we should fix that bug?)

Or was it user pilot error? (In which case, there are probably lots of places that could happen and we should talk about whether this is an eventuality that we should allow generally - because it doesn't make sense to just fix it in one place and not otehrs...)

@bstabler
Copy link
Contributor Author

This is a user created input file. I think we should check for this issue at the beginning when we read in inputs.

@toliwaga
Copy link
Contributor

For every field in every file that ought to have integers? Do we check to make sure it was a float int and not a float that will be rounded/truncated (in which case maybe it is an error?)

Just wondering why we are fixing this particular random bad input?

@bstabler
Copy link
Contributor Author

We don't have to fix bad data, but I think we should catch bad data / do more error checking / add more informative error messages / etc. It took me quite some time to figure this out and I'm pretty familiar with PopulationSim.

@toliwaga
Copy link
Contributor

Might as well address it opportunistically then - but we should throw an error if the values change when coerced to ints.

@bstabler
Copy link
Contributor Author

sounds good

@bstabler
Copy link
Contributor Author

This is also true for the geo crosswalk file as well - the data needs to be ints. We should fix this.

@nick-fournier-rsg
Copy link

late follow on.... but I have a fix on a fork. https://github.com/nick-fournier-rsg/populationsim/tree/v0.6.0

very long integers (e.g. full BG ids) resolved to NA or other erroneous values. My solution was to explicitly use np.int64 instead of base int which I think defaults to int32 or maybe something smaller.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants