Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handling of delimiters different from ";" #27

Merged
merged 4 commits into from
Mar 10, 2024
Merged

handling of delimiters different from ";" #27

merged 4 commits into from
Mar 10, 2024

Conversation

simei94
Copy link
Contributor

@simei94 simei94 commented Feb 19, 2024

for non german scenarios, the standard delimiter is not ";". This is an approach to handle this while auto-calibrating, more specifically, when analyzing the modal split of the current calib run. RIght now, the auto calib run crashes when trying to read in trips.csv and persons.csv when the delimiter is different to ";"

@simei94 simei94 requested a review from rakow February 19, 2024 14:36
@rakow
Copy link
Collaborator

rakow commented Feb 19, 2024

It looks like it will read the whole csv file to then read it again with a different delimiter. This is very inefficient. The docs say that pandas can handle sep=None and detect it automatically, but only with the slower python engine. Anyway it should be very easy to detect by only reading the first line of a file.

@simei94 simei94 merged commit 04cf940 into master Mar 10, 2024
1 check passed
@simei94 simei94 deleted the delimiter_fix branch March 10, 2024 18:30
@rakow
Copy link
Collaborator

rakow commented Mar 11, 2024

Have you tested that this works with ";" and "," delimiters? The docs read like that one has to specially define sep=None, otherwise "," will be the default. I have changed this now on the main branch.

@simei94
Copy link
Contributor Author

simei94 commented Mar 11, 2024

you are right, thanks for adding it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants