Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timestamps with timezones #80

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

Narnach
Copy link

@Narnach Narnach commented Jan 20, 2022

What problem does this address?

Using fast_excel to write Time or DateTime to XLSX corrupts the result when the timezone is not +00:00.

While exposing the issue, this was the result of converting various input values via FastExcel::WorksheetExt#write_value:

Equivalent                 Check                          Input                         Result
√                DateTime +00:00      2017-03-03T03:03:03+00:00      2017-03-03T03:03:03+00:00
x                DateTime +01:00      2017-01-01T01:01:01+01:00      2017-01-01T01:01:01+00:00
x                   DateTime CET      2017-01-01T01:01:01+01:00      2017-01-01T01:01:01+00:00
x                  DateTime CEST      2017-07-07T07:07:07+02:00      2017-07-07T07:07:07+00:00
√                    Time +00:00      2017-03-03 03:03:03 +0000      2017-03-03T03:03:03+00:00
x                    Time +01:00      2017-01-01 01:01:01 +0100      2017-01-01T00:01:01+00:00
x                    Time +02:00      2017-07-07 07:07:07 +0200      2017-07-07T05:07:07+00:00
√                           Date                     2017-03-01                     2017-03-01

Only the +00:00 offsets work as expected, which are the only use cases originally tested.

What have I done to solve this?

I've expanded the test cases to expose the problem and then applied a fix that ensures DateTime and Time use non-destructive calls to convert timestamps to be in +00:00 before writing them to XLSX.

I've tested it locally on Ruby 2.7.5. If older Rubies break, CI should show us.

Refactoring

Because of the (now) large number of test cases that duplicate the logic for writing a value to XLSX and reading it back, I've added a helper method to the test file to encapsulate this. The test cases are now much easier to read.

* Test names better reflect intentions
* Test variable naming is consistent
* Corrected flipped actual and expected values

I'm adding more tests and prefer they are all consistent. Afterwards
there is room for another refactoring to clean up some duplicated
wrapper code.
Previous test cases only covered +00:00 as timezone. Adding test cases
for +01:00 resulted in data getting returned with the "same" timestamp,
but in the wrong timezone (+00:00). This means you lost an hour in the
conversion.

Because the Excel timestamp format does not seem to store timezones, we
might as well convert everything to +00:00 / UTC and do a proper
conversion.

Because Date/DateTime and Time have different APIs, they use their own
method to convert to +00:00.
The process of converting a value via XLSX was duplicated across all
tests, obfuscating their intent. Extracting a helper method makes the
code easier to understand.
This updates test names to distinguish between offsets (i.e. +01:00) and time zones (i.e. CET).
It adds tests for CET and CEST, to distinguish between summer/winter time / daylight saving time.
@Paxa
Copy link
Owner

Paxa commented Dec 29, 2023

I think excel doesn't know about timezones, and internally it stores dates as number of days since 1900. And excel file is usually for people to see, so it should display local time.

When we write date or time, this library will call FastExcel.date_num(...)

def self.date_num(time, offset = nil)

And inside date_num it will try to read zone offset of parameter or Time.zone (for rails), if none then use UTC.

I think it's correct behaviour, if we call SpecialEvent.all() and print it in logs or in HTML, the dates will be in local timezone (zone of application or Time.zone inside rails http request), same time also shown in excel file

If you want always UTC then do something like

worksheet.write_number(1, 1, FastExcel.date_num(value, 0), nil) # 0 = offset in seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants