Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timestamps produced by broker::format::json::v1::encode are TZ dependent #434

Open
awelzel opened this issue Nov 14, 2024 · 2 comments
Open

Comments

@awelzel
Copy link
Contributor

awelzel commented Nov 14, 2024

This is related to #346 , but I wonder if for now we should enforce UTC?

The following snippet encodes the 0 timestamp, which IMO should lead to 1970-01-01 00:00.000 regardless of the environment.

#include <broker/format/json.hh>
#include <chrono>
#include <string>

int main() {
    std::string s;
    const auto inserter = std::back_inserter(s);
    broker::format::json::v1::encode(broker::timestamp{std::chrono::seconds::zero()}, inserter);
    std::fprintf(stderr, "%s\n", s.c_str());
}

Currently, the output of this program depends on the setting of TZ.

$ ./test 
{"@data-type":"timestamp","data":"1970-01-01T01:00:00.000"}
$ TZ=UTC ./test
{"@data-type":"timestamp","data":"1970-01-01T00:00:00.000"}
$ TZ=Asia/Tokyo ./test
{"@data-type":"timestamp","data":"1970-01-01T09:00:00.000"}
$ TZ=America/Los_Angeles ./test
{"@data-type":"timestamp","data":"1969-12-31T16:00:00.000"}

This means that two instances of broker with different TZ settings in their environment will not agree on the timestamps.

Before #346 is fixed, should we make sure to always use UTC instead of localtime?

--- a/libbroker/broker/format/json.cc
+++ b/libbroker/broker/format/json.cc
@@ -31,7 +31,7 @@ size_t encode_to_buf(timestamp value, std::array<char, 32>& buf) {
 #ifdef _MSC_VER
   localtime_s(&time_buf, &secs);
 #else
-  localtime_r(&secs, &time_buf);
+  gmtime_r(&secs, &time_buf);
 #endif
   buf[0] = '"';
   auto pos = strftime(buf.data() + 1, buf.size() - 1, "%FT%T", &time_buf) + 1;
@Neverlord
Copy link
Member

We held off on adding the timezone (see discussion in #360) because it would be a breaking change for clients. Using UTC would keep the format stable, but it would change the semantics. So the question is whether clients have relied on local time or not.

Are there assumptions on the time zone in the Go binding, @simeonmiteff?

@ckreibich: FWIW, I think falling back to UTC until we have proper time zones would be a reasonable change, but I don't know whether people are relying on the current local time or not. Any opinion on that?

@awelzel
Copy link
Contributor Author

awelzel commented Nov 19, 2024

FWIW, I think falling back to UTC until we have proper time zones would be a reasonable change,

Today it's environmental based which seems really bad :-)

awelzel added a commit to zeek/zeek that referenced this issue Nov 22, 2024
Broker's JSON serialization is TZ dependent (which seems a bug). For now
do the same as we do in btest.cfg and run doctests with TZ set to UTC.

Reported in zeek/broker#434.
awelzel added a commit to zeek/zeek that referenced this issue Nov 22, 2024
Broker's JSON serialization is TZ dependent (which seems a bug). For now
do the same as we do in btest.cfg and run doctests with TZ set to UTC.

Reported in zeek/broker#434.
awelzel added a commit to zeek/zeek that referenced this issue Nov 26, 2024
Broker's JSON serialization is TZ dependent (which seems a bug). For now
do the same as we do in btest.cfg and run doctests with TZ set to UTC.

Reported in zeek/broker#434.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants