Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't post/get NDJSON with string key called "ts" (regression at dcce5c2) #2740

Closed
philrz opened this issue May 19, 2021 · 1 comment · Fixed by #2745
Closed

Can't post/get NDJSON with string key called "ts" (regression at dcce5c2) #2740

philrz opened this issue May 19, 2021 · 1 comment · Fixed by #2745
Assignees
Labels
bug Something isn't working
Milestone

Comments

@philrz
Copy link
Contributor

philrz commented May 19, 2021

Repro is with Zed commit dcce5c2 which is associated with #2729 (cc: @mccanne).

At Zed commit 7c92f41 that came right before, the following was working. First I start a zed lake serve:

$ zed -version
Version: v0.29.0-340-g7c92f41f

$ zed lake serve
{"level":"info","ts":1621453519.363533,"msg":"Open files limit raised","limit":24576}
{"level":"info","ts":1621453519.365165,"logger":"core","msg":"Started"}
{"level":"info","ts":1621453519.365385,"logger":"httpd","msg":"Listening","addr":"[::]:9867"}

Then I create a Pool and post sample data https://github.com/brimdata/brim/blob/main/itest/testdata/sample.ndjson to it.

$ zapi new foo
foo: pool created

$ zapi post -p foo sample.ndjson
posted 13.01KB in 6.883174ms

$ zapi get -p foo -z
{_path:"capture_loss",_write_ts:"2020-02-25T16:03:17.838527Z",ts:"2020-02-25T16:03:17.838527Z",ts_delta:11.854892015457153,peer:"zeek",gaps:0,acks:6,percent_lost:0.}
...

However, advancing to Zed commit dcce5c2, now the get returns nothing.

$ zed -version
Version: v0.29.0-341-gdcce5c22

$ zed lake serve
{"level":"info","ts":1621453687.019802,"msg":"Open files limit raised","limit":24576}
{"level":"info","ts":1621453687.021408,"logger":"core","msg":"Started"}
{"level":"info","ts":1621453687.021627,"logger":"httpd","msg":"Listening","addr":"[::]:9867"}
$ zapi new foo
foo: pool created

$ zapi post -p foo sample.ndjson
posted 13.01KB in 7.48606ms

$ zapi get -p foo -z
[no output]

$ echo $?
0

Back in the zed lake serve window, I see no crash or other indications of a problem.

...
{"level":"info","ts":1621453709.27301,"logger":"http.access","msg":"Request completed","request_id":"1slijHS76xjG8e2Vw60KPdScLZq","host":"localhost:9867","method":"POST","proto":"HTTP/1.1","remote_addr":"[::1]:54870","request_content_length":64,"url":"/pool","elapsed":0.001180324,"response_content_length":50,"status_code":200}
{"level":"info","ts":1621453713.043034,"logger":"http.access","msg":"Request completed","request_id":"1slijhFYrx5qoM3CQolHHf0xqqn","host":"localhost:9867","method":"GET","proto":"HTTP/1.1","remote_addr":"[::1]:54874","request_content_length":0,"url":"/pool","elapsed":0.000018305,"response_content_length":52,"status_code":200}
{"level":"info","ts":1621453713.044298,"logger":"http.access","msg":"Request completed","request_id":"1slijlyVCC2bVU2Y17dhklzska5","host":"localhost:9867","method":"GET","proto":"HTTP/1.1","remote_addr":"[::1]:54875","request_content_length":0,"url":"/pool","elapsed":0.000009788,"response_content_length":52,"status_code":200}
{"level":"info","ts":1621453713.052016,"logger":"http.access","msg":"Request completed","request_id":"1slijinDTY0EHWuNyeUwZX4iSRP","host":"localhost:9867","method":"POST","proto":"HTTP/1.1","remote_addr":"[::1]:54875","request_content_length":-1,"url":"/pool/1slijAiDMSPfSrirOCjVo0mwP1A/log","elapsed":0.007054416,"response_content_length":101,"status_code":200}
{"level":"info","ts":1621453717.537225,"logger":"http.access","msg":"Request completed","request_id":"1slikBaW6Tw1dpmG7d2TMNpxWmT","host":"localhost:9867","method":"GET","proto":"HTTP/1.1","remote_addr":"[::1]:54880","request_content_length":0,"url":"/pool","elapsed":0.000011747,"response_content_length":52,"status_code":200}
{"level":"info","ts":1621453717.541022,"logger":"http.access","msg":"Request completed","request_id":"1slikEexnyHsXv6Ch4ikvQaDRHM","host":"localhost:9867","method":"POST","proto":"HTTP/1.1","remote_addr":"[::1]:54880","request_content_length":246,"url":"/search?format=zng","elapsed":0.002319086,"response_content_length":312,"status_code":200}
@philrz philrz added the bug Something isn't working label May 19, 2021
@philrz philrz added this to the Data MVP0 milestone May 19, 2021
@nwt nwt closed this as completed in #2745 May 20, 2021
brim-bot pushed a commit to brimdata/brimcap that referenced this issue May 20, 2021
…o span" by nwt

This is an auto-generated commit with a Zed dependency update. The Zed PR
brimdata/super#2745, authored by @nwt,
has been merged.

service/search: ignore request span if pool has no span

Closes brimdata/super#2740.
brim-bot pushed a commit to brimdata/brimcap that referenced this issue May 20, 2021
…o span" by nwt

This is an auto-generated commit with a Zed dependency update. The Zed PR
brimdata/super#2745, authored by @nwt,
has been merged.

service/search: ignore request span if pool has no span

Closes brimdata/super#2740.
brim-bot pushed a commit to brimdata/zui that referenced this issue May 20, 2021
…o span" by nwt

This is an auto-generated commit with a Zed dependency update. The Zed PR
brimdata/super#2745, authored by @nwt,
has been merged.

service/search: ignore request span if pool has no span

Closes brimdata/super#2740.
@philrz
Copy link
Contributor Author

philrz commented May 20, 2021

Verified in Zed commit 035b439.

$ zed -version
Version: v0.29.0-345-g035b439b

$ zed lake serve
{"level":"info","ts":1621526120.395707,"msg":"Open files limit raised","limit":24576}
{"level":"info","ts":1621526120.3973398,"logger":"core","msg":"Started"}
{"level":"info","ts":1621526120.397559,"logger":"httpd","msg":"Listening","addr":"[::]:9867"}
$ zapi new foo
foo: pool created

$ zapi post -p foo sample.ndjson
posted 13.01KB in 8.338362ms

$ zapi get -p foo -z
{_path:"capture_loss",_write_ts:"2020-02-25T16:03:17.838527Z",ts:"2020-02-25T16:03:17.838527Z",ts_delta:11.854892015457153,peer:"zeek",gaps:0,acks:6,percent_lost:0.}
...

This also allowed the Brim CI to pass such that its Zed pointer has now successfully advanced. This issue did cause us to notice some unexpected request behaviors in Brim in brimdata/zui#1656 before we close that one, but the primary issue has definitely been addressed.

Thanks @nwt!

@philrz philrz changed the title Can't post/get NDJSON with string key called "ts" (regression at dcce5c22) Can't post/get NDJSON with string key called "ts" (regression at dcce5c2) May 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants