Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: YOLO Coders is incorrect #3852

Open
zanieb opened this issue Aug 5, 2024 · 3 comments
Open

Bug: YOLO Coders is incorrect #3852

zanieb opened this issue Aug 5, 2024 · 3 comments
Assignees
Labels
🐛 bug Something isn't working 👀 needs triage

Comments

@zanieb
Copy link

zanieb commented Aug 5, 2024

Describe the bug

e.g. at https://app.opensauced.pizza/s/astral-sh/uv?hideBots=false

We have reports of people pushing to main:

Screenshot 2024-08-05 at 12 07 28 PM

But this is just a squash merged pull request:

Screenshot 2024-08-05 at 12 08 03 PM Screenshot 2024-08-05 at 12 09 05 PM

Steps to reproduce

See description above.

@zanieb zanieb added 🐛 bug Something isn't working 👀 needs triage labels Aug 5, 2024
Copy link
Contributor

github-actions bot commented Aug 5, 2024

Thanks for the issue, our team will look into it as soon as possible! If you would like to work on this issue, please wait for us to decide if it's ready. The issue will be ready to work on once we remove the "needs triage" label.

To claim an issue that does not have the "needs triage" label, please leave a comment that says ".take". If you have any questions, please comment on this issue.

For full info on how to contribute, please check out our contributors guide.

@jpmcb
Copy link
Member

jpmcb commented Aug 5, 2024

👀 thanks for reporting this! I'll look into this right away!

@jpmcb jpmcb self-assigned this Aug 5, 2024
@jpmcb
Copy link
Member

jpmcb commented Aug 5, 2024

It seems we're missing abit of data in our events data lake: how this works is we consume the GitHub events firehose off of api.github.com/events at scale, drop that into a time series database, and use that to correlate closed PRs with merge commit shas. The piece that's missing it seems is the closed event that we'd use to correlate the 4bc36c0cb85ac2b5efbc9c2bfecad6137666e908 sha on astral-sh/uv's main branch with the closed PR event's pr_merge_commit_sha in the data.

It seems we're missing data from ~July 8th, ~July 16th, and July 17th.

There aren't any disruptions on our side when consuming the events firehose. But, my best guess, it seems that there were GitHub disruptions those days around the time those PRs were merged:

We've been aware that at times the GitHub API can be unstable but this is the first I've seen it drop this much data. We'll have to think of some kind of fallback mechanism for how we want to approach this in our API and services.

This was super helpful @zanieb - thanks so much for surfacing this!


cc @isabensusan, until we narrow down a better fix for this, we should considered updating the copy in the Yolo coders section to note that this data may need to be manually verified

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working 👀 needs triage
Projects
None yet
Development

No branches or pull requests

2 participants