Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fakedata issue with test user rating deltas - generate them via cumulative sum instead #262

Open
jauggy opened this issue Apr 14, 2024 · 3 comments
Labels
feature request Not broken but something we want added good first issue Good for newcomers low priority

Comments

@jauggy
Copy link
Member

jauggy commented Apr 14, 2024

Here's what it looks like in production (which is correct)
1

my skill went from 22.13 to 23.0 which is a gain of 0.87

But in my dev environment (wrong deltas; ignore the lack of icons which is due to my fontawesome)
2

My skill went from 26.29 to 25.13 which is loss of 1.16 but it shows a gain of 1.18

The deltas are also wrong for both skill, uncertainty, and leaderboard rating. I noticed this bug when I was checking Exeler's last PR to add uncertainty to this page. But then I went back to an older state of master and this bug is still there; so the bug is not from his commit. Something wrong with fake data creation perhaps?

@jauggy
Copy link
Member Author

jauggy commented Apr 14, 2024

@Jordan-Cottle can you confirm that you're also getting mathematically incorrect deltas on the match ratings page on your local teiserver?

@Jordan-Cottle
Copy link
Contributor

Jordan-Cottle commented Apr 14, 2024

I noticed that some of the test data for the user did have incorrect deltas. The data in this table comes straight from the database tables, so it seems entirely reasonable that the pre-populated data from the script could just have stuff hard coded that isn't possible to have happen in prod with real data.

Example data from the database:
image

Notice it's just json stored in the value column, so really any data could be put in there with no correctness guarantee

@StanczakDominik
Copy link
Collaborator

Yeah, the fake data creation iirc basically uses random uncorrelated numbers for both the deltas and the ratings! So they don't make sense. Good find.

One way to improve this would be to use random numbers for the deltas only, and generate the ratings themselves with a cumulative sum over the deltas.

@StanczakDominik StanczakDominik changed the title The deltas for test users seem wrong in matches/ratings page fakedata issue with test user rating deltas - generate them via cumulative sum instead Jun 9, 2024
@StanczakDominik StanczakDominik added the good first issue Good for newcomers label Jun 9, 2024
@L-e-x-o-n L-e-x-o-n added feature request Not broken but something we want added low priority labels Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Not broken but something we want added good first issue Good for newcomers low priority
Projects
None yet
Development

No branches or pull requests

4 participants