Fix sunrise/sunset scatter plots #16
Conversation
Sunrise/sunset scatter plots are misplaced for * Date ranges that are a subset of the entire DB date range * Species whose first/last detections are inside the bounds of the current date range. This fixes those problems.
|
This is an interesting discovery. Not one that I had considered before. I must admit, I don't go into Species Stats much and tend to let that part of it run itself, however, on investigation, the logic is sound. I would have missed that problem. |
scripts/plotly_streamlit.py
Outdated
| latitude = df2['Lat'][0] | ||
| longitude = df2['Lon'][0] | ||
| def sunrise_sunset_scatter(date_range): | ||
| latitude = df2['Lat'][-1] |
There was a problem hiding this comment.
Use the most recent lat/long in the db for finding sunrise/sunset times.
Not strictly necessary but I had a secondary problem that others might have: My initial lat/long configuration (back when I first set up birdnet-pi) was wrong. Why does this matter? Well, the existing code uses the earliest lat/long it can find in the database! I cleaned up my data a bit, but decided to include this change.
This is a simple change that uses the latest lat/long in the database. I figure that's going to be as or more correct pretty much all the time. (Ideally, this would instead do an actual configuration lookup instead of inferring from db detection records.)
There was a problem hiding this comment.
Well-done! That is a good find! There's been quite a lot of people over the years who didn't initially set their lat/lon and did it afterwards. That would indeed result in the problem. I see what you mean about config lookup too. I agree that it would be ideal. Never looked at the code in Species Stats. Might have to familiarise myself with that!
scripts/plotly_streamlit.py
Outdated
| latitude = df2['Lat'][-1] | ||
| longitude = df2['Lon'][-1] |
There was a problem hiding this comment.
You could import
BirdNET-Pi/scripts/utils/helpers.py
Line 41 in 0c00e6f
and then:
| latitude = df2['Lat'][-1] | |
| longitude = df2['Lon'][-1] | |
| conf = get_settings() | |
| latitude = conf.getfloat('LATITUDE') | |
| longitude = conf.getfloat('LONGITUDE') |
If you don't feel like doing that (now), please add a your explanation in a comment.
Also, if your git foo allows it, I would prefer this change and the actual fix in two separate commits.
Apart from these nitpicks, great fix, Thanks!
There was a problem hiding this comment.
Yes! that's a great idea. This can be a separate change. Let me adjust this merge request to use the old lat/lon method, and I'll prep separate changes for this.
We'll enhance this separately
|
@Nachtzuster look for a separate MR for the lat/lon config settings changes. I'll do that after this goes in, just to simplify my own workflow. Assuming no other issues, I think this is ready for merge. |
|
Well-done to both of you. That is quite the elegant solution. Less fuss having two separate PRs for that! :) |
This bug has affected me for a while! I finally got to fixing it now that there's an active fork to benefit from the fix. :)
Sunrise/sunset scatter plots are misplaced for
This fixes those problems by using the same date range as the heatmap data. Before, it was making a simple date count from "now" without paying attention to the date bounds of the heatmap data.
To see these heatmaps and reproduce the problem:
Additional notes:
There are a lot of issues from this code that come from the original repo. The heat maps are beautiful if you've been running birdnet-pi for a while, but they are very slow!
I see a lot of room for improving the performance here, because it basically works with your entire DB starting with a "SELECT * FROM detections;" I have close to 700K detections on my system (mostly Canada Geese, I live by a river) and the Species Stats pane takes several seconds to work. Narrowing date ranges makes it incredibly slow. Plus, the way that the heatmap can be different than the date range selected isn't ideal, but I'm not fixing that here yet.
So this is a "simple" fix just to get the sunrise/set lines to align, as I get to understand how pandas and plotly work.
Below are some images of before/after this fix on my system:
Before: Eastern Phoebe (active just before dawn). Despite a full date range selected, the heatmap stops in October (when they left last fall) but the sunrise/sunset dates are for the full period.
After: the sunrise/sunset dates respect the inputted date range.
Before: Canada Geese, for a narrowed date range. The geese are most active around sunrise/sunset, but the sunrise/sunset lines don't match the dates (note how despite it being summer, sunrise is getting "later" - red arrow)
After: Canada Geese activity aligns with sunrise/sunset, and the summer solstice has the earliest sunrise again (green arrow)
Let me know if you have any questions or concerns. :)