-
-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Instability and OOMs with large playback queues #957
Comments
I'm literally doing nothing wrong now. I'm just going to artifically delay the service launch by a couple hundred milliseconds to work around whatever is race condition is causing this. |
Try 4.0.0-dev4 @foss-? |
App still crashes but gives no crash log when re-opened. This happens sometimes on first song after tapping shuffle, sometimes first song plays time and a second tap on shuffle is needed. App becomes unresponsive for a brief moment then closes and users is on home screen. |
Wait, this is two distinct issues.
I need a crash report for #1. @foss- can you reproduce the issue and then take a bug report? |
The bug report contains too much info. Is this logcat info sufficient? It was created right after auxio crashes after tapping shuffle. In this case no audio ever played. 1 month: https://bin.disroot.org/?bd13202f37cb43fa#5osTFqr5XvQRots15gnhMPwQFsxDa6DkFQbZgQ6mbSS3 |
Out of memory crash, need to figure out what's going on @foss- |
Okay, turns out Musikr's native code is full of memory leaks. It probably is an OOM on your library. Will fix. |
Try this @foss-? |
Tapping shuffle results in message
When tapping a song to play app crashes and closes. |
That error you provided is a music loading error, you need to take another logcat on the song playback crash @foss-. |
Does this logcat inlcude the information you are looking for? 1 month: https://bin.disroot.org/?e7a19007948e4331#2fKogXwUu3UyLdCAguwdjvkYju1Hn11X6tNrHSwVBcUG |
Yeah, apparently at some point I turn an internal object into a string, which results in a... 16mb allocation that instantly knocks out the VM. Might need to audit some accidental printf debugging I left in the app, give me a moment @foss- Thanks for stresstesting too, I dont have a big library so you and e-zk's libs are basically my only way of knowing if I broke those or not |
Here's a new APK @foss-. This removes the debug log that allocated too much memory and crashed the app, but I'm partially concerned that your phone also has a strangely low memory limit and could be GC'ing constantly during playback (causing the unresponsiveness). In this case, I need to deep dive on what's changing memory-wise between versions. |
sent logcat via email |
Yeah @foss-, your device has some very weird memory use patterns. Constant garbage collection, near total memory use, and eventually ExoPlayer just causes an OOM. Very strange. It also doesn't help that Auxio only has 256mb available to it, which is basically peanuts. I'm going to measure memory use on my own device and compare. |
Hi again @foss-, I've analyzed the memory use between v3.6.3 and v4.0.0 and there does seem to be something like +100mb of memory use for some unknown reason. I really don't know why, will try to investigate. |
Quick question: Does changing Something that could be happening is that the covers I'm loading are just too big. |
That could well be. Changed After the library is loaded Auxio crashes after tapping the first song. Double checked and the cover of that track is large: 1,2 MB. Created a 106 KB variant of the cover and replaced the file. Reloaded library in Auxio debug version and instead of crashing the file now played. Currently batch resizing all cover images to a reduced size. Will report back. |
That would make sense @foss-, ExoPlayer also tries to load the cover (I can't change this annoyingly), so it's probably heaping on more memory on top of the existing poor memory use. I need to debug some other deeper memory problem, there is at least double the memory use between Auxio v3.6.3 and v4.0.0 and I can't tell why. |
This was a major change to my library and the dust has again settled. No large cover image files. The shuffle crash is persisting in 4.0.0-dev4:
Oh this is also happening when just tapping another song while one song is playing, no shuffle involved. Re-testing with debug build: tapping a track and app crashes. 3.6.3 is the last working release as far as my library is concerned. |
Okay @foss-, I just have to bisect the memory use problem on my end and figure out what caused the surge. I don't know when I will have the time, hold tight. |
Okay, figured out the issue. The way I abused kotlin flows to distribute work several times over wound up making circular references that would leak huge chunks of memory. I have to completely rebuild the system. |
Okay, so not even that was the issue. It's a memory leak stemming from a bugged memory allocation in the Java standard library. After switching to manually allocating the memory in native code, the memory leak seems to have disappeared @foss-. Can you try this APK? |
crashes on first tap on shuffle and crashes on first tap on a track - sorry I don't have better news. Not getting any debug output, so here's a logcat (1 month): 20:06:00 song tapped |
Okay, this is inexplicable at this point @foss-. I shaved off something like 200mb of leaked memory and it's still not enough. I'll see if perhaps a release build "works", if not I think you might have to run a heap dump and send it to me so I can identify what's still leaking. |
Hi @foss-, can you see if dev5 is an improvement at all? I can't tell if it's just a debug artifact or not. |
Thanks, PID changes after auxio crashes so updated steps worked. Now I need to figure out:
|
|
Perfect - you got mail. |
Are you sure @foss-? I haven't gotten an e-mail yet. Did you send it to my profile email? |
Nevermind @foss-, just received it. |
Okay @foss-, looking at your heapdump, it seems like the problem is that ExoPlayer is completely overloaded with a huge queue. It seems like the new cover URIs finally pushed it over the edge. Is there any state restoring going on when you started the app? |
When I close the app and then reopen it does the loading process which takes 1-2 minutes. Then I see the library. Is that what you were asking? |
No, is there any playback going on @foss-? i.e the playback bar shows up. Because in the heapdump, I see a pretty big queue appear and use tens of megabytes of memory. |
Oh I see - yes, a track is loaded at the bottom and tapping the bottom controls and sliding up the queue I see a very large queue. is there a way to reset that? |
Not necessarily @foss-, there's #478, but I haven't implemented that since working with the bottom sheet UI is a nightmare. There's a few courses of action I can take immediately:
Long-term, I can try to mitigate a lot of the memory use with better management of string IDs, and waiting on Media3's planned paging player. I think Number 1 is probably the best course of action. |
Deleted app and re-installed to get rid of the queue. But that was not possible, it's still there. Good progress and options to move forward are on the table 👍 |
...?!?!?!? The playback state should in no way continue to persist like that. I am very confused about what's going on here @foss- Does clearing app storage work? |
Uhm this is interesting. The queue is peristing. And the queue apparently is all tracks sorted alphabetically starting with 01. I have no idea why all tracks keep being added to the queue, but that is what seems to be happening. Sadly did not notice this sooner although it's obvious 🙄 |
Lets say you edit the queue and then re-install. What happens? If it reverts then theres no persistence, the app is just jumping to play the entire library for no discernable reason. @foss- |
Tried dragging a song up to change order - but app crashes Tried removing single tracks and I am certain those re-appeared, this is if the app didn't crash on attempt to delete a track So yes, it seems the entire library keeps re-appearing as queue |
Okay, here's a debug build that will crash the moment a playback command is sent. Can you log the crash on this build so I can ID what codepath is even causing this? (Seriously, so many things can trigger a playback) @foss-? |
You got mail. Let me know if the debug data was helpful. OT question as I just started using Android Auto: is it correct that for Auxio to show in the Android Auto launcher it needs to be installed from PlayStore and that is pending as per #572 so while Android Auto is supported in theory it can not yet be used in practice? |
I don't need the heapdump but the logcat output containing the crash @foss-, can you send that?
You have to enable android auto's developer settings and then turn on "enable unknown apps". |
Logcat for latest debug build (1 month):
Fantastic - thank you. |
Did you start playback at all @foss-? I want to see what happens on a fresh install (uninstall and reinstall), without any playback occurring. Currently the codepath seems to be triggered by a UI interaction, which doesn't seem right. |
Maybe I misunderstand what you mean? I tap a song but no playback ever happens. So what do I do? Uninstall, reinstall latest debug and then what action do you want to trigger the crash? |
Uninstall, reinstall, and select a folder, then wait @foss-. Do nothing. Since for some reason song playback is appearing out of nowhere, I need to see why that is happening. |
You got mail: Logcat after launching auxio after re-install, selecting music folder, letting scan finish and then no further interaction. |
Okay, so I see no logs relating to playback @foss-. Did the playback bar show up when you were testing? The only thing I think that could trigger the bug now is a mysterious state restore error. |
Odd, re-opened 4.0.0-dev5, playback bar was there. Played a few tracks and when now looking at the queue it shows all tracks from a single album of which I played a track from the songs tab. No lag, no playback issues, skipping works. Re-opened your latest debug build. This opened without playback bar which imo also differs from what I saw in the past. So from what I see, the issue with the queue leading to unstable behavior is gone. Question is, should I press shuffle to see if I get back into unstable state? |
Makes sense, since you switched from a 36k song queue to a dozen-song queue. Way easier for Auxio to chew on.
?!?!?!?!?!?!?!. Looks like this is either a misunderstanding in communication or a heisenbug. Guess there's nothing that can be done.
If you do this, it'll just trigger the instability caused by kinda unavoidable issues. Don't really need that. Guess this issue is not a problem then for release. Leaving it open as a bigger thread for the issues behind huge queues. |
While playback did work, auxio keept howing the message "Auxio isn't responding" after attempting to alphabet scroll songs tab. When selecing any song, that message shows again. Song still plays. |
Makes sense, I don't think the new fast scroller is very efficient with huge lists (unsure why), and ExoPlayer is going to chug BADLY with 36k songs @foss-. Going to place this in the realm of huge library edge cases and continue w/4.0.0. |
Describe the Bug/Crash
Auxio 4.0.0-dev3
Is this #825 all over?
Reproducible by tapping shuffle button on large library.
Describe the intended behavior
Should not crash.
What android version do you use?
Android 13
What device model do you use?
Xiaomi lisa
ArrowOS 13.1
Provide a sample file
N/A
Bug report
Duplicates
The text was updated successfully, but these errors were encountered: