Comment by TwistedWave
5 months ago
I have just reported a bug where high memory pressure could cause memory pages from the real-time audio thread to get swapped out, causing audio glitches. I am not hopeful for a timely resolution now...
5 months ago
I have just reported a bug where high memory pressure could cause memory pages from the real-time audio thread to get swapped out, causing audio glitches. I am not hopeful for a timely resolution now...
I have this using Firefox on a 2019 Macbook Pro with 16GB RAM.
If the laptop has not been restarted for a while, Firefox tabs will eat up more and more memory over time until it makes the mac kind-of unresponsive (*) and will need to get hard power cycled.
The only signs that this is about to happen are two:
1. Any audio-emitting webapp will start to get very choppy, whether is youtube on FF, Google meet on chrome, or a slack call/huddle. Which is the problem I think you are referring to.
2. Switching workspaces animation will get very very choppy. Usually when this happens it's already almost too late.
(*) this means that while the pointer is still moving smoothily, even from a BT mouse, there is no reaction to any click or keyboard typing.
I have the exact same issue, it starts to be choppy at random time and I have to reboot, usually in the middle of my work. I'm considering switching to another OS because of this single issue that has been there for years.
I see the same with firefox on my FreeBSD daily driver machine.
I thought it was because I pause the entire firefox process when the screen is locked. When it resumes it sometimes acts up (I use the STOP and CONT signals for this). But maybe it's just a FF bug.
I just restart it once every few days, my OS doesn't actually become unstable due to it. But i do have 64 GB of ram (one of the benefits to no longer being tied to apple's ecosystem is that memory upgrades and amazingly cheap)
This seems unusual for such a serious memory leak to be present. Do you have any addons/plugins in Firefox? One of them may be the cause of the leak.
Uptime, 456 days 12 hours 8 minutes and 42 seconds. Firefox, 668 tabs. Countless other background tasks. Why doesn't my audio work!?!
3 replies →
Yup. I have found a strong correlation between memory pressure and audio glitches with my external dac. I filed a bug on this over two years ago and it has gotten no traction, but it sounds like you debugged it a bit more deeply than I did, so maybe yours will get noticed.
> I have just reported a bug where high memory pressure could cause memory pages from the real-time audio thread to get swapped out, causing audio glitches.
By any chance, is that problem related to the new Microsoft Teams and its incoming-call notification sound? Ever since the Sonoma update and the coinciding Teams switch it's glitching horribly.
> the new Microsoft Teams and its incoming-call notification sound
I absolutely have this issue and would love to know more about any suspected causes.
Not for me. I have this problem and I have never used (or even installed) Teams.
There might be something to it, higher than > 60% memory pressure and the issue will get a lot worse and more frequent.
This is not fixable. You’d need an RTOS.
(You can’t mlock every single code page possibly executed by the thread for instance.)
Why wouldn't it be possible to mlock all the memory pages used by the real-time thread?
I do my best to do it on the application side. It would be nice if Apple's CoreAudio did the same.
If you mean that it is difficult to identify the pages used, that is true, but I found a great way to do just that. I keep track of all the pages I mlock(), list all the remaining pages, and mprotect them to crash if I accidentally access memory that was not mlocked. That helped me identify a couple of pages I forgot, such as the one that contains the __stack_chk_guard
And regarding the code pages, for some reason I get a permission denied error if I try to mlock them. Not sure why.
There is no mlockall() on macOS, but if there was one, I could delegate the real-time audio handling code to a separate executable, and call mlockall(). I would be guaranteed all the pages used by the real-time thread would be mlocked.
And by the way if you are interested in the report I sent to Apple, you can find all the details with the sample code to reproduce the problem there:
https://twistedwave.com/AudioGlitches.zip
How much memory could a single system thread handling the audio use? Forgive my ignorance I don't deeply understand OS level memory management but I thought, much like the task scheduler has priority understanding for CPU resource management, the memory manager had an idea of "these pages are important find some other page to evict".
> How much memory could a single system thread handling the audio use?
It's not "use" but "ever potentially access", so it's unbounded unless they were careful. (Audio playback to speakers itself is bounded, but mixing isn't, and wireless headphones get complicated.)
> Forgive my ignorance I don't deeply understand OS level memory management but I thought, much like the task scheduler has priority understanding for CPU resource management, the memory manager had an idea of "these pages are important find some other page to evict".
It's difficult to know what the most important pages in memory are, because you'd have to observe every single memory access, and that's not worth the overhead.
Basically there's two ways it uses:
1. a program can tell the OS to keep something in memory (either by hinting it or forcing it, called mlock or wiring).
2. every so often, the OS will disconnect some pages such that accessing them faults (called "moving to the inactive queue"). If there's a fault, it know it's being used and moves it to active again. Eventually, if there aren't any for long enough, it gets to the end of the inactive queue and is paged out.
About 25 megabytes for an approximately 10-second buffer.
I believe 96KHz/24bit is the highest quality audio supported natively on the MacBook Pro.
24 bits * 96KHz * 2 channels * 4x safety factor = 2.3 Megabytes per second.
this is the fault of a poorly designed application and not the OS
Here is the contents of the bug I have reported with all the details and sample code to reproduce the problem:
https://twistedwave.com/AudioGlitches.zip
It contains a program where the only thing I do in the real-time callback is to count the glitches, and I made sure to mlock() the memory used by the glitch counting code. When the memory pressure increases, it glitches.
The point is to avoid the memory pressure from increasing. Have you investigated the allocation behaviour of ALL the code in your application?
1 reply →