macOS window tracking broken?

[nomeata]

Original report by [email protected] (Bitbucket: asbreckenridge, ).

---

hey! i’ve just installed this on an Intel macOS 11.2.2 machine, arbtt-dump reads

2021-03-01 16:45:52 (295ms inactive):
2021-03-01 16:46:52 (18580ms inactive):
2021-03-01 16:47:52 (490ms inactive):
2021-03-01 16:48:52 (218ms inactive):
2021-03-01 16:49:52 (973ms inactive):
2021-03-01 16:50:52 (6ms inactive):
2021-03-01 16:51:52 (5930ms inactive):
2021-03-01 16:52:52 (23ms inactive):
2021-03-01 16:53:52 (33136ms inactive):
2021-03-01 16:54:52 (93139ms inactive):
2021-03-01 16:55:52 (153144ms inactive):
2021-03-01 16:56:52 (559ms inactive):
2021-03-01 16:57:52 (4ms inactive):
2021-03-01 16:58:33 (37ms inactive):
2021-03-01 16:59:33 (105ms inactive):
2021-03-01 17:00:33 (140ms inactive):
2021-03-01 17:01:33 (129ms inactive):
2021-03-01 17:02:33 (2599ms inactive):
2021-03-01 17:03:33 (142ms inactive):
2021-03-01 17:04:33 (148ms inactive):
2021-03-01 17:05:33 (419ms inactive):
2021-03-01 17:06:33 (190ms inactive):
2021-03-01 17:07:33 (33ms inactive):

‌

[nomeata]

Original comment by nomeata (Bitbucket: nomeata, GitHub: nomeata).

---

Hi,

sorry, I didn’t write the OSX support, nor can I test it. Maybe it
stopped working on new OSXs? You’ll have debug that yourself, or find
people who might know more.

Cheers,
Joachim

[HarrisonMc555]

I just installed this tool to try it out and I'm seeing the same behavior. There's a very real chance that I don't have my setup fully functional yet, but I can vouch that I also see this behavior. Here are my specs:

MacBook Pro (15-inch, 2017)
Version 11.5 (20G71)
Big Sur

Do you have any tips for debugging? I'm willing to spend a bit of time looking into this, but I didn't see any debugging flags for arbtt-capture or anything. Not sure if I missed something.

[nomeata]

My assumption is MacOS has changed in the last years, so this might require someone who knows the OSX APIs and Haskell to play around with the code in https://github.com/nomeata/arbtt/blob/master/src/Graphics/OSX/Window.hs

[HarrisonMc555]

I have a passing familiarity with Haskell and essentially no knowledge of OSX APIs. If I get some time to look into this I'll try, but no promises on any results 😉

[VitamintK]

I experienced this problem as well, and I figured out a fix!

This apple dev forums page says that to get kCGWindowName, you need screen recording permissions. (kCGWindowOwnerName is fine without permissions, though.)

On Mac OS 11.4 (Big Sur) I added Terminal.app to Screen Recording permissions (it appears to be unnecessary to add arbtt-capture) and when I start arbtt-capture from Terminal, it works!

[nomeata]

That's great news!

So we just have to document this? Or is there a way for arbtt-capture to ask for these permissions?

[VitamintK]

Good question! I'm not sure. From this stackoverflow answer, there should be a pretty simple way to request permissions. Only problem is: I'm not sure if it'll request permissions for arbtt-capture and not Terminal and thus useless.

Also, I'm not sure what program you'd need to grant permissions to if starting arbtt-capture from a cron or at startup or however else people run it.

Related discussion on a separate project encountering a similar issue: lwouis/alt-tab-macos#33

I'm not sure if I'll look into this further in the future, so it's probably best to simply document for now, and add permissions-asking in the future once I/someone else decides to look into it.

[waldyrious]

I'm not sure what program you'd need to grant permissions to if starting arbtt-capture from a cron or at startup or however else people run it.

is there a way for arbtt-capture to ask for these permissions?

In #30 I was able to get a permission prompt for sh, by executing a shell script with the /bin/sh shebang that simply calls the /usr/sbin/screencapture executable. I could then use those permissions to run arbtt-capture wrapped in a similar shell script.

So I would say that if arbtt-capture either did the same thing that /usr/sbin/screencapture does (probably using some of the code mentioned in the stackoverflow answer that @VitamintK mentioned above), or invoked /usr/sbin/screencapture directly when executed for the first time, then the permissions prompt would show up for the arbtt-capture binary (rather than sh) and we'd then be able to use the launchd.plist as described in the README to start arbtt-capture automatically on startup.

Otherwise, we would have to change the docs to include something like the steps listed in @LiamHz's comment in #30, i.e. request permissions for sh, and execute arbtt-capture via a thin wrapper shell script. But that would be a rather hacky way to do this. It would be cleaner if arbtt-capture could request the permissions either directly or via /usr/sbin/screencapture.

[nomeata]

That would be my hope as well. So should I try to add a call like this to arbtt-capture? Is that synchronous, i.e. will it block while the dialog is open?

[waldyrious]

I would assume so. In the process I did in #30, the dialog indeed blocked the script, IIRC (but unfortunately I can't test that anymore because I no longer use a macbook).

[adam-dakota]

From my testing, it seems to no longer be possible on Monterey to use the /bin/sh method to make macOS assign the permission to sh. It insists on the terminal window getting the permission instead. I also tried various ways of forcing the TCC database to give the permission directly to the arbtt-capture binary and it seems not to want to do that. Tried to disable SIP and write manually to the TCC.db like the user in #30 did (obviously, replacing the placeholder string with my actual path & updating timestamp for good measure):

sudo sqlite3 /Library/Application\ Support/com.apple.TCC/TCC.db "INSERT or REPLACE INTO access VALUES('kTCCServiceScreenCapture','your binary path',1,1,1,NULL,NULL,NULL,'UNUSED',NULL,0,1547635615)"

The write succeeded, but no effect on the daemon's ability to capture. Not sure why; my write to the DB does remain after re-enabling SIP. The script method does make running arbtt-capture in a terminal window trivial, but not as a daemon.

Here is a JS example to request the permission—it was on the linked stackoverflow, so I'm guessing you have seen it already. I am in no way experienced with Haskell nor, more importantly for this situation, any of this macOS permissions nonsense, so I gave up there. However, I did find a good hacky way that uses intended macOS capability.

1. Open macOS Automator
2. Create a workflow like so or similar (with the proper path to binary of course—you don't even need to write a script!):
   
3. Save it as an Application
4. Add it to your user's Login Items menu in System Preferences
   

Clunky, but effective. The right approach here may be to feature this workaround in README or documentation somewhere rather than adding anything macOS-specific to the project itself. I'd be glad to write it up if you tell me where to do that :)

[nomeata]

Thanks for the offer! https://github.com/nomeata/arbtt/blob/master/doc/arbtt.xml is the source for the manual at http://arbtt.nomeata.de/doc/users_guide/index.html and probably the best place for additional instructions.

[waldyrious]

Great investigation, @ogrok! Since both the README and the website include brief installation instructions for macOS, it would be nice to mention this there too, even if just as a link to the user manual.

[adam-dakota]

Sad news: After a couple good days, my workaround doesn't consistently work either :( In a stunning turn of events, upon about the second restart, macOS seems to have forgotten my Automator app has the screen recording permission. I noticed arbtt-stats reported the same output an hour ago as it did last night, so I stopped the running script and started it again. macOS then prompted me to give my Automator app the screen recording permission, which is... interesting because it already has it according to System Preferences. I have confused macOS and am not sure how to proceed.

Anyway, no pending documentation change until I get some time to dig deeper into this and hopefully fix it

[jmatsushita]

I manage to make this work with nix-darwin in the following way:

• I used this launchd configuration (with my username) https://github.com/toonn/nix-config/blob/3fe2890ea838f63f1d6e9c130dc4057c3ce24fce/darwin/configuration.nix#L44-L58
• I rebuilt by nix darwin configuration.
• I then looked at the generated launchd config with more ~/Library/LaunchAgents/org.nixos.arbtt.plist, specifically:

        <key>ProgramArguments</key>
        <array>
                <string>/bin/sh</string>
                <string>-c</string>
                <string>exec /nix/store/fgf95hy12y9rbsa30q3y5y17g920nmi1-arbtt-start</string>
        </array>

• I then looked at the script in question, and the #! line:

> more /nix/store/fgf95hy12y9rbsa30q3y5y17g920nmi1-arbtt-start                                                                  01:26:00
#! /nix/store/60qp4q78hlg1fsvq4np6iv0gpqrl4v4p-bash-5.2p26/bin/bash
(...)

• This is the process that we want to give Screen & System Audio Recording permission to (that's the new name in MacOS Sonoma, in Privacy & Security).
• Click the + button and to navigate to the right path in the nix store use this trick:
  • Use Cmd+Shift+G to open the Go to Folder widget.
  • Paste the beginning of the path to the bash executable, in my case /nix/store/60qp4q78hlg1fsvq4np6iv0gpqrl4v4p-bash-5.2p26/bin
  • Now the window opens in the right folder, and you can select bash
• Now we need to restart the launchd service we "kickstart" it, with the -k option to kill the process if it's already running.
  • launchctl kickstart -k org.nixos.arbtt

Now check with arbtt-dump ~/.local/share/arbtt/2024.capture if you have some window titles. (you might need to install haskellPackages.arbtt in your user environment, or you could run nix shell nixpkgs#haskellPackages.arbtt -c arbtt-dump -f ~/.local/share/arbtt/2024.capture).

This will probably break when you nix flake update, in which case you'll need to give permissions to the bash executable in its new location in the nix store.

Hopefully this is applicable to other MacOS environments than nix-darwin!