Some official/compatible AirTags don't work #90
Comments
|
Hmm that's strange, I recently used the scanner for another project of mine and could've sworn it received multiple "nearby" reports. That said, I have received a report from someone else saying that the accessory key generation is not working properly, which might be related to your problem. I've been quite busy but will investigate a bit more soon, will keep you posted. |
thisiscam
changed the title
To clarify, I can see there are nearby devices, but I can't match them with my own devices with known private keys. |
|
Ahh alright, thanks for the clarification. I'll look into it! |
|
Could you try whether #91 fixes your issue? You can install it as a drop-in replacement using |
Unfortunately that didn't fix the problem. I was already doing a timeshift adjustment in my own code. Is it possible that Apple changed their "5 matching bytes" protocol? |
|
Some possibilities:
Reason 3) is possible is that one of my Mac was updated to macOS 15 recently and I can no longer decrypt the plist on that machine. Ref: MartinPham/FindMySync#38 |
|
That's really strange, and I'm honestly not sure what's going wrong. Of course it's possible that Apple changed the spec, but that's assuming that they somehow have the ability to update both airtags and 3rd party tags over the air. I'm picking up my own 3rd party tag soon to do some experiments on this. |
|
I'm leaning towards 1) at the moment. If I put my phone on flight mode, I get one (new) separated device, which presumably is my airtag. But it also doesn't match with any known public key in the time shift range. |
|
Ok this should eliminate possibility of 2): I put my phone flight mode, got one separated device: I checked that >>> import base64
>>> s = "XLBrweEg0MRQwAt5gk0zk0rRDOQtdW2vr1WCqw=="
>>> base64.b64decode(s)[1:6].hex()
'b06bc1e120'So it seems like the 5 bytes trick is still valid! It then seems likely that they changed the rotating key algorithm. They could be changing only the official airtag's algorithm afaik. I can see an airtag software version number on my phone --- not sure if recently got updated though! |
|
The public key generated by the library is partially derived from those bytes in the mac address, so I'm not surprised it checks out ;-). And that's interesting, though I guess it makes sense that Apple would want to update the firmware on their own tags. Theoretically speaking it would be possible for Apple to use a new, updated algorithm only on their devices / AirTags, since owner devices are aware of the type of accessory that they're tracking. If possible, could you try to re-pair the tag, then dump the plist again and see if that fixes the issue, at least temporarily? |
Yeah, I re-paired, dumped and decoded the plist on macOS 13. The plist correctly reflects that I have repaired. But that did not fix the issue. |
|
That unfortunate; I'll probably pick up a tag next week and experiment a bit on my own. Will let you know how it goes. |
|
Sorry for the ping --- is there an update to this? Should I an opensource tag instead? |
|
Depends on what you call an update - I've been able to pair a tag to one of my apple accounts, but for whatever reason I cannot get it to show up on my hackintosh. My best guess is that's because I only booted it up after having logged out of the device I paired it with. I don't exactly have regular access to an iPhone or iPad, let alone one I can use with a temporary account, so it's all taking way longer than necessary. Currently I'm just regularly saving the data this tag is broadcasting, so as soon as that plist is dumped I should know more. |
|
Thanks for the update! I am fairly certain that the BLE nearby device tracking functionality is not working for official airtag and official clones anymore. In the mean time while you look at this, do you have any recommendation for a open source drop in replacement tag? As in, do you happen to know (to a good degree of certainty) that particular hardware should work with the BLE nearby device tracking? |
|
The BLE device tracking should work with any "static" OpenHaystack tag. There are a bunch of people working on different firmwares for different tags, so it mostly depends on which hardware you can get your hands on. I am personally using these NRF52810 tags with this firmware, but there are likely cheaper options available. |
Brother, I have some problems that need your help. Please leave your contact information and I can pay you. |
If it's related to this issue then please leave your question here, that way everyone can benefit from the shared knowledge. If it's something really private you want to share, feel free to use my git email. |
|
Just an update on my end: I actually tried to pair my official airtag with an iOS 17 device, and got the plist out from a macOS 13 --- it still does not work. At this point, the only possibility is that they changed the key rotation. |
Actually, I spoke too soon! For the above comment, I was testing iOS 17 + macOS 13 with the latest version of the FindMy.py library, which wasn't working. However, after switching back to FindMy.py version 0.6 (prior to the nearby scan was implemented) and using my old code from #46, everything is working again! This suggests that an issue in the current FindMy.py scanner code. Since I am able to get everything to work again using iOS 17 + macOS 13 combo, it appears that Apple hasn't changed the AirTag key rotation algorithm itself. But, I am likely using the wrong master key after upgrading to iOS 18 + macOS 15, since the new keychain app does not allow easy extraction of the BeaconStore key to decode the plists, and I had to use macOS 13 to do the plist decoding! Interestingly, if I pair the official AirTag with iOS 18 but extract the plist using macOS 13, I get a plist file that still allows querying the AirTag's location from Apple's server. However, it’s possible that Apple is concealing the true private key with the iOS18 + macOS13 combo. |
|
That's great news, thanks for investigating! So, just to summarize:
Correct? |
Right. The fix is in #97
Correct. |
|
Cool, thanks for confirming. I'll look at your PR tomorrow when I have my laptop available. I think it might be time to seriously start looking into how keychain syncing works. That way we should able to obtain the keys directly without having to dump them from a mac. |
Keychain (as in the macOS app) syncing? Or do you simply mean how the airtag communicate with an iOS device to decide on the master private key? I do suspect that we can potentially have an middle man when the airtag pairs for the first time. Not sure how feasible that is though! |
|
Setting up an AirTag still requires communication with Apple servers for registration, and I'd much prefer trying to reverse engineer a service running on macOS than one that runs exclusively on iOS 😅. I also think that keychain syncing would be more useful in general, because it would allow people to use AirTags without having to reset and pair them through this library. That said, both are interesting to look into. |
I'm lacking knowledge of what is "keychain syncing"? |
|
Keychain as in the icloud keychain that stores secrets in your account. Since the AirTag master secrets are stored in the keychain, that should make it possible to pull them directly from an Apple account after logging in, instead of having to dump them from a mac. |
|
Hi anything new to this? I am not having access to macOS < 15.0 or iOS<18. Can i do anything? BR kami |
|
I am still looking into this, as my hackintosh still refuses to sync for whatever reason. To the best of my knowledge people have not yet been able to dump the master secrets from macOS 15, so if you don't have access to a < 15 machine it's going to be a challenge. |
|
(continuation from #31) @aircable: Do you have the exact pairing date of your tag for me? It should also be in the plist, but it appears to be missing for some reason. Did you edit that part out or is it missing from the original file? You mentioned that your tag was paired in 2023, but I was able to find the private key belonging to the scan you posted at index ~13291. That's roughly 13291 * 15 minutes = 4.5 months after pairing time, so it can't have been in 2023, unless there's an issue with the library. If I select September 12th 2024 as pairing date and use the values from the plist you posted, I can pull location reports from Apple that were generated earlier today. |
I have, my Macbook Air M1 is Sonoma 14.4. Let me know what to do. |
How did you do that?
What app do you use to decrypt that? |
|
I did mention that the tag was off for an extensive amount of time. That's why you could not find it. But tell me, the analyze_key.py file I shared did not go to index 31291? |
|
finding more information, from OwnedDeviceKeyRecord. The deviceIdentifier is the one for that Airtag. |
|
Sorry, missed your responses.
My plan was to pair a tag and obtain the very first key it broadcasts, then start from there. That should allow me to verify the assumptions about timezones etc. as well, since I'm not 100% sure that's correct yet (even though it shouldn't cause any issues in practice) Thanks for the pairing date! Do think it'd be possible to connect the tag to an owner device to make it update its internal clock? I'm curious to see if the key rotation algorithm would immediately jump to the "correct" key or not.
I used the same debugging script you posted but set the pairing date all the way back to 2018 or so. Then filled in the public key (not the lookup key) from your scan and let it run until it found a match. The date of the match is wrong but relative to the pairing date that is set, and since it was ~4.5 months after the pairing date I used, I figured it must've been paired ~4.5 months ago. But I forgot about your tag being turned off for a while, so if it was paired in 2023 that still checks out. Using the above method to find the "real" (wrong in theory but real according to the accessory's internal clock) pairing date, I plugged it into the I'm not sure what those SecureLocationInfo and OwnedBeacon files are supposed to represent. The private/public key lengths don't match up with the EC keys used in FindMy. However, the following strings look interesting: ACSN (anonymous crowd-sourced network) is the server-sided part of FindMy running at Apple. It looks like there are lower OS version bounds for when to enable participation in the network? I don't think it's related to this issue, but interesting nonetheless. |
So, basically you changed the airtag plist file and put some different numbers in and just ran real_airtag.py airtag.plist, right? What did you change in the file, can you please post? I just want to make sure I understand what you did. |
|
I left the plist intact, but manually set the pairingDate (since it was missing). I've filled in your details in the files below. Run this with the path to the plist to find the private key: I then played around with the pairing date around september last year until it found the most recent location reports: |
|
Thank you so much, I was able to follow your steps. Got data. But not those steps:
Using the airtag_testerl I find the key at
That is vs the reported pairing time: Then, as you did, searching for the data using datetime(2024, 9, 11) leads to the most keys found. But no other date determined by the airtag_tester script. My interpretations maybe is that the possible_keys you generate are just not enough. How many are you actually generating? I also found out that a span of 4 days will give you the maximum number of reports, in out case 2024/9/10-13. Do those numbers ring a bell? Are you creating a 4 day window with the key generator? |
|
I have actually verified that the pairing date for that airtag was indeed 2023/8/31. The plist file is correct. |
|
Let me try to explain the AirTag's inner workings in a bit more detail. I hope that clarifies things a bit, while at the same time serving as an attempt at more understandable public documentation :-). I'll try to elaborate on (almost) everything start to finish, so it might include some redundant information. As you are probably aware of by now, the FindMy network works using cryptographic key pairs. The public key is broadcasted by a device, and we use the private key to obtain the device's location. For custom tags, we almost always use static keys, meaning that the key broadcasted by our custom device does not change. This means that the private key is very predictable (it's always the same!), so fetching locations from Apple is very easy. However, AirTags and compatible 3rd-party accessories rotate their keys in an almost unpredictable way in order to increase privacy (+ obfuscation??). While we can use the same method to obtain an AirTag's location as we use for our custom tags, we also need to know which keys the AirTag has broadcasted during the time range that we would like to query. Otherwise, we'd just be throwing a LOT of random keys at Apple's servers in the hope that some of them will work. That's just inefficient and very slow. AirTags have two distinct key derivation algorithms. The exact cryptographic details of how these work does not matter, but what does matter is that each of these algorithms outputs a single key for a certain value The main issue is actually finding When the tag has recently actively connected to its owner device, it broadcasts a "lite" version of the regular FindMy broadcast data. This data is not enough for other devices to identify and upload its location, but is sufficient for owner devices to recognize it as one of their own. They do this by only broadcasting part of the primary key. This key changes by incrementing When in "nearby" state, the tag also broadcasts its primary key, but in full this time. This "enables" the stranger-finds-my-tag features of FindMy. Again, this key changes every 15 minutes, because When in "separated" mode, the tag starts broadcasting its secondary key. This key only changes once per day. Note that even while in separated mode, The annoying part is that we are essentially operating in the dark: we do not know which state the tag is in, or what the "last" value of The above means that at any arbitrary point in time after the pairing date, we have two or three potential keys that the tag could be using:
Note how we can now guess the value of Now in order to find The debugging script essentially iterates through the possible values of paired_at = datetime.now().astimezone(timezone.utc).replace(year=2023)Right now, this evaluates to february 7th, 2023. However, note that this date could be completely arbitrary for the purpose of this debugging script. All we care about is the offset between this supplied pairing date and the datetime at which the script finds this key among the potential keys at that time. No matter which pairing date you supply, as long as the entered public key is the same, this will always result in roughly the same offset between the supplied pairing date and the final date indicated by the script. Some slight variations are still possible, but this is because it is much more likely for the found key to be a secondary key (which are valid for 24 hours) than for it to be a primary key. In order to account for these potential offsets, we have a few options:
I dislike every single one of these solutions since I would very much like to keep the library usable, stateless and account-agnostic, however I do believe (3) could be implemented in an OK way. I'm imagining keeping the setup with the pairing date as it is, but introduce an offset parameter for I do realize this is a lot of text, but hopefully this clarifies everything 🙂 |
|
Wow, thank you so much. Your description is incredible. I believe it will be very helpful, for others too. So, I worked on the airtag_tester a bit and created a program that counts backwards and tries to find the correct key as well as the computed number of days offset, which then leads to the corrected pairing date you can use in your real_airtag script. And it worked. Output:
Which is exactly the date you would use to find the real Airtags. Maybe we should now integrate the airtag-decryptor-3.swift, the aitrag_tester_2.py and the real_airtag_2.py scripts into just one "get location from real airtags". And scan_devices.py of course to get the current public key, which airtag_tester_2 uses to start searching. |
|
Nice! I don't think the scripts would need to be fully integrated though, since "calibration" could be a one-time thing. Once we get at least one location report we can use that to do further offset calibration. I'm thinking of something similar to the following workflow:
If we make the FindMyAccessory class serializable, the offset could be stored in the JSON representation, and calibration after that can be done without referring to the plist. |
|
Sounds about right, which means that the library need to track i and j. |
|
Probably in the library. We somehow need to keep track of the expected generation time for each key, then if it does not align with the time of the reports for that key we can derive a new offset from the difference in those times. It also doesn't need to be very accurate as long as there's a 24h margin. |
malmeloo
changed the title
|
Thank you @malmeloo for the very detailed explanation. I have this exact problem with two dozen tags being paired several months ago. I re-inserted the battery saver tab after pairing. Now when tags are powered and field deployed (separated mode), tag key indexes are off compared to the time-window calculated by FindMy.py. I presume the tag clocks may also naturally drift due to battery voltage dropping (this would be interesting to investigate by itself, and could be quantified by observing a faraday-caged tag and ble sniffer for some time). I therefore think your suggestion (3) with automatic continous calibration towards reported time, and saving of estimated i,j indexes, is a necessity for providing reliable lookups and also has a benefit reducing the report query load. Key question: would an key-index-centric design make more sense than a time-based design? Idea: after initial calibration (based on time, opportunistic probing or full probing), each invocation of FindMy.py would If FindMy.py is run at least once per week, it will in the worst case be the same as the current "fetch_last_reports" that queries 7*24 hours + 12+12 hours = 768 id queries only based on time. By searching backwards, and stopping when at least one report has been found, unnecessary queries are avoided (if only the last position is wanted - to get location history, or additional accuracy, more or all reports can of course be fetched. For stationary tags, combining several reports should improve accuracy). If FindMy.py has not been run for a long time (say months), there can be some opportunistic/sampling discovery, for example instead of 96 reports per day, try 10 reports per day and do a normal re-synchronization after an initial round. This method requires a local state to be kept and updated, but I think the value outweighs the drawbacks - and the time-based algorithm can still be used of course, if the user chooses. In my use-case with many severely de-synchronized tags, a key index offset is a necessity (edit - after seeing the airtag-tester script, I could also adjust pairing date to accomodate for the time offset... I'll do this for now) |
|
Yeah I've been thinking about this a bit more recently and I'm increasingly less opposed to tracking internal state for airtags. The library already has an abstract base class for things that need proper state handling, so we could just extend that to the accessory class. Maybe with a warning if it gets garbage collected with unsaved runtime data, to really get the message home. I think the final implementation will probably look similar to what you're proposing. Keep track of the last fetch time and synchronized i and j, then calculate offsets from there. Your idea about opportunistic discovery is quite interesting, but I think we can actually reduce the number of keys per day to only 1 for this. Since the vast majority of location reports will likely originate from secondary keys, and those only regenerate once per day. However, that does mean that we will still need that 12-24 hour margin, because we cannot recalibrate the indices super accurately using those secondary keys. After all, they're valid for 24 hours instead of 15 minutes. |
|
I did a PoC (attached) that scans secondary keys for 200 days and saves the last key index for the found report in a cache json file, reused at next run. It now finds my missing tags (great!), but there is still some problem with reusing indexes (seems keys_between don't produce the same result with a full-range scan [1, 200*24*60/15 ] or a subset scan. I'll look into this a bit more. edit - keys_between() returns a set, so indexing can't be done this way. Either the library can use a sequence, or I can just do the same iteration as the library does ( while start < end: keys.update(self.keys_at(start)) ) and use a list instead. Anyway, this approach seem promising for very efficient lookup and responses, which enables lookup scaling of many tags (after calibration, 1-2 lookups needed per tag that can be packed in 256 id chunks) vs quering several hundred id's per tag and getting possibly thousand of reports back. |
|
I updated the the indexing code (iterating over keys_at()), and it's now consistent from initial calibration to subsequent runs. Index now matches expected operating time (key index of the last successful report is 1055 => ca 11 days of operation which is according to expectations (paired in december, powered off, and deployed 1,5 week ago. |
You could perhaps use https://pypi.org/project/ordered-set/ |
Nice! The use of sets is indeed a bit of an issue, but once it's implemented in the library that shouldn't be an issue anymore. I'll see if I can find some time this week to create a first version with the fix. |
Recently the 5 bytes matching trick discussed in #46 appears to stop working.
I also just tried #50 and seems like I can't find devices that are in nearby state using the example device_scanner.py
The text was updated successfully, but these errors were encountered: