mmap time bomb on iOS (with large realms)

[peppy]

We are in the process of migrating a large project from ef-core to realm. I am being very careful not to get ourselves boxed in a corner. While setting up a testbed and schema, considering past user metrics we could expect the realm to grow to several hundred megabytes in intensive usage. For hardcore (edge case) users it may even enter the gigabyte range.

The Limitations section documents a limitation surrounding the usage of mmap and references against a rdar closed by apple in 2015 as "expected behaviour".

This looks to imply that if a realm on a mobile device grows beyond (roughly) the size of the physical RAM the application will no longer be able to read from the file. Several issues I've read propose solutions such as "send the database to a pc for repair" (one such case is documented at realm/realm-swift#5966). This to me sounds like a time bomb with an unknown fuse length, dependent on the user's device hardware and current state.

Also saw discussion at #2581 (allowing at least compact to work) and #1935, the second of which seems to point to some work already being done to resolve this issue, but it was closed without any linked documentation explaining if that happened, as far as I can tell.

Is there any documentation on best practice maximum realm sizes? Has there been any discussion since the linked issues or any potential workarounds at a realm (native) level to fix this? I understand that it's ingrained in how realm works at a very low level, and also a limitation from apple's end, but in the interest of not bricking user's applications - especially those that would likely be making the most intensive usage of the app - it feels like this is quite an important consideration.

---

Note that I am writing this without yet testing the limits of iOS physical devices. Our development is primarily targeting desktop, but I do want to make sure we don't hit a brick wall with iOS users when we do release the app to a wider audience.

I am also posting this to realm-core even though we are using the realm-dotnet SDK, as it does seem like a localised issue.

[nirinchev]

Hey, we have an offsite this week and that's one topic I'll bring up with the team, but for now, one workaround we can think of is to use encryption. When encryption is enabled, we do manual memory management because we need to handle page decryption and dirtying, so (intuitively) it should not be subject to the mmap limitations on iOS. You do pay about 10% of performance price for using encryption, so it's not exactly free, but assuming you're not hitting performance bottlenecks, it may be a viable workaround.

Note that I haven't tried it and haven't heard from the Core team on how viable it is, but if you have some spare cycles, it should be reasonably straightforward to test out. After we're done with the offsite, I'll also try and verify the hypothesis, and - of course - will let you know how the team feels about getting rid of mmap for iOS.