[HOLD for payment 2024-03-11] [HOLD for payment 2024-03-07] [$500] iOS - Attachment - PDF files aren't cached and flashes once while loading

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Version Number: 1.4.5.1
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:

Action Performed:

1. Navigate to https://staging.new.expensify.com/
2. Log in
3. Upload a larger PDF (10-20MB) to any conversation
4. Open the PDF
5. Go back to the chat
6. Open the same PDF again

Expected Result:

PDF files should be cached and no flashing once while loading

Actual Result:

PDF files aren't cached and flashes once while loading. They are downloaded every time a request is made

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

Bug6295195_1701295268248.Cache.mp4

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01aa9414036934592a
• Upwork Job ID: 1729988103538311168
• Last Price Increase: 2023-11-29
• Automatic offers:
• getusha | Reviewer | 27965277
• tienifr | Contributor | 27965278

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01aa9414036934592a

[melvin-bot]

Triggered auto assignment to @garrettmknight (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @getusha (External)

[ZhenjaHorbach]

Proposal

Please re-state the problem that we are trying to solve in this issue.

iOS - Attachment - PDF files aren't cached

What is the root cause of that problem?

We don't use special param for using cache

What changes do you think we should make in order to solve the problem?

Update this line like

source={{uri: this.props.sourceURL, cache: true}}

App/src/components/PDFView/index.native.js

Line 158 in f7124d5

source={{uri: this.props.sourceURL}}

What alternative solutions did you explore? (Optional)

NA

[tienifr]

Proposal

Please re-state the problem that we are trying to solve in this issue.

PDF files aren't cached. They are downloaded every time a request is made

What is the root cause of that problem?

We're not setting caching configuration properly for the PDF in native, see here

What changes do you think we should make in order to solve the problem?

We need to set cache: true here. Now this presents a few problems:

1. The app will get bloated quickly since by default the cache won't expire so it will stay there forever, consuming storage
2. This presents a security risk since even if user logged out or the authToken becomes invalidated, the PDF will stay in the cache and be visible in the file system. This also contributes to the bloating issue since anytime the authToken normally rotated, it will becomes a new PDF URI and a new PDF cache. So the same PDF could be opened 10 times and cached 10 times with different authToken.

To fix this we need to

1. Set a reasonable TTL for the cache, using the expiration, let's say if 10 days, we can set expiration: 864000
2. In onLoadComplete, we'll receive the pdfPath, which we should store. And any time the authToken changes, that means the PDF URI with the earlier authToken that we cache is no longer valid, we should delete the cache in the pdfPath, this is also the approach recommended by the library author here

Detail implementation:

When users logout or authToken get changed we run the following code

import {unlink, exists} from 'react-native-fs';

const pdfPaths; // get from Onyx
        Promise.all(pdfPaths.map(async p=>{
            const exist = await exists(p)
            if(exist){
                return unlink(p);
            }
        }))

What alternative solutions did you explore? (Optional)

NA

[giltron]

Proposal

Please re-state the problem that we are trying to solve in this issue.

PDF files are not cached and are re-downloaded everytime.

What is the root cause of that problem?

The use of the react-native-pdf module is not using the caching options

What changes do you think we should make in order to solve the problem?

There are 4 items to address it reasonably and completely.

1. Add the cache: true in the source parameter as per react-native-pdf documentation. Note this will keep the files indefinitely without any further configuration.

 source={{uri: this.props.sourceURL, cache: true }}

2. If we need to address security of attachments staying on device after user logs out or different user logs in (note: user would need to physical device access) then the following can be done: (Note: PDFs downloaded go into the CacheDir as configured in react-native-pdf):

 source={{uri: this.props.sourceURL, cache: true, cacheFileName: cacheFileName}}

Where cacheFileName is specified by taking the original filename (or id) and prepending the sanitized user's id to it. This way we can easily clean any files not of the current user later by scanning the all files in the CacheDir on Login/Logout with react-native-blob-util RNFetchBlob.fs.lstat

3. As per the source parameter specified react-native-pdf:

We also should a reasonable expiration time to avoid the Apps storage from growing too large. For example 3 days which is 259200 seconds.

const expirationPDFTime =  259200; 

 source={{uri: this.props.sourceURL, cache: true, cacheFileName: cacheFileName, expiration: expirationPDFTime }}

4. We can also optionally add a generic method to cull the CacheDir if it reaches past a certain size. This can be down on login/logout or on a lengthy interval.

It will be similar to 3 but will involve checking the size of CacheDir folder usingRNFetchBlob.fs.readDir (or RNFetchBlob.fs.lstat and summing the file sizes ) and removing the oldest cached pdfs past an size threshold.

What alternative solutions did you explore? (Optional)

N/A

[getusha]

@tienifr how are we going to delete the cache after the user logs out? could you share a code snippet?

[garrettmknight]

Just a note that I've reproduced.

[tienifr]

@getusha I just added the detail implementation to delete the cache

[getusha]

The more detailed proposal, from @tienifr looks good to me. it makes sense to use the unlink function from react-native-fs to clear cache as we are using it already, we should open a discussion on slack to determine what the best expiration interval is.

🎀👀🎀 C+ reviewed!

[melvin-bot]

Triggered auto assignment to @MonilBhavsar, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[MonilBhavsar]

Sounds good. Do we have caching in web? why it is only reproducible on iOS native

[tienifr]

@MonilBhavsar In web, attachments are already cached by default so It doesn't happen on web

[MonilBhavsar]

Thanks! wanted to ensure, it's the browser only and we're not using any specific mechanism.

[melvin-bot]

📣 @getusha 🎉 An offer has been automatically sent to your Upwork account for the Reviewer role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job

[melvin-bot]

📣 @tienifr 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

[getusha]

Working on the PR on specific cases, @tienifr will update soon.

[garrettmknight]

PR is ready for review, still moving forward.

[garrettmknight]

PR is still in progress.

[melvin-bot]

⚠️ Looks like this issue was linked to a Deploy Blocker here

If you are the assigned CME please investigate whether the linked PR caused a regression and leave a comment with the results.

If a regression has occurred and you are the assigned CM follow the instructions here.

If this regression could have been avoided please consider also proposing a recommendation to the PR checklist so that we can avoid it in the future.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.4.45-6 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Fix: PDF is not cached #32678
• [CP Staging]Fix: App crashes when upload PDF #37484

If no regressions arise, payment will be issued on 2024-03-07. 🎊

For reference, here are some details about the assignees on this issue:

• @getusha requires payment automatic offer (Reviewer)
• @tienifr requires payment automatic offer (Contributor)

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@getusha] The PR that introduced the bug has been identified. Link to the PR:
• [@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@getusha] Determine if we should create a regression test for this bug.
• [@getusha] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@garrettmknight] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.4.46-2 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• [CP Staging]Fix: App crashes when upload PDF #37484

If no regressions arise, payment will be issued on 2024-03-11. 🎊

For reference, here are some details about the assignees on this issue:

• @getusha requires payment automatic offer (Reviewer)
• @tienifr requires payment automatic offer (Contributor)

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@getusha] The PR that introduced the bug has been identified. Link to the PR:
• [@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@getusha] Determine if we should create a regression test for this bug.
• [@getusha] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@garrettmknight] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[getusha]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[@getusha] The PR that introduced the bug has been identified. Link to the PR: This is a new feature
[@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/a
[@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/a
[@getusha] Determine if we should create a regression test for this bug. Yes
[@getusha] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.

Regression Test Proposal

1. Open any chat
2. Upload a large PDF file (15-20MB)
3. Open the PDF
4. Go back to the chat
5. Open the same PDF again
6. Verify that the PDF loads quickly

Do we agree 👍 or 👎

[garrettmknight]

Paid up at 50% based on the regression, closing!