[$500] Creating an optimistic action with a default avatar (SVG component) causes a failed write to Onyx storage

[marcaaron]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Action Performed:

Coming from: https://github.com/Expensify/Expensify/issues/313434#issuecomment-1747916128 an investigation into why storing data to Onyx can fail...

• I am not sure exactly. But there are several code flows where we might default to an SVG avatar in ReportUtils.js.

App/src/libs/ReportUtils.js

Lines 2660 to 2661 in 0d55bc9

	avatar: lodashGet(allPersonalDetails, [currentUserAccountID, 'avatar'], UserUtils.getDefaultAvatar(currentUserAccountID)),
	created: DateUtils.getDBTime(),

App/src/libs/ReportUtils.js

Lines 2641 to 2642 in 0d55bc9

	avatar: lodashGet(allPersonalDetails, [currentUserAccountID, 'avatar'], UserUtils.getDefaultAvatar(currentUserAccountID)),
	created: DateUtils.getDBTime(),

App/src/libs/ReportUtils.js

Lines 2602 to 2603 in 0d55bc9

	avatar: lodashGet(allPersonalDetails, [currentUserAccountID, 'avatar'], UserUtils.getDefaultAvatar(currentUserAccountID)),
	created: DateUtils.getDBTime(),

App/src/libs/ReportUtils.js

Lines 2485 to 2486 in 0d55bc9

	avatar: lodashGet(currentUserPersonalDetails, 'avatar', UserUtils.getDefaultAvatar(currentUserAccountID)),
	isAttachment: false,

App/src/libs/ReportUtils.js

Lines 2411 to 2412 in 0d55bc9

	avatar: lodashGet(currentUserPersonalDetails, 'avatar', UserUtils.getDefaultAvatar(currentUserAccountID)),
	created: DateUtils.getDBTime(),

App/src/libs/ReportUtils.js

Lines 2337 to 2338 in 0d55bc9

	avatar: lodashGet(currentUserPersonalDetails, 'avatar', UserUtils.getDefaultAvatar(currentUserAccountID)),
	isAttachment: false,

App/src/libs/ReportUtils.js

Lines 2298 to 2299 in 0d55bc9

	avatar: lodashGet(currentUserPersonalDetails, 'avatar', UserUtils.getDefaultAvatar(currentUserAccountID)),
	isAttachment: false,

App/src/libs/ReportUtils.js

Lines 1970 to 1971 in 0d55bc9

	avatar: lodashGet(allPersonalDetails, [currentUserAccountID, 'avatar'], UserUtils.getDefaultAvatarURL(currentUserAccountID)),
	created: DateUtils.getDBTime(),

App/src/libs/actions/Task.js

Lines 626 to 627 in 0d55bc9

	avatar: lodashGet(allPersonalDetails, [assigneeAccountID, 'avatar'], UserUtils.getDefaultAvatarURL(assigneeAccountID)),
	displayName: lodashGet(allPersonalDetails, [assigneeAccountID, 'displayName'], assigneeEmail),

• IndexedDB cannot save this to storage and throws an error.
• This error is confirmed throwing in the logs.

Expected Result:

1. The optimistic report action is saved correctly

Actual Result:

1. The user cannot persist the action in the device storage and an error is thrown

Workaround:

Yes - it will largely not be felt by many people.
However if they create the action while offline, then kill the app, then reopen then it could lead to some data loss or unexpected behavior

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: main
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
Expensify/Expensify Issue URL:
Issue reported by: @marcaaron
Slack conversation:

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01f221fc1261af7d28
• Upwork Job ID: 1710091720770150400
• Last Price Increase: 2023-12-12

[melvin-bot]

Triggered auto assignment to @twisterdotcom (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[pradeepmdk]

Proposal

Please re-state the problem that we are trying to solve in this issue.

svg component not able store the onyx.

What is the root cause of that problem?

App/src/libs/UserUtils.ts

Line 73 in 1eae800

function getDefaultAvatar(accountID = -1): React.FC<SvgProps> {

getDefaultAvatar will return the local reference as functions we can use as a component like <SVG/>

so we can't store the function in onyx store

What changes do you think we should make in order to solve the problem?

we need change getDefaultAvatarURL it will as a url string. and url image will be cached in the Image tag by default.

App/src/libs/UserUtils.ts

Line 91 in 1eae800

function getDefaultAvatarURL(accountID: string | number = '', isNewDot = false): string {

Alternative solution

Option 1) https://github.com/Expensify/App/blob/1eae8007bb79e7913b039182224c1ac3efc04e8c/src/libs/UserUtils.ts#L85C27-L85C57

here Avatar${accountIDHashBucket} just store in the onyx this key. and retrieve time we can use this string to retrieve the original image defaultAvatars[Avatar${accountIDHashBucket}];

Option 2)
@s77rt we are using https://www.npmjs.com/package/@svgr/webpack plugin for loading the svg file.

we can change this to any one of them

use: ['@svgr/webpack', 'file-loader']
@svgr/webpack this will return as a component
, if we use file-loader we will get like this /f69bf6542f9d0b403b9b1e3ec5679fa8.svg (like other file import .png, .jpeg)
url-loader we will get an base64
'raw-loader' we will get original svg string

These three we can able to store in the onyx.

we concat the above configuration because use an array only.

and if we change this configuration we do need some changes in the below components (Icon.js, GenericErrorPage.js, Header.js) because we are combining the other loader
ex:
import starUrl, { ReactComponent as Star } from './star.svg'

because some components directly render the SVG component

<LogoWordmark
                                height={30}
                                width={80}
                                fill={defaultTheme.textLight}
                            /> 

so like example we need to change the import statement.

[twisterdotcom]

@marcaaron can I just slap External on this?

[marcaaron]

yes. But I'm curious to understand whether we ever supported saving SVG to IndexedDB.

Maybe @thienlnam or @Beamanator remember the answer to this one. I think both of y'all have investigated some things related to Onyx and image storage IIRC.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01f221fc1261af7d28

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @s77rt (External)

[pine901]

My proposal
I think Onyx only accepts javascript code.
So traditionally you can't store SVG file to the Onyx.
In this case , you have to convert SVG to the base64 encoded code for saving SVG to Onyx.

import { SvgXml } from 'react-native-svg';
import { encode } from 'base-64';

const svgXml = '';
const svgBase64 = encode(svgXml);

import Onyx from 'react-native-onyx';

Onyx.set('svgImage', svgBase64);

[melvin-bot]

📣 @talent-dev610! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Make sure you've read and understood the contributing guidelines.
2. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
3. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
4. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[thienlnam]

But I'm curious to understand whether we ever supported saving SVG to IndexedDB.

This was the PR where we added file handling in Onyx - I haven't been following along with the changes that we've made since then but there's perhaps a chance that indexdb doesn't support storing SVGs or maybe we need to do something to the SVG to be able to store it as a blob

[Beamanator]

I think I've seen this issue before when we tried to store default avatars in Onyx & from what i've seen, there's always errors when we try to store SVGs in Onyx - cc @grgia who also worked on avatars a lot in the past

[s77rt]

@pradeepmdk Thanks for the proposal. I don't think we can relay on the url as being offline you won't be able to access that url.

[s77rt]

@talent-dev610 Thanks for the proposal. Encoding will work but it will add more space and computation. Regardless your proposal is not complete as it's not clear when to encode / decode. Is this supposed to be done in react-native-onyx?

[pradeepmdk]

@s77rt Proposal updated #28885 (comment)

[s77rt]

@pradeepmdk Thanks for the update. I think option 1 will do it but we will have a lot of code to update / test. Ideally we would want functions to be supported in Onyx store but we will have serialize / deserialize data internally.

e.g.:

// Insert phase
myFun = getDefaultAvatar(1);
if (typeof myFun === 'function') {
    myFun = myFun.toString();
}

// Retrieval phase
myFun = eval('(' + myFun + ')');

But eval is evil

[pradeepmdk]

Ideally we would want functions to be supported in Onyx store but we will have serialize / deserialize data internally.

serialize / deserialize we are going to do in App/react-native-onyx ?

[pradeepmdk]

if we use file-loader we will get like this /f69bf6542f9d0b403b9b1e3ec5679fa8.svg (like other file import .png, .jpeg)

@s77rt
we can use this as well

but we have an issue on offline in web(on native it will work fine). because offline on the web it will fail to load to this reference.
so we need to implement the service worker to cache those files.

[s77rt]

@pradeepmdk

serialize / deserialize we are going to do in App/react-native-onyx ?

If we are going with the serialize / deserialize path then it must be done within react-native-onyx.

I'm not sure about the file loader as we currently use the imported svg files as components.

[s77rt]

Not overdue. Still looking for proposals

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[s77rt]

@pradeepmdk I think just a string is enough if there is no benefit in using the Ast format.

[pradeepmdk]

@s77rt so what is next progress here ?

[s77rt]

@pradeepmdk Can you please explain how you will achieve the serialize/deserialize functionality in react-native-onyx?

[s77rt]

Not overdue. Still looking for proposals

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[melvin-bot]

@twisterdotcom, @s77rt Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[s77rt]

Not overdue. Still looking for proposals

[pine901]

My proposal
I think Onyx only accepts javascript code.
So traditionally you can't store SVG file to the Onyx.
In this case , you have to convert SVG to the base64 encoded code for saving SVG to Onyx.

import { SvgXml } from 'react-native-svg';
import { encode } from 'base-64';

const svgXml = '';
const svgBase64 = encode(svgXml);

import Onyx from 'react-native-onyx';

Onyx.set('svgImage', svgBase64);

[s77rt]

@talent-dev610 Please follow the proposal template and submit a more close to proof of concept solution for serialize/deserialize approach inside react-native-onyx.

[twisterdotcom]

Not overdue still waiting on proposals.

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[s77rt]

Still waiting for proposals

[s77rt]

Same ^

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[s77rt]

Still looking for proposals

[twisterdotcom]

Okay, I will allow @marcaaron to reopen if he thinks differently, but given we are moving towards a world where only real world reported bugs will be fixed, I'm going to close this.