[HOLD for payment 2023-10-18] [$500] Chat - Adding bracket after link removes link

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Action Performed:

1. Open the app
2. Open any report
3. Copy current report URL or copy any URL with at least one '/' after base URL
4. Paste in composer and send
5. Edit the message, at closing bracket at the end and save
6. Observe that link is now displayed as text

Expected Result:

Text outside of link format should not affect display of link in reports

Actual Result:

Closing bracket outside of link format makes the link appear as normal text

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.74-2

Reproducible in staging?: Yes

Reproducible in production?: Yes

If this was caught during regression testing, add the test name, ID and link from TestRail:

Email or phone of affected tester (no customers):

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos: Any additional supporting documentation

windows.chrome.App.removes.link.on.closing.bracket.after.link.mp4

android.chrome.App.removes.link.on.closing.bracket.after.link.mp4

Recording.99.mp4

Screen_Recording_20230928_091753_New.Expensify.1.mp4

braket.mp4

Expensify/Expensify Issue URL:

Issue reported by: @dhanashree-sawant

Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1695747673717249

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01c2d4a022033397cc
• Upwork Job ID: 1707112795924254720
• Last Price Increase: 2023-09-27
• Automatic offers:
• hoangzinh | Reviewer | 26974571
• eh2077 | Contributor | 26974572
• dhanashree-sawant | Reporter | 26974573

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01c2d4a022033397cc

[melvin-bot]

Triggered auto assignment to @Christinadobrzyn (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @hoangzinh (External)

[studentofcoding]

Proposal

Please re-state the problem that we are trying to solve in this issue

Chat - Adding bracket after link removes link

What is the root cause of that problem?

The MARKDOWN_LINK_REGEX on https://github.com/Expensify/expensify-common/blob/8bcd79047d8b6c5e718897edcacee102602a5f52/lib/ExpensiMark.js#L5-L7

does not account for the possibility of multiple closing brackets following the link, causing it to fail in recognizing and properly formatting the link.

What changes do you think we should make in order to solve the problem?

To solve this problem, we need to modify the regex that is responsible for link formatting to be

const MARKDOWN_LINK_REGEX = new RegExp(`\\[([^\\][]*(?:\\[[^\\][]*][^\\][]*)*)]\\(${MARKDOWN_URL_REGEX}\\)+(?![^<]*(<\\/pre>|<\\/code>))`, 'gi');

With adding +, allows the regex to stop capturing when it encounters multiple closing brackets, thus properly formatting the link, e.g https://new.expensify.com/r/538639631607142) will be rendered as a clickable hyperlink https://new.expensify.com/r/538639631607142 with an extra closing bracket ) following it.

What alternative solutions did you explore? (Optional)

None

cc: @hoangzinh @Christinadobrzyn

[kameshwarnayak]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Chat - Adding bracket after link removes link

What is the root cause of that problem?

The regular expression in removeLinksFromHtml is not accurate and it reads till we get a ).

App/src/libs/actions/Report.js

Line 1141 in e2c9203

const regex = new RegExp(`<(a)[^><]*href\\s*=\\s*(['"])(${Str.escapeForRegExp(link)})\\2(?:".*?"|'.*?'|[^'"><])*>([\\s\\S]*?)<\\/\\1>(?![^<]*(<\\/pre>|<\\/code>))`, 'g');

What changes do you think we should make in order to solve the problem?

Change the regular expression to read till we get the close tag. Basically replace the above line with the below line

        const regex = new RegExp(`<(a)[^><]*href\\s*=\\s*(['"])(${Str.escapeForRegExp(link)})\\2(?:".*?"|'.*?'|[^'"><])*>([\\s\\S]*?)<\\/\\1>(?![^<]*(<\\/pre>|<\\/code>))$`, 'g');

Result :

Screen.Recording.2023-09-28.at.10.39.05.AM.mov

What alternative solutions did you explore? (Optional)

NA

[Christinadobrzyn]

Thank you for the amazing report @dhanashree-sawant! I can reproduce this and it doesn't look like it's linked to any other jobs. I think this can be external?

[Christinadobrzyn]

@hoangzinh can you review the above proposals?

[hoangzinh]

@Christinadobrzyn yes, I will provide updates soon.

[hoangzinh]

Thank you for your proposal @studentofcoding. I tried to apply your suggestion but it makes current unit tests of expensify-common lib fail. (or if you think current unit tests has wrong expectation atm, please explain the reason). Thanks

[eh2077]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Adding bracket after link removes link.

What is the root cause of that problem?

The root cause of this issue is that, when saving draft, we call method getRemovedMarkdownLinks to extract links which include extra closing parentheses, see method extractLinksInMarkdownComment. Method extractLinksInMarkdownComment uses different way to extract links than method ExpensiMark.replace, see modifyTextForUrlLinks which handles ending closing parentheses.

So, method extractLinksInMarkdownComment returns

['https://google.com/sadsafds/saf)']

for draft comment

[test](https://google.com/sadsafds/saf))

And the original link therefor is removed by method removeLinksFromHtml.

What changes do you think we should make in order to solve the problem?

To fix this issue, we should let method extractLinksInMarkdownComment use same logic to extract links as method modifyTextForUrlLinks does.

We can convert comment to html and then extract link from <a> tag, so we won't include unwanted parentheses at the end of link. To achieve it, we can change following lines of method extractLinksInMarkdownComment

            const escapedComment = _.escape(comment);
            const matches = [...escapedComment.matchAll(MARKDOWN_LINK_REGEX)];

            // Element 1 from match is the regex group if it exists which contains the link URLs
            const links = _.map(matches, match => Str.sanitizeURL(match[2]));

to

            const html = this.replace(comment);
            const matches = [...html.matchAll(new RegExp(`<a href="${MARKDOWN_URL_REGEX}" target="_blank" rel="noreferrer noopener">`, 'gi'))];

            // Element 1 from match is the regex group if it exists which contains the link URLs
            const links = _.map(matches, match => Str.sanitizeURL(match[1]));

The above change passes all test cases of Expensify-common lib.

See demo

Screen.Recording.2023-10-01.at.7.30.10.PM.mov

What alternative solutions did you explore? (Optional)

N/A

[kameshwarnayak]

@hoangzinh Hope you had a look at my proposal

[hoangzinh]

sure thing @kameshwarnayak. I tested with your proposal and it works but I'm checking whether we're fixing it properly or it's just a workaround solution. I will update soon.

[hoangzinh]

@kameshwarnayak Thanks for your proposal and patience. I tested your proposal #28344 (comment) and it works. But I don't think we have a correct RCA. I'm inclining the RCA of @eh2077 here #28344 (comment). The parser.getRemovedMarkdownLinks func is returning an incorrect list of the links removed in a new comment (it should return an empty array because there is no removed link in this case), thus it makes our removeLinksFromHtml func work incorrectly.

App/src/libs/actions/Report.js

Lines 1161 to 1162 in b403ab5

	const removedLinks = parser.getRemovedMarkdownLinks(markdownOriginalComment, newCommentText);
	return removeLinksFromHtml(htmlForNewComment, removedLinks);

---

So I think we should fix our parser parser.getRemovedMarkdownLinks instead of removeLinksFromHtml

[hoangzinh]

@eh2077 Thanks for your proposal. Your proposal #28344 (comment) has a correct RCA and your solution looks good to me. It does not only fix the current issue but also potentially fix upcoming issues if we already fixed them in the ExpensiMark.replace util. Therefore I suggest we should go with his proposal.

🎀👀🎀 C+ reviewed

[melvin-bot]

Triggered auto assignment to @lakchote, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[lakchote]

Thanks for your detailed review @hoangzinh and attentiveness. I'm inclined to go proposal 2 too as it solves the core issue. Assigning now.

[melvin-bot]

📣 @hoangzinh 🎉 An offer has been automatically sent to your Upwork account for the Reviewer role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job

[melvin-bot]

📣 @eh2077 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

[melvin-bot]

📣 @dhanashree-sawant 🎉 An offer has been automatically sent to your Upwork account for the Reporter role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job

[eh2077]

I'll prepare PR for the App to bump the version a little bit later as now we're facing login API failed issue.

[eh2077]

@hoangzinh I found a failed test case of App after applying the fix. The current fix breaks the feature to remove link, like www.google.com, see below video

Screen.Recording.2023-10-06.at.4.39.43.PM.mov

I'll update soon to figure out a way to fix it.

[eh2077]

Please re-state the problem that we are trying to solve in this issue.

We fail to remove link after applying the fix. For example, we have following initial draft of a link

[www.google.com](https://www.google.com)

Then edit it to

www.google.com

and save it. The text www.google.com should be displayed as plain text.

RCA

We apply autolink rule here

const htmlString = this.replace(comment);

So, we can extract link https://www.google.com from input www.google.com because we apply autolink rule and sanitize URL.

Solution

A removed link in draft comment doesn't have complete markdown link syntax, [Link](url).

We only need to apply link rule here, like

            const htmlString = this.replace(comment, {filterRules: ['link']});

By doing this, we can pass all tests from both App and expensify-common repo.

Also see demo

Screen.Recording.2023-10-06.at.5.44.24.PM.mov

@hoangzinh @lakchote Please help to review it when you got time. I'll submit another PR to fix it if the above solution makes sense to you.

[hoangzinh]

Great @eh2077. LGTM ^

[eh2077]

@hoangzinh PR Expensify/expensify-common#585 is ready for review, thanks.

cc @lakchote

[hayata-suenaga]

@Luke9389 I was somehow auto-assigned to the app pr for this. I'm assigning you to the App PR

[eh2077]

@hoangzinh I updated and retested PR #28985 and it's ready for review, thanks!

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @eh2077 got assigned: 2023-10-02 07:15:36 Z
• when the PR got merged: 2023-10-09 16:37:35 UTC
• days elapsed: 5

On to the next one 🚀

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.80-3 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Fix: adding parentheses after link removes link #28985

If no regressions arise, payment will be issued on 2023-10-18. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @hoangzinh requires payment offer (Reviewer)
• @eh2077 requires payment offer (Contributor)
• @dhanashree-sawant requires payment offer (Reporter)

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@hoangzinh] The PR that introduced the bug has been identified. Link to the PR:
• [@hoangzinh] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@hoangzinh] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@hoangzinh] Determine if we should create a regression test for this bug.
• [@hoangzinh] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@Christinadobrzyn] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[Christinadobrzyn]

hey @hoangzinh can you let us know if there should be a regression test for this?

[hoangzinh]

yes, gonna do BZ checklist today

[Christinadobrzyn]

looks like payment will be the below breakdown - let me know if I'm missing anything

Payouts due:

Issue Reporter: $50 @dhanashree-sawant
Contributor: $500 @eh2077
Contributor+: $500 @hoangzinh

Eligible for 50% #urgency bonus? N - based on #28344 (comment)

Upwork job is here.

[hoangzinh]

BugZero Checklist:

• The PR that introduced the bug has been identified. Link to the PR: N/A, it's just an improvement to handle another sophisticated case in order to support markdown content.
• The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/A
• A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/A
• Determine if we should create a regression test for this bug. ❌ I think we don't need it, we have added unit tests for such cases inside expensify-common repo

[Christinadobrzyn]

No regressions so paying this out based on this payment structure. #28344 (comment)

No regression test to create
closing this!