[$500] Status - Error message & emoji status remains when clear status timer has elapsed

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Version Number: 1.4.13-0
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:

Issue found when executing PR #24620

Action Performed:

Precondition: user should be Signed In

1. Open https://staging.new.expensify.com/
2. Click on Profile icon > Status
3. Set any status emoji and click on Save
4. Click on Status
5. Click on "Clear after" > Custom
6. Set the time 5 min ahead and click on Save button
7. Wait about 6 min or more
8. Open the Clear status option

Expected Result:

The status should be cleared when the clear timer has elapsed

Actual Result:

Please choose a time at least one minute ahead" error message is displayed, and the emoji status remains when the clear status timer has elapsed

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

Bug6313647_1702589216993.Screen_Recording_2023-12-14_at_19.48.49.mp4

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~013df2712d0a3d5327
• Upwork Job ID: 1737192168446410752
• Last Price Increase: 2024-01-09

[github-actions]

👋 Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open `StagingDeployCash` deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

1. Identify the pull request that introduced this issue and revert it.
2. Find someone who can quickly fix the issue.
3. Fix the issue yourself.

[melvin-bot]

Triggered auto assignment to @deetergp (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

[stitesExpensify]

Looking into this

[melvin-bot]

⚠️ Looks like this issue was linked to a Deploy Blocker here

If you are the assigned CME please investigate whether the linked PR caused a regression and leave a comment with the results.

If a regression has occurred and you are the assigned CM follow the instructions here.

If this regression could have been avoided please consider also proposing a recommendation to the PR checklist so that we can avoid it in the future.

[abzokhattab]

Proposal

Please re-state the problem that we are trying to solve in this issue.

The status is not sent to the backend in UTC format

What is the root cause of that problem?

we dont convert the clearAfter Date to UTC before its sent to the backend

What changes do you think we should make in order to solve the problem?

There is two ways here:

1. either we convert the dates inside the dateutil functions so that the User.js inside the actions folder and the status components don't have to deal with the conversion of timezone .. however, this solution requires changes in multiple areas and might cause breaking changes. (you can find more details below)
2. or we convert the date into UTC only on sending the data to the database which means the currentUserPersonalDetails will always have the status as utc however the DRAFT status will be in local date because the draft status is not yet sent to the server so it shouldnt be parsed as UTC.

with that being said the changes of the second solution should not introduce breaking change so lets dig into it:

first, when writing the data to the server we need to convert it into UTC format so we need to change the update function to:

function updateCustomStatus(status) {
    const clearAfterInUtc = DateUtils.formatWithUTCTimeZone(status.clearAfter, 'yyyy-MM-dd HH:mm:ss');
    const newStatus = {
        text: status.text,
        emojiCode: status.emojiCode,
        clearAfter: clearAfterInUtc,
    };
    API.write('UpdateStatus', newStatus, {
        optimisticData: [
            {
                onyxMethod: Onyx.METHOD.MERGE,
                key: ONYXKEYS.PERSONAL_DETAILS_LIST,
                value: {
                    [currentUserAccountID]: {
                        status: newStatus,
                    },
                },
            },
        ],
    });
}

then for the reading part, we need to add a new function inside the DateUtils file that converts the UTC string into a local date string given a certain format (this is the opposite of formatWithUTCTimeZone)

so i have defined this funciton :

function formatUTCToLocal(dateString: string, dateFormat: string = CONST.DATE.FNS_FORMAT_STRING) {
    if (dateString === '' || !dateString) {
        return dateString;
    }

    const date = parse(dateString, dateFormat, new Date());
    // Check if the date is valid
    if (!isValid(date)) {
        return '';
    }

    const utcDate = zonedTimeToUtc(date, 'UTC');
    const localDate = zonedTimeToUtc(utcDate, timezone.selected);
    // the timezone.selected is the timezone that the user selected at profile/timezone
     return tzFormat(localDate, dateFormat, {timeZone: timezone.selected});
}

then we have two solutions:

1. either to add a new middleware function that would update the status.clearAfter data returned from the server to local time (this way is preferrable)

to do so we need to add a new middleware to this folder: https://github.com/Expensify/App/blob/82b76c1a36cf1a01b00c14510a5812a591e7f2fd/src/libs/Middleware
then we add the following middleware file:

FormatStatusClearAfter.ts

 import DateUtils from '@libs/DateUtils';
import type {Middleware} from '@libs/Request';
import ONYXKEYS from '@src/ONYXKEYS';
import type {PersonalDetailsList} from '@src/types/onyx';

/**
 * Middleware function to format the 'clearAfter' field in personal details list.
 *
 * @param {any} requestResponse - The response from the request.
 * @param {any} request - The original request data.
 * @param {boolean} isFromSequentialQueue - Flag to indicate if the request is from a sequential queue.
 * @returns {Promise<any>} - The processed request response.
 */
const formatStatusClearAfter: Middleware = (requestResponse) =>
    requestResponse.then((response) => {
        const responseOnyxData = response?.onyxData ?? [];
        responseOnyxData.forEach((onyxData) => {
            if (onyxData.key !== ONYXKEYS.PERSONAL_DETAILS_LIST) {
                return;
            }

            const personalDetailsList = onyxData.value as PersonalDetailsList;
            Object.keys(personalDetailsList).forEach((accountID) => {
                const status = personalDetailsList[accountID]?.status;
                const clearAfter = status?.clearAfter;

                if (!clearAfter || clearAfter === '') {
                    return;
                }

                if (status) {
                    status.clearAfter = DateUtils.formatUTCToLocal(clearAfter, 'yyyy-MM-dd HH:mm:ss');
                }
            });
        });
        return response;
    });

export default formatStatusClearAfter;

then we need to add the new middleware to the index file, then we need to register it in the app file:

Request.use(Middleware.FormatClearAfterInPersonalDetails);

this will convert the value returned from the backend to a local date in the middle without the need to update the time in each component that uses it

2. or we update the time to localdate in the component that use the status.
   so we need to use the prev function in the StatusClearAfterPage, StatusPage, ReportActionItemSingle, and OptionRowLHN

like this:

    const clearAfter = DateUtils.formatUTCToLocal(lodashGet(currentUserPersonalDetails, 'status.clearAfter', ''), 'yyyy-MM-dd HH:mm:ss');

Alternatively

Converting the date inside DateUtils

This will require quite a lot of change here are the areas where we need to consider converting from UTC to local when showing the data to the user and visa versa when writing the date into the database:

will introduce two functions convertUtcToLocal and convertLocalToUtc Both take a string date and input format and return the date in the same format but in utc or local date ..

Here are the places where we need to convert the input string from utc to local time :

1. App/src/libs/DateUtils.ts

   Line 493 in 5049f8c

   function getStatusUntilDate(inputDate: string): string {

2. App/src/libs/DateUtils.ts

   Line 477 in 5049f8c

   function getLocalizedTimePeriodDescription(data: string): string {

3. App/src/libs/DateUtils.ts

   Line 629 in 5049f8c

   const isTimeAtLeastOneMinuteInFuture = ({timeString, dateTimeString}: {timeString?: string; dateTimeString: string}): boolean => {

4. App/src/libs/DateUtils.ts

   Line 400 in 5049f8c

   function getEndOfToday(): string {

5. App/src/libs/ValidationUtils.ts

   Line 392 in a3c4288

   const validateDateTimeIsAtLeastOneMinuteInFuture = (data: string): {dateValidationErrorKey: string; timeValidationErrorKey: string} => {

6. App/src/libs/DateUtils.ts

   Line 529 in 5049f8c

   const combineDateAndTime = (updatedTime: string, inputDateTime: string): string => {

And here are the places where we need to convert the UTC to localtime to show the date to the user:

1. App/src/pages/settings/Profile/CustomStatus/SetTimePage.js

   Lines 42 to 47 in 03685bb

   	<TimePicker
   	inputID="timePicker"
   	defaultValue={clearAfter}
   	style={styles.flexGrow1}
   	onSubmit={onSubmit}
   	/>

2. App/src/pages/settings/Profile/CustomStatus/SetDatePage.js

   Lines 60 to 65 in 03685bb

   	>
   	<DatePicker
   	inputID="dateTime"
   	label={translate('statusPage.date')}
   	defaultValue={DateUtils.extractDate(customClearAfter)}
   	minDate={new Date()}

[stitesExpensify]

@abzokhattab have you tested that? I don't think that is the correct cause. We don't need to set the status when we are just creating a draft, and when you actually click save it looks like the request is working properly.

When you refresh the page after your timer, the status disappears. That leads me to believe that the issue is that we are not pushing the updates to all users properly.

IMO this is not a blocker even though it is genuinely broken. The Status feature that we merged was massive, and reverting it is not feasible. I will work on a fix in web-e first thing next week

[abzokhattab]

@stitesExpensify , I found out the comment wasn't working, so I hid it. After testing, I think the problem is related to time zones. I tried using a date from 6 hours ago, but the status was not cleared after that time had passed. However, when I checked it just now, the status was updated. It seems there's a mismatch in how the time zone is interpreted. I suspect the server, where the backend is running, is in the USA, causing the entered date to be parsed as a US date. This might be delaying the job trigger by a few hours.

My suggestion is to send the date from the frontend as UTC. This way, the backend can accurately parse and handle it. Additionally, it's worth noting that the backend doesn't have information about the client's timezone. The time string is sent to the backend without specifying the zone. In the provided data sample for updating the status, there is no information about the client's browser timezone:

To address this, two actions are recommended:

1. The frontend should either send the timezone along with the status time, OR the frontend should parse it as UTC before sending it to the backend OR we can use the user's timezone which is saved under profile/timezone but this will require some changes in parsing the dates because now we just call new Date() in parsing the clearAfterTime value.
2. The backend should parse the incoming date as UTC.

[abzokhattab]

Just tried to test it again .. this time the status is not cleared 😂 even after almost 11 hours of the custom date .. so i guess the backend doesnt clear it.

If we dont want to tackle the clearing process in the frontend we can prevent the status from being shown up in case the clearTime has passed.

[abzokhattab]

If we have decided to tackle the status clearing process in the frontend, here is a blueprint for how we can achieve that :

inside the User.js file we need to add this function that would return the current status and (update it if the clear date has passed):

function getUserStatus(userPersonalDetails) {
    const statusEmojiCode = lodashGet(userPersonalDetails, 'status.emojiCode', '');
    const statusText = lodashGet(userPersonalDetails, 'status.text', '');
    const statusClearAfter = lodashGet(userPersonalDetails, 'status.clearAfter', '');

    if (statusClearAfter !== '' && new Date(statusClearAfter) < new Date()) {
        if (userPersonalDetails.accountID === currentUserAccountID) {
            clearCustomStatus();
            clearDraftCustomStatus();
        }
        return {
            emojiCode: '',
            text: '',
            clearAfter: '',
        };
    }
    return {
        emojiCode: statusEmojiCode,
        text: statusText,
        clearAfter: statusClearAfter,
    };
}

then we need to replace this:

with:

const statusData = User.getUserStatus(currentUserPersonalDetails);
const currentUserEmojiCode = lodashGet(statusData, "emojiCode", "");
const currentUserStatusText = lodashGet(statusData, "text", "");
const currentUserClearAfter = lodashGet(statusData, "clearAfter", "");

same here:

change it to:

const status = User.getUserStatus(currentUserDetails);
const emojiCode = status.emojiCode;

and here change this:

to:

const { avatar, login, pendingFields, fallbackIcon } = personalDetails[actorAccountID] || {};
const status = User.getUserStatus(personalDetails[actorAccountID]);

and change this:

to:

const emojiStatus = User.getUserStatus(currentUserPersonalDetails).emojiCode;

and finally change this:

to

const statusData = User.getUserStatus(optionItem);
const emojiCode = lodashGet(statusData, "emojiCode", "");
const statusText = lodashGet(statusData, "text", "");
const statusClearAfterDate = lodashGet(statusData, "clearAfter", "");

OR

alternatively we can make the above change on connecting to the App so it will update status if needed as we currently do with the timezone

Result

Screen.Recording.2023-12-17.at.2.26.59.AM.mov

[Pujan92]

Seems a backend issue where the status is not cleared after time expiration and the pusher event isn't receieved.

[stitesExpensify]

Seems a backend issue where the status is not cleared after time expiration and the pusher event isn't receieved.

I agree

[stitesExpensify]

I think there are actually 2 issues here. One is that we aren't sending the time to the server in UTC (frontend bug), and the second is that the user who set the status does not get a pusher notification (backend bug).

When the bedrock job gets created on the server, it uses local time so if you are west of UTC it runs instantly and clears your status, and if you are east of UTC it happens late.

Your status also disappears for other users, but not you.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~013df2712d0a3d5327

[abzokhattab]

will be ready tonight i believe

[dylanexpensify]

Nice! @abdulrahuman5196 to get ready for review

[abzokhattab]

PR changes are ready .. will post the videos today.

[dylanexpensify]

bump @abzokhattab

[abzokhattab]

The pr is ready.. thanks

[dylanexpensify]

Nice, thank you! @abdulrahuman5196 to review!

[dylanexpensify]

bump @abdulrahuman5196

[abdulrahuman5196]

We are working on PR review. Still issues to solve.

[deetergp]

Any progress on that PR?

[abdulrahuman5196]

We are facing issues currently, Waiting on @stitesExpensify response for here #35976 (comment)

[dylanexpensify]

@stitesExpensify any chance you've been able to tackle/ @abdulrahuman5196 ETA?

[stitesExpensify]

Not yet. This is held on me but it is not a priority at the moment unfortunately

[dylanexpensify]

Closing as low priority

[stitesExpensify]

@dylanexpensify I think that we should pay out for this issue. There was a lot of back and forth as well as development/reviewing time

[dylanexpensify]

Apologies both! Yes, will do!

[dylanexpensify]

Payment summary:

• Contributor: @abzokhattab $500
• Contributor+: @abdulrahuman5196 $500

Please request/apply!

[abzokhattab]

Thank you @dylanexpensify i have just applied

[dylanexpensify]

offer sent @abzokhattab

[abzokhattab]

Awesome .. just accepted the offer

[abdulrahuman5196]

@dylanexpensify I have also applied.

[abdulrahuman5196]

@dylanexpensify Gentle ping on the above

[abdulrahuman5196]

Received payment