Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Round-robin mail details calls when exporting #8421

Open
wants to merge 34 commits into
base: dev-mail
Choose a base branch
from

Conversation

hrb-hub
Copy link
Contributor

@hrb-hub hrb-hub commented Jan 29, 2025

Close #8411

@hrb-hub hrb-hub linked an issue Jan 29, 2025 that may be closed by this pull request
@hrb-hub hrb-hub force-pushed the 8411-distribute-export-mail-details-load branch 2 times, most recently from 60473b3 to 48dfdca Compare January 29, 2025 12:58
@hrb-hub hrb-hub force-pushed the dev-mail branch 2 times, most recently from d02721e to ce3139f Compare February 20, 2025 12:32
@hrb-hub hrb-hub force-pushed the 8411-distribute-export-mail-details-load branch 3 times, most recently from 9973e82 to 2fe4600 Compare February 21, 2025 16:00
@hrb-hub
Copy link
Contributor Author

hrb-hub commented Feb 21, 2025

tutadb: !732

Adds TutanotaModelV84

Close #8373

Co-authored-by: Willow <[email protected]>
Co-authored-by: bir <[email protected]>
@@ -163,7 +163,7 @@ export class MailExportController {
break
}

const downloadedMailDetails = await this.mailExportFacade.loadMailDetails(downloadedMails)
const downloadedMailDetails = await this.mailExportFacade.loadMailDetails(downloadedMails, this.getServerUrl())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In most cases MailDetails will not use it because most mails are not drafts. I wonder if it makes sense to not rotate the URL just for MailDetails and maybe sure the same one as in previous step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, I don't quite understand what you mean 😅
This commit also rotates URL for mail detail blobs. Or were we not supposed to do that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most requests to loadMailDetails() will not use the URL that we give to it. It only has effect when we are downloading MailDetails of drafts. But each call to getServerUrl() will rotate the server URL. In combination this means that we will effectively use 2 servers for most requests instead of 3. That's why I thought that maybe we shouldn't call getServerUrl() just for mail details since it most likely won't be used anyway and maybe we can just use the URL from a previous step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless our implementation is incorrect, I believe that loadMailDetails() also uses the passed in URL to download non-draft MailDetails. We do so by favouring the passed in URL when choosing servers in loadMultipleBlobElements, or were we not supposed to do that?

let serversToTry = blobServerAccessInfo.servers
if (opts.baseUrl) {
	const preferredServer = blobServerAccessInfo.servers.find((server) => server.url === opts.baseUrl)

	if (preferredServer) {
		// preferredServer takes precedence over the rest
		serversToTry = [preferredServer].concat(blobServerAccessInfo.servers.filter((server) => server.url !== opts.baseUrl))
	}
}

Or is our implementation incorrect?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, dang, sorry, you are right, I remember I was confused why we need to do it but I think we should leave it like you wrote it. Sorry for the confusion.

charlag and others added 18 commits March 3, 2025 09:57
Re-implement downloading of mails during search indexing:
 - download mail details instead of mails from mail bags to ensure
 correctness of date range
 - always download MailSetEntry's in full chunks to avoid cache-related
 issues

Close #8564

Co-authored-by: bir <[email protected]>
Co-authored-by: wrd <[email protected]>
Using direct array accesses makes the code harder to understand.

Change it to use listIdPart/elementIdPart so the intention is clearer.
Instead of indexing day-by-day, we can process after we've reached some
threshold. We should still iterate by day, but we should index multiple
days at once.

This reduces the number of requests by over 80% on some mailboxes, and
it may cut loading time in half in such cases.

Closes #8599

Co-authored-by: bir <[email protected]>
- This list model groups all emails by conversation in the mail list
- Also adds setting for grouping by conversation (i.e. enables this)
- The setting is overridden for SENT and DRAFT folder types (it uses the
  other per-mail list model instead)
- Adds MailSetListModel which provides a common interface for the
  MailViewModel so that additional list models can be created; it is
  used for MailListModel and the new ConversationListModel, but it can
  be extended to even more list models in the future if it is ever
  desired.

Note: This does not include actions. Any action you do (delete, archive,
etc.) will be applied to the selected email. The purpose of this commit
is to just implement the list model. See #5051 for more information.

Closes #8223

Co-authored-by: bir <[email protected]>
Co-authored-by: ivk <[email protected]>
Makes it different from MailListView component so it is easier to search
This commit moves the service call to MailFacade and adds tests, too.

Closes #8507
When mails in list are grouped by conversation, reply/forward apply to
the latest mail in the conversation.

Close #8462

Co-authored-by: bir <[email protected]>
Co-authored-by: ivk <[email protected]>
hrb-hub and others added 15 commits March 5, 2025 10:36
Co-authored-by: hrb-hub <[email protected]>
Co-authored-by: hrb-hub <[email protected]>
Move moves the whole conversation except for mails in sent folder.

Trash moves the whole conversation to trash.

Delete only happens when:
 - directly in trash or spam folder
 - in search, when all mails are deleted.

Delete has a separate icon.

Close #8508
Close #8509
Close #8463
Close #8613
Close #8634

Co-authored-by: paw-hub <[email protected]>
Co-authored-by: ivk <[email protected]>
It is a bit annoying to have to click here.

Co-authored-by: bir <[email protected]>
@hrb-hub hrb-hub force-pushed the 8411-distribute-export-mail-details-load branch from 2fe4600 to 267d28a Compare March 5, 2025 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mailbox export load not distributed for mail details
5 participants