Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export task stops on "key does not exist" with no way to know if it completed or not #536

Closed
jaywink opened this issue Jan 2, 2024 · 3 comments

Comments

@jaywink
Copy link
Contributor

jaywink commented Jan 2, 2024

image

It looks like an export task will just abort if it can't find a key. This would be ok if the task itself would go into some kind of error state, but polling the task just shows it as finished:

{
	"task_id": 2,
	"task_name": "export_data",
	"params": {
		"export_id": "redacted",
		"include_s3_urls": false,
		"server_name": "domain.tld"
	},
	"start_ts": 1704034098316,
	"end_ts": 1704034914159,
	"is_finished": true
}

It's impossible for a script to follow whether the export finished correctly, except by tailing the logs of the container.

Suggestions:

  • An export should try to export everything without stopping on one missing key?
  • An is_success or similar boolean on the task to indicate whether the task finishes successfully or not?
@jaywink
Copy link
Contributor Author

jaywink commented Jan 3, 2024

The particular key not found error seems to come from the AppendMedia call btw: https://github.com/turt2live/matrix-media-repo/blob/master/archival/entity_export.go#L69

@turt2live
Copy link
Member

#538 at least exposes the error

@turt2live
Copy link
Member

The key not found error is a bit more concerning, as it means the media was likely deleted without the knowledge of MMR (or worse, there's a bug in MMR). I feel it's correct to abort on that error, so for now I don't intend to make it non-fatal.

If it turns out the media was deleted because of a MMR bug, please open a new issue with detailed reproduction steps (or a security issue if a user can invoke it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants