Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fast imports: [maybe] put progress callback url into spec on s3 #10996

Open
Tracked by #10188
NanoBjorn opened this issue Feb 26, 2025 · 0 comments
Open
Tracked by #10188

fast imports: [maybe] put progress callback url into spec on s3 #10996

NanoBjorn opened this issue Feb 26, 2025 · 0 comments
Labels
c/storage/pageserver Component: storage: pageserver t/feature Issue type: feature, for new features or requests

Comments

@NanoBjorn
Copy link
Contributor

NanoBjorn commented Feb 26, 2025

There is import_pgdata_upcall_api in pageserver config and points to cplane where to send import progress when pageservers are done. It is ok for most environments to have single url per pageserver, apart from local setup, where we share pageservers between docker cplane and k8s cplane regions.

I understand that it might be too narrow use-case, but maybe we can consider putting callback url into the spec? This way pageserver would always know where to call about progress from the spec itself and it would be convenient for all kinds of setups. Maybe, it is also worth keeping config option as a default, but being able to override it from spec might be convenient.

Spec:

pub struct Spec {
pub project_id: String,
pub branch_id: String,
}

Smth like callback_progress_url makes sense, where we do POST as now.

@NanoBjorn NanoBjorn changed the title [maybe] put progress callback url into spec on s3 fast imports: [maybe] put progress callback url into spec on s3 Feb 26, 2025
@VladLazar VladLazar added t/feature Issue type: feature, for new features or requests c/storage/pageserver Component: storage: pageserver labels Feb 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/storage/pageserver Component: storage: pageserver t/feature Issue type: feature, for new features or requests
Projects
None yet
Development

No branches or pull requests

2 participants