Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the take over mode to the input implementation #36777

Open
rdner opened this issue Oct 6, 2023 · 0 comments
Open

Move the take over mode to the input implementation #36777

rdner opened this issue Oct 6, 2023 · 0 comments
Assignees
Labels
enhancement Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@rdner
Copy link
Member

rdner commented Oct 6, 2023

Describe the enhancement:

Currently, the take-over mode used for migrating log inputs to filestream inputs is running before we start any inputs and migrates everything at once for the best possible performance.

We might be able to move this logic to the filestream's prospector when it starts instead. The prospector has access to the state via this interface:

type StateMetadataUpdater interface {
// FindCursorMeta retrieves and unpacks the cursor metadata of an entry of the given Source.
FindCursorMeta(s Source, v interface{}) error
// UpdateMetadata updates the source metadata of a registry entry of a given Source.
UpdateMetadata(s Source, v interface{}) error
// Remove marks a state for deletion of a given Source.
Remove(s Source) error
// ResetCursor resets the cursor in the registry and drops previous state
// updates that are not yet ACKed.
ResetCursor(s Source, cur interface{}) error
}

We might want to extend the interface and add the take over function.

This will allow us to support take-over for dynamic use-cases but would most likely affect the performance.

Describe a specific use case for the enhancement or feature:

If we allow dynamic take-over it will allow us to use it in autodiscover and running under elastic-agent.

@rdner rdner added enhancement Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Oct 6, 2023
@rdner rdner self-assigned this Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

1 participant