Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exposing the node capacity via chitchat #4083

Merged
merged 3 commits into from
Nov 9, 2023
Merged

Exposing the node capacity via chitchat #4083

merged 3 commits into from
Nov 9, 2023

Conversation

fulmicoton
Copy link
Contributor

No description provided.

@fulmicoton fulmicoton force-pushed the issue/4082-capacity-in-node-config branch 5 times, most recently from b58eee0 to dacaa38 Compare November 6, 2023 04:33
@fulmicoton fulmicoton requested a review from fmassot November 6, 2023 04:34
@fulmicoton fulmicoton force-pushed the issue/4082-capacity-in-node-config branch 2 times, most recently from 9eff4dc to f3e5b7e Compare November 8, 2023 03:27
@fulmicoton fulmicoton requested a review from guilload November 8, 2023 03:28
@fulmicoton fulmicoton force-pushed the issue/4082-capacity-in-node-config branch 3 times, most recently from 881aac0 to 2ee9fca Compare November 8, 2023 05:56
guilload
guilload approved these changes Nov 8, 2023
View reviewed changes
Copy link
Member

@guilload guilload left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: CpuCapacity is also sometimes used for measuring a load or a threshold. It might be better to use CpuUnits for the type, and let the variable names tell us whether it's a capacity, load, etc.

quickwit/quickwit-config/src/node_config/mod.rs Outdated Show resolved Hide resolved
quickwit/quickwit-config/src/node_config/mod.rs Show resolved Hide resolved
quickwit/quickwit-config/src/node_config/mod.rs Outdated Show resolved Hide resolved
quickwit/quickwit-control-plane/src/indexing_scheduler/mod.rs Outdated Show resolved Hide resolved
quickwit/quickwit-control-plane/src/indexing_scheduler/scheduling/mod.rs Outdated Show resolved Hide resolved
quickwit/quickwit-control-plane/src/indexing_scheduler/scheduling/mod.rs Outdated Show resolved Hide resolved
quickwit/quickwit-proto/src/indexing/mod.rs Show resolved Hide resolved
@fulmicoton
Exposing the node capacity via chitchat
eca3978
@fulmicoton
Refactoring introducing the Cpu capacity.
d1eb143 
This new CpuCapacity type is used to both expressed
the available capacity in indexer nodes, and
the load associated on shards.

As a result, the quantity we use for the latter
is semantically shifted from the CPU spent on the
indexer actor, to the estimated CPU cost of an
entire pipeline. (x4)
@fulmicoton fulmicoton force-pushed the issue/4082-capacity-in-node-config branch from 870904f to aa540f9 Compare November 9, 2023 02:33
@fulmicoton fulmicoton enabled auto-merge (squash) November 9, 2023 02:33
@fulmicoton fulmicoton force-pushed the issue/4082-capacity-in-node-config branch from aa540f9 to 7185fb1 Compare November 9, 2023 03:25
@fulmicoton fulmicoton merged commit 2613aee into main Nov 9, 2023
@fulmicoton fulmicoton deleted the issue/4082-capacity-in-node-config branch November 9, 2023 03:48
@fmassot fmassot mentioned this pull request Nov 10, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guilload guilload guilload approved these changes

@fmassot fmassot Awaiting requested review from fmassot

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fulmicoton @guilload