[sn-platform][sn-platform-slim]: upgrade pulsar to 3.0 and enable 3.0 new features #1083
Conversation
Signed-off-by: ericsyh <[email protected]>
Signed-off-by: ericsyh <[email protected]>
Signed-off-by: ericsyh <[email protected]>
charts/sn-platform-slim/values.yaml
Outdated
| configData: {} | ||
| configData: | ||
| # Enable DirectIO by default | ||
| PULSAR_PREFIX_dbStorage_directIOEntryLogger: "true" |
There was a problem hiding this comment.
Pulsar 3.0 may lose data if it enables direct IO. This PR will fix it.
Signed-off-by: ericsyh <[email protected]>
charts/sn-platform-slim/values.yaml
Outdated
| delayedDeliveryTrackerFactoryClassName: "org.apache.pulsar.broker.delayed.BucketDelayedDeliveryTrackerFactory" | ||
| # Enable transaction buffer segmented snapshot | ||
| transactionBufferSegmentedSnapshotEnabled: "true" | ||
| transactionBufferSnapshotSegmentSize: "262144" |
There was a problem hiding this comment.
We don't need set this one, just use the default value that pulsar provided.
charts/sn-platform/values.yaml
Outdated
| delayedDeliveryTrackerFactoryClassName: "org.apache.pulsar.broker.delayed.BucketDelayedDeliveryTrackerFactory" | ||
| # Enable transaction buffer segmented snapshot | ||
| transactionBufferSegmentedSnapshotEnabled: "true" | ||
| transactionBufferSnapshotSegmentSize: "262144" |
There was a problem hiding this comment.
We don't need set this one, just use the default value that pulsar provided.
| managedLedgerDefaultEnsembleSize: "3" | ||
| managedLedgerDefaultWriteQuorum: "3" | ||
| managedLedgerDefaultAckQuorum: "2" | ||
| PULSAR_PREFIX_systemTopicEnabled: "true" | ||
| PULSAR_PREFIX_topicLevelPoliciesEnabled: "true" | ||
| # Enable the new delayed message | ||
| delayedDeliveryTrackerFactoryClassName: "org.apache.pulsar.broker.delayed.BucketDelayedDeliveryTrackerFactory" |
There was a problem hiding this comment.
Well, I think maybe we should support dynamic calculate of the segment size or bucket size by cluster resources in the future. :)
/cc @coderzc WDYT?
There was a problem hiding this comment.
Yes, but we need to design a set of calculation formulas.
charts/sn-platform/values.yaml
Outdated
| transactionBufferSegmentedSnapshotEnabled: "true" | ||
| transactionBufferSnapshotSegmentSize: "262144" | ||
| # Enable the new Load Balancer | ||
| loadManagerClassName: "org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl" |
There was a problem hiding this comment.
To use ExtensibleLoadManagerImpl, we recommend upgrading the pulsar to 3.0.x before enabling the new load balancer.
cc @heesung-sn
Signed-off-by: ericsyh <[email protected]>
Signed-off-by: ericsyh <[email protected]>
Signed-off-by: ericsyh <[email protected]>
charts/sn-platform/values.yaml
Outdated
| @@ -987,6 +987,8 @@ bookkeeper: | |||
| gcLoggingOptions: [] | |||
| custom: {} | |||
| configData: {} | |||
| # For Pulsar 3.0, please do not enable Direct IO. Needs a fix https://github.com/apache/bookkeeper/pull/4041 | |||
| # PULSAR_PREFIX_dbStorage_directIOEntryLogger: "true" | |||
There was a problem hiding this comment.
Please change the default value to false
There was a problem hiding this comment.
@hangc0276 fixed in 462b4a7
Signed-off-by: ericsyh <[email protected]>
|
@hangc0276 @codelipenghui PTAL again. Thx! |
(If this PR fixes a github issue, please add
Fixes #<xyz>.)Fixes #1068
(or if this PR is one task of a github issue, please add
Master Issue: #<xyz>to link to the master issue.)Master Issue: #
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
Modifications
Describe the modifications you've done.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Documentation
Check the box below.
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)