Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request]: Allow consumers to start consuming from specified offset #584

Open
Waddas opened this issue Jul 30, 2024 · 0 comments · May be fixed by #585
Open

[Feature Request]: Allow consumers to start consuming from specified offset #584

Waddas opened this issue Jul 30, 2024 · 0 comments · May be fixed by #585
Labels
enhancement New feature or request

Comments

@Waddas
Copy link

Waddas commented Jul 30, 2024

Is your request related to a problem you have?

I need to be able to tell the consumers to start from a specified offset. This is implemented in Confluent Kafka by calling Assign with a TopicPartitionOffset.

Describe the solution you'd like

There is currently the ability to set ManualAssignPartitions in the Consumer Configuration, It would be nice to also have ManualAssignPartitionOffsets. Configuration would look like:

consumer.ManualAssignPartitionOffsets("my-topic", new Dictionary<int, long> { { 0, 100 }, { 1, 120 } })

The implementation would look like:

private void ManualAssignPartitionOffsets(IEnumerable<TopicPartitionOffsets> topics)
    {
        var partitionOffsets = topics
            .SelectMany(
                topic => topic.PartitionOffsets.Select(
                    partitionOffset => new Confluent.Kafka.TopicPartitionOffset(
                        topic.Name, new Partition(partitionOffset.Key), new Offset(partitionOffset.Value))))
            .ToList();

        _consumer.Assign(partitionOffsets);

Are you able to help bring it to life and contribute with a Pull Request?

Yes

Additional context

No response
@Waddas Waddas added the enhancement New feature or request label Jul 30, 2024
@Waddas Waddas linked a pull request Jul 30, 2024 that will close this issue
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Allow consumers to start from a specified offset Waddas/kafkaflow
1 participant
@Waddas