Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: external state storage for netflow/ipfix template & sampling rate data #289

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

iqbalaydrus
Copy link
Contributor

Inspired from vflow discovery to sync its templates between instances. But instead of using multicast ip, it's using redis kv & pubsub because it's more friendly in a cloud environment. I created this for two purposes:

  • Per packet udp load balancing
  • Minimizing data loss in case of restart (basically template persistence)

But also worth noting, the in memory engine benchmark is worse than the current implementation, 72.90ns vs 28.12ns per operation, or 17 vs 35 Mops/s, or 109 Gbps vs 280 Gbps of flows (assuming 1000 bytes size per packet). It's allocating more for extra metadata when marshalling. But it's still on the tens of millionth operation per second, I think the bottleneck will still be somewhere else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant