Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Providing code for performance test metrics #2955

Closed
cisionmarkwalls opened this issue Nov 14, 2022 · 5 comments
Closed

Providing code for performance test metrics #2955

cisionmarkwalls opened this issue Nov 14, 2022 · 5 comments
Assignees
Labels
content/missing-information More information requested or needed
Milestone

Comments

@cisionmarkwalls
Copy link

cisionmarkwalls commented Nov 14, 2022

What content needs to be created or modified?
For the performance page: https://docs.dapr.io/operations/performance-and-scalability/perf-service-invocation/ there is no link to the code used for this performance test, so it is not reproducible and reduces trust making it harder to get buy-in from DevOps vs known solutions like Istio.

Describe the solution you'd like

  1. A github link to a reproducible test case
  2. A suggestion would be to copy the Istio test case here: https://github.com/istio/tools/tree/release-1.15/perf/load so you can see an apples to apples comparison between the stacks.
  3. Perhaps looking at https://istio.io/latest/blog/2019/performance-best-practices/ for performing valuable load testing metrics.

Where should the new material be placed?
A link in the documentation would be excellent.

The associated pull request from dapr/dapr, dapr/components-contrib, or other Dapr code repos
N/A

Additional context
A barrier to getting this implemented in large enterprises is concerns about utilization of resources and latency for all the services that DAPR provides. Giving an open source test case, especially one that mimics a well known tech stack like Istio, would provide strength to the argument that we receive more with DAPR with less resources.

@cisionmarkwalls cisionmarkwalls added the content/missing-information More information requested or needed label Nov 14, 2022
@berndverst
Copy link
Member

See perf for all performance tests at https://github.com/dapr/dapr/tree/master/tests

See this article for instructions to run the perf tests https://github.com/dapr/dapr/blob/master/tests/docs/running-perf-tests.md

@tmacam might be able to comment on whether the perf numbers are different since switching from Fortio to K6.

@berndverst
Copy link
Member

@cisionmarkwalls please note that Dapr does not compete with Istio / is not a replacement for Istio. It solves a different use case. For that reason using Istio test cases also does not make sense.

I'll let runtime maintainers @yaron2 and @artursouza chime in more.

@artursouza
Copy link
Member

There are overlaps in some functionalities between Istio and Dapr, mainly service invocation and tracing. Also, Dapr requires app to change endpoint while Istio does not require any code change.

@yaron2
Copy link
Member

yaron2 commented Feb 23, 2023

In addition to @berndverst pointing to the perf tests source and the instructions for how to reproduce them, I'd mention that we took inspiration from Istio when building the perf tests, all the way from using Fortio as the test tool, to using the same test parameters (QPS, connection size) and more. Another thing to mention is that the current perf stats on our docs are not up-to-date, and today we're seeing ~1.10ms end to end latency as opposed to the 1.4ms in the current version of the docs.

@hhunter-ms
Copy link
Collaborator

closing in favor of related docs issue #3468

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content/missing-information More information requested or needed
Projects
None yet
Development

No branches or pull requests

6 participants