Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of VariableOrigin instances #3164

Merged
merged 3 commits into from
Jul 17, 2024

Conversation

eduar-hte
Copy link
Contributor

@eduar-hte eduar-hte commented Jun 2, 2024

what

  • The previous approach would create a std::unique_ptr and store it in a std::list in VariableValue's Origins.
  • The new approach stores Origins in a std::vector and constructs VariableOrigin elements in-place on insertion.
  • Instead of having two heap-allocations for every added VariableOrigin instance, this performs only one.
  • If multiple origins are added, std::vector's growth strategy may even prevent the heap-allocation on additional insertions.
    • When the vector is resized, there's the cost of copying the current elements.
  • Introduced reserveOrigin method to notify when multiple insertions will be made, so that a call to std::vector::reserve can be made to perform a single allocation (and copy of previous elements) if necessary, and then just initialize the new elements in-place.

why

This change increases performance of the benchmark test by about 2.5%.

Additionally, the refactored code simplifies code used to initialize VariableOrigin instances.

misc

Included script to download OWASP Core Ruleset v4 for benchmark execution.

This is part of a series of PRs to improve performance of the library (1/n).

@eduar-hte eduar-hte force-pushed the variable-origin branch 2 times, most recently from 790fbcc to b165584 Compare June 2, 2024 22:15
Copy link

sonarqubecloud bot commented Jun 2, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@airween
Copy link
Member

airween commented Jul 10, 2024

Thanks for this PR - the performance improvement is awesome! As I could measure - based on result of own benchmark tool it's better than 2.5-3% with CRS, I think it's about 5-6%.

I requested a change: could you remove the CRS2 test case? It's completely unnecessary.

@marcstern marcstern added the 3.x Related to ModSecurity version 3.x label Jul 11, 2024
- The previous approach would create a std::unique_ptr and store it in
  a std::list in VariableValue (Origins)
- The new approach now stores Origins in a std::vector and constructs
  VariableOrigin elements in-place on insertion.
- Instead of having two heap-allocations for every added VariableOrigin
  instance, this performs only one.
- If multiple origins are added, std::vector's growth strategy may even
  prevent a heap-allocation. There's a cost on growing the size of the
  vector, because a copy of current elements will be necessary.
  - Introduced reserveOrigin method to notify that multiple insertions
    will be made, so that we can use std::vector's reserve and do a
    single allocation (and copy of previous elements), and then just
    initialize the new elements in-place.
- Simplified clone & checkout of CRS repository
- Removed no longer maintained OWASP Core Ruleset v2
@airween airween merged commit 3dda900 into owasp-modsecurity:v3/master Jul 17, 2024
48 checks passed
@eduar-hte eduar-hte deleted the variable-origin branch July 17, 2024 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.x Related to ModSecurity version 3.x
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants