Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement second bitmap, ~2x speedup for train #21

Merged
merged 1 commit into from
Aug 21, 2024

Conversation

a10y
Copy link
Contributor

@a10y a10y commented Aug 21, 2024

image

^ this is a sad flamegraph. bzero is not a fun place to be spending 60% of your time.

image

How did we get from one to two?

  1. Avoid initializing our counts1 and counts1 vectors
  2. Implement a second bitmap index that limits our outer-loop iterations in optimize. Because counts1 and counts2 are not initialized, we check the bitmap before all accesses

This also gives us another 2x speedup on the train benchmark, which is nice

@a10y a10y merged commit d15cc4e into develop Aug 21, 2024
3 checks passed
@a10y a10y deleted the aduffy/counters-bitmap branch August 21, 2024 15:47
@github-actions github-actions bot mentioned this pull request Aug 21, 2024
a10y pushed a commit that referenced this pull request Aug 21, 2024
## 🤖 New release
* `fsst-rs`: 0.2.1 -> 0.2.2

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [0.2.2](v0.2.1...v0.2.2) -
2024-08-21

### Other
- implement second bitmap, ~2x speedup for train
([#21](#21))
- remove spurious check
([#18](#18))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/MarcoIeni/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant