Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mito): Init the write cache in datanode #3100

Merged
merged 17 commits into from
Jan 9, 2024

Conversation

evenyag
Copy link
Contributor

@evenyag evenyag commented Jan 4, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR initializes the WriteCache according to the mito config. By default, the write cache is under DATA_HOME/write_cache and is disabled.

It addresses an issue that the file cache doesn't use get() to access cache entries.

It also refactors some mods to make the integration easier

  • Adds a builder for the CacheManager
  • Recovers the FileCache in WriteCache::new
  • MitoEngine::new is an async function now
  • Provides a util to attach layers to ObjectStore

Now write cache is still an experimental feature so I added prefixes to its config items.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

@evenyag evenyag force-pushed the feat/write-cache-config branch from 12558fc to c856ef9 Compare January 4, 2024 15:49
@evenyag evenyag changed the title feat(mito): Init the write cache feat(mito): Init the write cache in datanode Jan 5, 2024
@evenyag evenyag marked this pull request as ready for review January 5, 2024 06:17
Copy link

codecov bot commented Jan 5, 2024

Codecov Report

Attention: 70 lines in your changes are missing coverage. Please review.

Comparison is base (e0a43f3) 85.66% compared to head (ea008e4) 85.19%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3100      +/-   ##
==========================================
- Coverage   85.66%   85.19%   -0.48%     
==========================================
  Files         807      807              
  Lines      132167   132354     +187     
==========================================
- Hits       113227   112755     -472     
- Misses      18940    19599     +659     

@github-actions github-actions bot added Size: L and removed Size: M labels Jan 5, 2024
@evenyag evenyag mentioned this pull request Jan 5, 2024
3 tasks
src/mito2/src/cache.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@QuenKar QuenKar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@evenyag evenyag added this pull request to the merge queue Jan 9, 2024
Merged via the queue into GreptimeTeam:main with commit 8bd4a36 Jan 9, 2024
15 checks passed
@evenyag evenyag deleted the feat/write-cache-config branch January 9, 2024 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants