Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filestore] issue-2467: use file handle to restore invalidated node cache entries in local filestore #2938

Merged
merged 6 commits into from
Feb 11, 2025
Merged

[Filestore] issue-2467: use file handle to restore invalidated node cache entries in local filestore #2938

merged 6 commits into from
Feb 11, 2025

Conversation

budevg
Copy link
Collaborator

@budevg budevg commented Jan 28, 2025 •
edited
Loading

#2467

We can invalidate entries in node cache if we can find them later by inode id. This can be done on several shared filesystems like weka and lustre using open_by_handle_at. The handle serialization in these filesystem can be generated from inode id.

Invalidation of node cache entries done using the following algorithm

  • if we reach max nodes we forget NodeCleanupBatchSize least recently used nodes immediately
  • Then we forget 1 more node for each lookup until number of nodes in cache drops to half of the limit
  • This is supposed to amortize the cost of dropping many nodes (one close system call for each) on subsequent lookup calls.

@budevg budevg added the filestore Add this label to run only cloud/filestore build and tests on PR label Jan 28, 2025
@github-actions GitHub Actions
Copy link
Contributor

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f913853.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2005 1980 0 16 5 4

debnatkh
debnatkh reviewed Feb 2, 2025
View reviewed changes
cloud/filestore/libs/service_local/index.cpp Outdated Show resolved Hide resolved
cloud/filestore/libs/service_local/index.h Outdated Show resolved Hide resolved
cloud/filestore/libs/service_local/index.h Outdated Show resolved Hide resolved
cloud/filestore/libs/service_local/index.h Outdated Show resolved Hide resolved
cloud/filestore/libs/service_local/index.h Outdated Show resolved Hide resolved
cloud/filestore/libs/service_local/index.h Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 4, 2025

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit c857078.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2033 2033 0 0 0 0

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 5, 2025

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 19e7867.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2033 2033 0 0 0 0

@budevg budevg force-pushed the users/evgenybud/local-filestore-fixes-fh branch from 19e7867 to cc8e7df Compare February 5, 2025 11:02
@debnatkh debnatkh added tsan Launch builds with thread sanitizer along with regular build large-tests Launch large tests for PR labels Feb 5, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 5, 2025

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit cc8e7df.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2051 2051 0 0 0 0

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 5, 2025

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-release-tsan: all tests PASSED for commit cc8e7df.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1951 1951 0 0 0 0

debnatkh
debnatkh previously approved these changes Feb 6, 2025
View reviewed changes
@budevg budevg force-pushed the users/evgenybud/local-filestore-fixes-fh branch from cc8e7df to 618c378 Compare February 9, 2025 09:43
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 9, 2025

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 618c378.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2054 2054 0 0 0 0

yegorskii
yegorskii previously approved these changes Feb 9, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 1410dfc.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2054 2054 0 0 0 0

@github-actions GitHub Actions
Copy link
Contributor

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-release-tsan: all tests PASSED for commit 1410dfc.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1954 1954 0 0 0 0

@github-actions GitHub Actions
Copy link
Contributor

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-release-tsan: all tests PASSED for commit 2931c18.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
1954 1954 0 0 0 0

@github-actions GitHub Actions
Copy link
Contributor

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 2931c18.

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
2054 2054 0 0 0 0

@budevg budevg merged commit a2c69e5 into main Feb 11, 2025
14 of 15 checks passed
@budevg budevg deleted the users/evgenybud/local-filestore-fixes-fh branch February 11, 2025 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SvartMetal SvartMetal SvartMetal approved these changes

@yegorskii yegorskii yegorskii approved these changes

@debnatkh debnatkh Awaiting requested review from debnatkh

Assignees
No one assigned
Labels
filestore Add this label to run only cloud/filestore build and tests on PR large-tests Launch large tests for PR tsan Launch builds with thread sanitizer along with regular build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@budevg @SvartMetal @debnatkh @yegorskii