[SYCL] Implement eviction for in-memory program cache #16062
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
had a problem deploying
to
WindowsCILock
GitHub Actions
Error
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
commented
Nov 14, 2024
There was a problem hiding this comment.
@maarquitos14 Since your last review, I have also converted the E2E test case that I added earlier into unit tests and fixed a few data races.
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
|
It took me some time to understand which program it was. |
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
maarquitos14
approved these changes
Nov 18, 2024
|
@intel/dpcpp-doc-reviewers / @steffenlarsen gentle ping. |
uditagarwal97
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
uditagarwal97
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
|
@intel/llvm-gatekeepers the PR is ready to be merged. |
sarnex
pushed a commit
that referenced
this pull request
Nov 25, 2024
…iction (#16129) This PR updates KernelProgramCache.md by: 1. Making the doc in-sync with recent changes in in-memory cache (Like #11351) 2. Adding details of in-memory cache eviction (#16062) Partially fixes: #16017 --------- Co-authored-by: Marcos Maronas <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: CMPLRLLVM-27640, #2517
The PR implements LRU cache eviction policy for in-memory program caches.
The high-level idea is to store programs in a linked-list, called eviction list. When the program is first added to the cache, it is also added to the eviction list. When a program is fetched from cache, we move the program to the end of the eviction list. So, that the programs at the beginning of the eviction list are always least recently used.
When adding a new program to cache, we check if the size of the program cache exceeds the threshold, if so, we evict the program from cache and corresponding kernels from Kernel and fast kernel cache.
This PR also adds a new environment variable,
SYCL_IN_MEM_CACHE_EVICTION_THRESHOLDthat user can use to control the size of in-memory cache. By default, cache eviction is disabled.