You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Caching right now is weight agnostic, but at the cost of creating lower performance engines.
Describe the solution you'd like
If we know that weights would be identical, then we can cache engines that are higher performance. The caching system would need to be able to distinguish these two caches and based on user settings select the right one
TensorRT has a flag called kREFIT_IDENTICAL for this workflow
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Caching right now is weight agnostic, but at the cost of creating lower performance engines.
Describe the solution you'd like
If we know that weights would be identical, then we can cache engines that are higher performance. The caching system would need to be able to distinguish these two caches and based on user settings select the right one
TensorRT has a flag called kREFIT_IDENTICAL for this workflow
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: