- - We present CASS, object-level Context-Aware training-free open-vocabulary Semantic Segmentation model. - Our method distills the vision foundation model's (VFM) object-level contextual spectral graph into CLIP's attention and refines query text embeddings towards object-specific semantics. -
+
+ + We present CASS, object-level Context-Aware training-free open-vocabulary Semantic Segmentation model. + Our method distills the vision foundation model's (VFM) object-level contextual spectral graph into CLIP's attention and refines query text embeddings towards object-specific semantics. +
+
- - Detailed illustration of our object presence prior-guided text embedding adjustment module. - The CLIP text encoder generates text embeddings for each object class, and the object presence prior is derived from both visual and text embeddings. - Within hierarchically defined class groups, text embeddings are selected based on object presence prior, then refined in an object-specific direction to align with components likely present in the image. -
++ Detailed illustration of our object presence prior-guided text embedding adjustment module. + The CLIP text encoder generates text embeddings for each object class, and the object presence prior is derived from both visual and text embeddings. + Within hierarchically defined class groups, text embeddings are selected based on object presence prior, then refined in an object-specific direction to align with components likely present in the image. +
+
- - Attention score visualization for various query points. Left: Vanilla CLIP (\(A_{\text{CLIP}}\)) shows noisy, unfocused attention. Center: VFM-to-CLIP distillation without low-rank eigenscaling shows partial object grouping with limited detail. Right: Incorporating our low-rank eigenscaling captures object-level context, improving grouping within a single object. -
+
+ + Attention score visualization for various query points. Left: Vanilla CLIP (ACLIP) shows noisy, unfocused attention. Center: VFM-to-CLIP distillation without low-rank eigenscaling shows partial object grouping with limited detail. Right: Incorporating our low-rank eigenscaling captures object-level context, improving grouping within a single object. +
+