enabler: guardrail inserting values to vector store

[kdziedzic68]

Feature description

The effect could be achived in a few ways

1. store full embedding model name in metadata (currently only image or text are stored) - would help identify eventual broken entries
2. keep embedder as the attribute of vector store - seems nice and elegant approach to me, but would require significant considerations of system architecture
   3, develop some outlier detection model for the store - quiet risky, may not be 100% accurate, from the other hand could be helpful with managing store size - eg. serve as noise detector

Motivation

Right now we are able to insert any vectors to the given store. Let assume that somebody has chosen model that has the same number of dimensions as the ones already stored - but different model. In current setup we would insert it and break the store - without possibility of identyfing broken entries

Additional context

No response

[mhordynski]

2. keep embedder as the attribute of vector store - seems nice and elegant approach to me, but would require significant considerations of system architecture

and later implementing

1. store full embedding model name in metadata (currently only image or text are stored) - would help identify eventual broken entries

seems like good option, but we need to dig deeper to understand all consequences.