InstructorDocumentEmbedder is breaking

[touhi99]

Hi, I am trying to load InstructorDocumentEmbedder:

doc_embedder = InstructorDocumentEmbedder(
model="intfloat/multilingual-e5-large-instruct",
instruction=doc_embedding_instruction,
batch_size=32
)

but on sentence-transformers 2.7.0 getting this error,

TypeError: INSTRUCTOR._load_sbert_model() got an unexpected keyword argument 'token'

tried reverting it back to sentence-transformers 2.2.2 but then getting another error, .cache/torch/sentence_transformers/intfloat_multilingual-e5-large-instruct/sentence_xlnet_config.json'

Please let me know if there's other way around to make it running or alternative to run instruct base embeddings. Much appreciated.

Describe your environment (please complete the following information):

• OS: ubuntu 22.04
• Haystack version: haystack-ai==2.0.1
• Integration version: instructor-embedders-haystack==0.4.0

[anakin87]

Hey!

You should use https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder. It's well documented.

This integration focuses instead on INSTRUCTOR embedding models: https://haystack.deepset.ai/integrations/instructor-embedder

[touhi99]

ok, should i use for e5-instruct in this approach?

doc_embedder = SentenceTransformersDocumentEmbedder(model="intfloat/multilingual-e5-large-instruct",
prefix=TASK_SPECIFIC_INSTRUCTION)

I saw instruct embedding integration, so got confused if I should use that one instead.

[anakin87]

I'm closing the issue.
Feel free to open another issue for bugs or a discussion at https://github.com/deepset-ai/haystack/discussions.