Port selected 1.x components to 2.x

[julian-risch]

We identified the following key components from Haystack 1.x that still need to be ported to Haystack 2.0:

Tasks

Preview Give feedback

1. CohereRanker haystack-core-integrations#645
   6 of 6
   +0
2. DiversityRanker #7367
   +0
3. RecentnessRanker #7012
   +0
   
4. LostInTheMiddleRanker #7011
   +0
5. TextClassifier (porting TransformersQueryClassifier) #7017
   +0
   
6. NamedEntityExtractor #5679
   2.x P2 +2
   
7. Support for Crawler in Haystack 2.x #6609
   2.x type:feature +2

[sjrl]

Hey @julian-risch could we also add the nltk support for the DocumentSplitter in this list? That is a very common feature we are using right now in Haystack v1.

[sjrl]

And to clarify, is the TextClassifier in this list referring to the TransformersQueryClassifier from Haystack v1?

[julian-risch]

@sjrl Yes, it's about porting TransformersQueryClassifier but could become two classes TransformersTextClassifier and TransformersDocumentClassifier. That would be consistent with the SentenceTransformersDocumentEmbedder and the SentenceTransformersTextEmbedder we have in 2.x. Should behave similar to https://github.com/deepset-ai/haystack/blob/main/haystack/components/classifiers/document_language_classifier.py

[sjrl]

@julian-risch thanks for the clarification! @bglearning might also be taking a look at porting the TransformersQueryClassifier, so tagging him here.