This module is closely related to EmbeddingModule.

Cross-encoder models use Q and A pairs and are trained return a relevance score for rank().
The existing rerank APIs in EmbeddingModule had to encode Q and A
separately and use cosine similarity as a score. So the API is the same, but the results
are supposed to be better (and slower).

Cross-encoder models do not support returning embedding vectors or sentence-similarity.

Support for the existing tokenization and model_info endpoints was also added.

Signed-off-by: Mark Sturdevant <[email protected]>

* mostly removing unnecessary code
* some better clarity

Signed-off-by: Mark Sturdevant <[email protected]>

* The already borrowed errors are fixed with tokenizers per thread,
  so there were some misleading comments about not changing params
  for truncation (which we do for cross-encoder truncation).

Signed-off-by: Mark Sturdevant <[email protected]>

Default is 32.
Can override with embedding batch_size in config or EMBEDDING_BATCH_SIZE env var.

Signed-off-by: Mark Sturdevant <[email protected]>

* Moved the truncation check to a place that can determine
  the proper index for the error message (with batching).

* Added test to validate some results after truncation.
  This is with a tiny model, but works for sanity.

Signed-off-by: Mark Sturdevant <[email protected]>

The part that really tests that a token is truncated was wrong.

* It was backwards and passing because the scores are sorted by rank
* Using the index to get scores in the order of the inputs
* Now correctly xx != xy but xy == xyz (truncated z)

Signed-off-by: Mark Sturdevant <[email protected]>

Signed-off-by: Mark Sturdevant <[email protected]>