Code

Implement ORDER BY BM25#1434

Code

Open

jbellis wants to merge 31 commits intomaindatastax/cassandra:mainfrom scoreordering-6datastax/cassandra:scoreordering-6Copy head branch name to clipboard

+2,789-564

Commits

Commits on Nov 25, 2024

• remove unnecessary generification of IndexColumnComparator

  jbellis

  committed
  ddd9b69

• Simplify the ordering logic by making IndexColumnComparator only responsible for ANN index queries. Other global orderings will be represented by a SingleColumnComparator with clustered=true instead.

  jbellis

  committed
  9f1b794

Commits on Nov 26, 2024

• CNDB-11725 use +score pseudo-column to order ANN results with instead of recomputing scores on the coordinator

  jbellis

  committed
  9986108

• CNDB-11725 add SYNTHETIC ColumnMetadata.Kind to represent the score column

  jbellis

  committed
  e0ea872

Commits on Dec 2, 2024

• merge with main

  jbellis

  committed
  478ca65

Commits on Dec 6, 2024

• implement BM25

  jbellis

  committed
  237cec4

• re-disallow DESC with ORDER BY ANN

  jbellis

  committed
  56a6e0f

Commits on Dec 9, 2024

• cleanup and comments

  jbellis

  committed
  30b6545

Commits on Dec 11, 2024

• address review notes

  jbellis

  committed
  6c9a0e6

• remove unused limit parameter from IndexSearcher::search

  jbellis

  committed
  e107fcc

• eliminate currentRowIds

  jbellis

  committed
  cfe204a

• handle eq in an analyzed index by transforming it into a Match restriction in SingleColumnRelation.newEQRestriction. This eliminates the need for skipMerge and special cases in doMergeWith, and mov…

  Show description for cef71e3
  jbellis

  committed
  cef71e3

• add testMatchingAllowed and make it work via shouldMerge

  jbellis

  committed
  3d17e2f

Commits on Dec 12, 2024

• disambiguate the BM25 error message when the index isn't analyzed

  jbellis

  committed
  907a2ee

• validateOptions treats analyzed and un-analyzed indexes as distinct, testTwoIndexes passes

  jbellis

  committed
  3315a12

• detect and reject ambiguous equality predicates; testAmbiguousPredicates passes

  jbellis

  committed
  c0de416

Commits on Dec 13, 2024

• don't inject +score unless coordinator requests it; this is a cleaner approach than ignoring it when serialization fails later

  jbellis

  committed
  3967e7c

• fix getEqBahavior, this is most of the test failures

  jbellis

  committed
  893b87b

• LongBM25Test

  jbellis

  committed
  c1eaa63

• misc bugfixes related to zero matches for a term

  jbellis

  committed
  ddbfd16

• ramIndexer deduplicates (term, row) pairs

  jbellis

  committed
  7ff2374

• need to use compareUnsigned once we have more than 4 KINDs

  jbellis

  committed
  cfa1157

• simplify

  jbellis

  committed
  1620d3e

• make SYNTHETIC the first Column Kind instead of the last. This avoids breaking the assumption in BTreeRow that complex regular/static columns sort last

  jbellis

  committed
  73f35df

• fix tests

  jbellis

  committed
  3ad8ae2

Commits on Dec 16, 2024

• DRY refactor

  jbellis

  committed
  dbbc678

• add tests for unknown query terms, duplicate query terms, no query terms

  Show description for 1b57d55
  jbellis

  committed
  1b57d55

Commits on Dec 17, 2024

• // since doc frequencies can be an estimate from the index histogram, which does not have bounded error,

  Show description for 0b5ce5c
  jbellis

  committed
  0b5ce5c

• parameterize version to test with/without histograms

  jbellis

  committed
  f3f7a15

• merge with main

  jbellis

  committed
  3416389

• actually parameterize both versions

  jbellis

  committed
  d83e18d