[ISSUE] Broken comparison plots (scenario level)

[dharabor]

Algorithm comparison plots look to be buggy in some cases; e.g., orz900d-even-1. It looks like LaCAM solutions are being assigned zero cost?? See here

[bshen95]

Don't think it is a bug. The plot compares the solution cost of each algorithm with best known solution cost found so far. In that case, LaCAM is the only one that found a feasible solution.

[dharabor]

The plot is a mess though and doesn't communicate the situation very effectively. So I still think this is a bug.

In my opinion we should either:

1. Track how many best-known claims have been submitted for each instance, which we could do with a stacked barplot (one coloured segment per bar for each selected algorithm being compared)
2. We indicate the size of the gap for the best known solution and in the plot area underneath we indicate which algorithms have matched this best-known result (maybe by plotting veritcally stacked symbols)

[dharabor]

Another related issue: the list of claims (in the table) tracks only how many algorithms have achieved the best-known cost. But the "Compare algorithms" plot shows dominated solutions if one exists for a selected solver. It's a bit confusing because the plot should be a summary of the table data. I think there's a place for comparing all results submitted for a specific solver vs. SOTA, on a particular map or scenario or instance, but we might want to do this as a separate feature.

[bshen95]

The "solution cost" function is fixed.
The "lower bound" one is waiting for trivial bound data

[dharabor]

We seem to have trivial bounds but the plots still appear messed up. Why is there still an infinite gap? e.g.,
https://imgur.com/a/HR2oqpW