per-group baseline in so.Bar

[FirefoxMetzger]

I am learning how to use the new objects interface (kudos btw it looks amazing 🚀) and I'm trying to create a simple box plot for different groups. My problem is that I can't work out how to move the baseline of a bar up so that it begins at the bottom quartile.

Here is how far I've gotten thus far:

import seaborn.objects as so
import numpy as np

data = sns.load_dataset("iris")

(
    so.Plot(data, x="species", y="sepal_length")
    .add(so.Range(), so.Est(errorbar=("pi", 95)))
    .add(so.Dash(), so.Agg("median"))
    .add(so.Dash(width=0.1), so.Agg(lambda x: np.percentile(x, 2.5)))
    .add(so.Dash(width=0.1), so.Agg(lambda x: np.percentile(x, 97.5)))
    
    # NOTE: How can I "lift" the bar / set baseline for each box?
    # .add(so.Bar(), so.Agg(lambda x: np.percentile(x, 75)))
    
    # NOTE: How do I add _only_ outliers (outside some threshold)?
    # .add(so.Dot(alpha=0.2, edgealpha=1))
)

There is also a question on how to add the outlier marks that you would get with sns.boxplot. I'm also not sure how to do this, but my most pressing question is on baseline, so I figured to start here first.

[mwaskom]

I think the question here is mostly "will there be a boxplot mark", and the answer is yes.

[mwaskom]

BTW you could make this existing plot a bit simpler with the Perc stat:

(
    so.Plot(iris, x="species", y="sepal_length")
    .add(so.Dash(), so.Agg("median"))
    .add(so.Dash(width=0.1), so.Perc([2.5, 97.5]))
    .add(so.Range(), so.Perc([2.75, 97.5]))
)

[mwaskom]

As to the more specific questions — I'm not sure about a per-group bar baseline because I have on the roadmap a Span mark that would be essentially an area mark parameterized by width/{dv}min/{dv}max which is what you're looking for.

(
    so.Plot(iris, x="species", y="sepal_length")
    .add(so.Dash(), so.Agg("median"))
    .add(so.Dash(width=0.1), so.Perc([2.5, 97.5]))
    .add(so.Range(), so.Perc([2.75, 97.5]))
    .add(so.Span(), so.Perc([25, 75]))
)

I'm not sure about an "identify outliers" stat but it could make sense. By the way, a standard boxplot uses a more complicated rule than what you're using here to draw the whiskers: they're usually not showing a 95%ile interval, but a rule of "draw up to 1.5 * IQR +/- 75/25%ile". In your approach you'd always have some "outliers" which isn't the case with the Tukey approach.

[FirefoxMetzger]

I'm not sure about a per-group bar baseline because I have on the roadmap a Span mark that would be essentially an area mark parameterized by width/{dv}min/{dv}max which is what you're looking for.

That would work nicely, too. Do you know a (rough) ETA for this? (are we talking months? quarters?) Also, what does {dv}min mean here (in particular the {dv})?

I actually have a second use-case for either Span(..) or baseline="variable": making a waterfall chart. I have matplotlib code that builds it from a dataframe, but I always thought it is a rather ugly solution. I can see a lot of the building blocks for this in the objects API, so once Span(..) becomes available I'd be interested in revisiting this :).

I'm not sure about an "identify outliers" stat but it could make sense.

Maybe not specifically for identifying outliers, but a more general filter/mask could have merit; at least that's what I intuitively started searching for while trying to build the box plot. I expected something where I could provide a callable that produces an index array which allows me to mask some data. For a boxplot this would take the form of masking everything except for outliers, for something like an SVM visualization it could mask everything but support vectors (so I can highlight them), and for analyzing a classifier it could mask out everything that was correctly classified so that I can explore potential sources of error.

By the way, a standard boxplot uses a more complicated rule than what you're using here to draw the whiskers

Yep, I'm aware that this is less common. Wikipedia mentioned it as an alternative for the whiskers (though it uses the 2nd and 97th percentile), so I figured it's close enough for a reproducible example. For the actual IQR, I couldn't figure out how to get this done with the objects API. The closest I got was to hack it into existence using my own class:

from dataclasses import dataclass
from typing import Callable
import seaborn.objects as so
import numpy as np
import pandas as pd


@dataclass
class CustomStat(so.Stat):
    stat_fn : Callable
    group_by_orient = True

    def _wrapper(self, data, var):
        return pd.DataFrame({var: self.stat_fn(data)})

    def __call__(self, data, groupby, orient, scales):
        var = {"x": "y", "y": "x"}[orient]
        return groupby.apply(data, self._wrapper, var)


def whiskers(data):
    lower, upper = np.percentile(data, [25, 75])
    iqr = np.diff([lower, upper])[0]
    lb, ub = lower - 1.5 * iqr, upper + 1.5 * iqr

    bottom_wisker = np.min(np.where(data.values < lb, ub, data.values))
    top_wisker = np.max(np.where(data.values > ub, lb, data.values))

    return bottom_wisker, top_wisker


iris = sns.load_dataset("iris")

(
    so.Plot(iris, x="species", y="sepal_length")
    .add(so.Dash(), so.Agg("median"))
    .add(so.Dash(width=0.1), CustomStat(whiskers))
    .add(so.Range(), CustomStat(whiskers))
)

[mwaskom]

That would work nicely, too. Do you know a (rough) ETA for this? (are we talking months? quarters?)

I only do estimation at my actual job :) but probably pretty soon.

Also, what does {dv}min mean here (in particular the {dv})?

Ah I was indicating the "depending variable" i.e. opposite the orient variable. I don't really have a good consistent terminology for this yet.

I actually have a second use-case for either Span(..) or baseline="variable": making a waterfall chart.

Yeah this would be the right mark to use there and then you'd just need a transform to set the min/max (and maybe color) values appropriately for each group.

Maybe not specifically for identifying outliers, but a more general filter/mask could have merit ... I expected something where I could provide a callable that produces an index array which allows me to mask some data.

Sure, something like this makes plenty of sense.