[ENH] ShapeletTransform: binary ig calculation problem

[zjeqw]

Describe the bug

The current _calc_binary_ig( ) evaluates split points between data points with the same feature values but different labels, which might not be suitable for datasets that contain a lot of such data points.

Steps/Code to reproduce the bug

from aeon.transformations.collection.shapelet_based._shapelet_transform import _calc_binary_ig
orderline = [(2,-1),(2,-1),(2,1),(3,1),(3,1)]
c1, c2 = 3, 2
_calc_binary_ig(orderline,c1,c2)

Expected results

0.42

Actual results

0.97

Versions

System:
python: 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)]
executable: c:\xxx\python.exe
machine: Windows-10-10.0.19041-SP0

Python dependencies:
pip: 22.3.1
setuptools: 57.4.0
scikit-learn: 1.4.0
aeon: 0.7.1
statsmodels: None
numpy: 1.24.0
scipy: 1.10.1
pandas: 2.0.3
matplotlib: 3.5.0
joblib: 1.3.2
numba: 0.58.1
pmdarima: None
tsfresh: None

[TonyBagnall]

thanks for this, we will take a look next week

[TonyBagnall]

next week became next month sorry about that....

I dont think this really constitutes a bug really, its true to the algorithm.

I guess for the above you are recommending ignoring splits such as
[(2,-1), (2,-1)], [(2,1),(3,1),(3,1)]
so we would then evaluate (default split)
[ ] [(2,-1), (2,-1),(2,1),(3,1),(3,1)]
skip
[(2,-1)] [(2,-1),(2,1),(3,1),(3,1)] split == 0 I think by the logic
and
[(2,-1),(2,-1)] ,[(2,1),(3,1),(3,1)] split == 1

then continue with
[(2,-1), (2,-1),(2,1)] [(3,1),(3,1)] split == 2

I can enforce this

    # evaluate each split point
    for split in range(len(orderline)):
        next_class = orderline[split][1]  # +1 if this class, -1 if other
        # Check here that the distance is different to the next one
        if split == 0 and orderline[split][0] == orderline[split+1][0]:
            continue
        elif orderline[split][0] == orderline[split-1][0]:
            continue

need to double check the logic a bit confusing about first item, but this gives me IG
0.770950 not of 0.42