[BUG] MacOS test for RDST Transformer

[baraline]

Describe the bug

During the integration of the RDST transformer (see #310), the output of the transformation was different than what was expected on macOS systems.

The failing tests have been removed for now, but should re-introduced for coverage. Test which were failing were written as follows:

• Confirm that the problem comes from MacOS system and not a bug in the code
• Investigate why the tests are failing on MacOS and not on other platforms

Steps/Code to reproduce the bug

import pytest
from numpy.testing import assert_array_almost_equal

from aeon.datasets import load_basic_motions, load_unit_test
from aeon.transformations.panel.dilated_shapelet_transform import (
    RandomDilatedShapeletTransform
)


def test_rdst_on_unit_test():
    """Test of ShapeletTransform on unit test data."""
    # load unit test data
    X_train, y_train = load_unit_test(split="train")
    indices = np.random.RandomState(0).choice(len(y_train), 5, replace=False)

    # fit the shapelet transform
    st = RandomDilatedShapeletTransform(max_shapelets=3, random_state=0)
    st.fit(X_train[indices], y_train[indices])

    # assert transformed data is the same
    data = st.transform(X_train[indices])
    assert_array_almost_equal(data, shapelet_transform_unit_test_data, decimal=4)


def test_rdst_on_basic_motions():
    """Test of ShapeletTransform on basic motions data."""
    # load basic motions data
    X_train, y_train = load_basic_motions(split="train")
    indices = np.random.RandomState(4).choice(len(y_train), 5, replace=False)

    # fit the shapelet transform
    st = RandomDilatedShapeletTransform(max_shapelets=3, random_state=0)
    st.fit(X_train[indices], y_train[indices])

    # assert transformed data is the same
    data = st.transform(X_train[indices])
    assert_array_almost_equal(data, shapelet_transform_basic_motions_data, decimal=4)

Expected results

shapelet_transform_unit_test_data = np.array(
    [
        [1.90317756, 8.0, 2.0, 2.87919021, 10.0, 3.0, 0.0, 1.0, 1.0],
        [2.16550181, 8.0, 2.0, 0.0, 10.0, 2.0, 1.52148128, 3.0, 1.0],
        [0.0, 8.0, 1.0, 3.41218663, 10.0, 2.0, 1.00243477, 1.0, 2.0],
        [2.76771406, 8.0, 2.0, 5.75682976, 10.0, 1.0, 1.66589725, 3.0, 1.0],
        [2.95206323, 8.0, 2.0, 2.82417348, 10.0, 3.0, 0.91588726, 1.0, 1.0],
)

shapelet_transform_basic_motions_data = np.array(
    [
        [32.45712774, 25.0, 5.0, 58.52357949, 5.0, 0.0, 56.32267413, 21.0, 4.0],
        [59.8154656, 69.0, 0.0, 64.16747582, 37.0, 0.0, 0.0, 18.0, 5.0],
        [58.27369761, 11.0, 0.0, 67.49320392, 53.0, 0.0, 61.18423956, 31.0, 1.0],
        [62.49300933, 13.0, 0.0, 0.0, 13.0, 5.0, 59.51080993, 34.0, 3.0],
        [0.0, 12.0, 12.0, 64.73843849, 13.0, 0.0, 62.52577812, 8.0, 0.0],
    ]
)

Actual results

=========================== short test summary info ============================
 FAILED aeon/transformations/collection/tests/test_dilated_shapelet_transform.py::test_rdst_on_basic_motions - AssertionError: 
Arrays are not almost equal to 4 decimals

Mismatched elements: 38 / 45 (84.4%)
Max absolute difference: 61.86727018
Max relative difference: 6.8
 x: array([[6.3691e+01, 5.5000e+01, 0.0000e+00, 6.6495e+01, 3.9000e+01,
        0.0000e+00, 6.1568e+01, 6.0000e+00, 0.0000e+00],
       [1.3954e-14, 5.1000e+01, 5.0000e+00, 5.7427e+01, 3.8000e+01,...
 y: array([[32.4571, 25.    ,  5.    , 58.5236,  5.    ,  0.    , 56.3227,
        21.    ,  4.    ],
       [59.8155, 69.    ,  0.    , 64.1675, 37.    ,  0.    ,  0.    ,...
FAILED aeon/transformations/collection/tests/test_dilated_shapelet_transform.py::test_rdst_on_unit_test - AssertionError: 
Arrays are not almost equal to 4 decimals

Mismatched elements: 43 / 45 (95.6%)
Max absolute difference: 7.
Max relative difference: 5.
 x: array([[1.5215e+00, 1.0000e+00, 2.0000e+00, 2.0003e+00, 1.3000e+01,
        1.0000e+00, 1.3931e+00, 5.0000e+00, 2.0000e+00],
       [0.0000e+00, 3.0000e+00, 1.0000e+00, 2.5269e+00, 1.3000e+01,...
 y: array([[ 1.9032,  8.    ,  2.    ,  2.8792, 10.    ,  3.    ,  0.    ,
         1.    ,  1.    ],
       [ 2.1655,  8.    ,  2.    ,  0.    , 10.    ,  2.    ,  1.5215,...

Versions

aeon == 0.3.0
numpy == 1.23.5
numba == 0.56.4

[baraline]

Since the fixes introduced in #483, this may now work properly, I will try to re-include those tests now.

The expected results have also been updated to use Manhattan distance with the changes introduced in #483

[baraline]

Could someone with access to a MacOS system try to run the MacOS test sequentially by hand ? I don't have any ...
Some other issues on other repository with the same problem seem to point to a problem with the shared file system state when running tests in parallel.

[TonyBagnall]

I now have access to a Mac, and will try this out when I get round to learning how to drive it :)

[TonyBagnall]

we can close this now the mac testing has beeen revised in #1585

[TonyBagnall]

RDSTClassifier still has the tag

        "non_deterministic": True,  # due to random_state bug in MacOS #324

so we should test if this is still needed @baraline

[baraline]

Did you find anything useful in the PR you closed @TonyBagnall ?

[TonyBagnall]

lets close this and @baraline can open a more specific issue