Question : multi dimensional mstumped with T_B for non self join?

[vade]

Hello

Firstly, thank you for the sharing this project, its really awesome.

I am experimenting with stumpy and am able to annotate a matrix profile a with similarity to matrix profile b using the T_A and T_B variables in the stumpy.stump(T_A = c1, m = m, T_B = c2, ignore_trivial=False) method.

I want to experiment with multi dimensional stumpy but note that stumpy.mstump does not have the ability to pass in a second time series.

Is there a method for doing non self joins (arbitrary a/b joins) on two multi dimensional time series?

Thank you.

[seanlaw]

@vade Thank you for your question and welcome to the STUMPY community. So, the first thing to clarify is that multi-dimensional matrix profiles are NOT simply 1D matrix profiles stacked one on top of each other and I recommend going over our multidimensional matrix profile tutorial if you haven't done so already. Having said that, there is no interpretable multi-dimensional matrix profile for AB-joins. So, no, it is not possible using mstump because it would be meaningless. At least, that was the conclusion that we came to when we looked at it a few years ago.

If you want 1D matrix profiles (from AB-joins) stacked on top of each other then you'll need to do this manually with stump.

[vade]

Hi @seanlaw - thanks.

Ill be honest, im new to matrix profiles and stumpy in general, so apologies if some of these questions are self evident after spending some more time with the library / methodology.

One question I have, which came up when looking at the multi dimensional self join / matrix profile example - im curious how the matrix profile for multi dimensional time series is built. I ask because the method I was imaging and which I believe would work for my data would end up looking more like a 1 dimensional matrix profile.

Let me explain

I am working with frames of video, each frame being a data point in t = 0 to t = n. Each data point at a specific t is a vector of floating point numbers in high dimensionality (lets call that d)

To construct a multi dimensional matrix profile, as I understand it, each row for each data point (t) would have its distance compared to the same row for every other t, resulting in a large matrix of (t * t - 1) * d distance measures (more or less).

With a single dimensional matrix profile, each single data point at t results in a one dimensional matrix profile of distance measures (more or less)

For my use case, not doing a row-wise similarity, but vector to vector similarity (in my case, cosine distance has been our metric of choice) would allow us to compute a single dimensional distance value for each t.

Given that profile, we should be able leverage the arbitrary a / b joins for a single dimensional time series?

I hope what I've written above makes some sense - I've yet to have my second coffee ;)

Thank you again @seanlaw for your time and answers!

[seanlaw]

@vade Unfortunately, I don't think I understand. Are you able to provide a simple but small (fictitious) example, say, with n = 20? In matrix profiles, we only refer to "dimensions" as the total number of independent 1-D time series that you have. So, for:

import numpy as np

d = 3
n = 20
T = np.random.rand(d * n).reshape(d, n)

This would produce a 3-dimensional data (i.e., 3x 1D time series) and each of the 1D time series would be of length n = 20. Note that each of the dimensions have no relation to each other and, when computing the multi-dimensional matrix profile, the subsequences in the first dimension (T[0]) would only be compared to all other subsequences in T[0] (and never compared with T[1] or T[2]).

If you want to compare all three dimensions with each other then you need to concatenate all dimensions together into a single 1D time series (and append an np.nan to the end of each time series so that you know where one time series begins/ends):

T_concat = np.concatenate([T, np.full((T.shape[0],1), np.nan)], axis=1).flatten()

[vade]

I think I see, I misunderstood what was meant by dimensionality in the matrix profile world. Interesting ;)

in my example, I have

d = 1280 (a vector describing a single frame - in my example)
n = lets say 20 frames
T = np.random.rand(d * n).reshape(n, d) 

however, I dont treat independent vector indices as an individual independent time seres. I still have 'a single time series'

In my use case (for dimensionality reduction reasons as well as semantic reasons) we would consider this a single time series where where the entire vector (of dimensionality d) at T[0] is compared to all other T's using a vector distance metric like cosine similarity.

I dont know what the proper terminology is now in matrix profile land for this sort of time series. Its a 'single' time series, where each data point can be compared to every other, they just happen to be 1280 dimensional vectors, and the distance metric would be cosine. We ignore looking at individual vector indexes.

Is that helpful?

[vade]

Yup, im aware. Apologies if it came off otherwise. In my case, the continuous sets of numbers might be the distance metric between frames for the vectors, rather than the the 'd' distances for each dimension of the vectors.

This would in theory allow me to compute arbitrary joins as each dimensions of each vector do not constitute individual time series, but a "single semantic data point".

It sounds like I'd need to manually make my own matrix profile?

[vade]

Ah. I think I see what you are saying - the issue is that the computation is for pairwise distances between groups of sequential observations - and those become the m dimensional coordinates (m being the sliding window size). Given that I have d dimensional vectors, what is the methodology to compute pairwise distances over m d dimensional vectors? Perhaps compute the centroid for each pair and then the distance between them?

Im beginning toI see your point.

[NimaSarajpoor]

@seanlaw: please feel free to correct me if I am wrong

@vade
I think you should try to draw your multi-dim data!!!

So, when I read this:

In my use case (for dimensionality reduction reasons as well as semantic reasons) we would consider this a single time series where where the entire vector (of dimensionality d) at T[0] is compared to all other T's using a vector distance metric like cosine similarity.

I quickly started plotting something for myself so I can better understand what's going on:

Now, there are a few questions that you may want to ask yourself:
Are you just interested in finding the distance between T[i] and T[j]? If yes, then simply use pairwise distance on data set S, which is a 2D array of size len(T)-by-d, where each row represent one single point of your time series, which is an array of size d.

Or, are you interested in distance between T[i:i+m] and T[j:j+m]? In this case, you should ask yourself:

• how should I combine the cosine distances across the d dimensions? If you know how and you want to use STUMPY, then you should find a connection between your distance measure and the z-normalized Euclidean distance or p-norm distance. Can you find a relationship? If yes, then maybe you can use some functions in stumpy/core.py

• If you want to use (z-normalized) Euclidean distance (instead of cosine distance), then you may want to take a look at lines 444-447 of function match(...) in stumpy/motifs.py. It uses the average of distance profiles of all dimensions. You can use the same approach to calculate 1D distance profile in your case and then build 1D matrix profile.

---

Just a heads up:
The "HOW" of combining distances in multi-dim data is not easy. This often depends on the domain / application. You should think if the way you want to calculate the distances between two subseqs, where each subseq is multi-dim data makes sense or not.

[vade]

Thanks all for the feedback, its much appreciated. I know from my own OSS projects how much time / energy goes into answering questions, so its much appreciated.

I am interested in the latter, and actually comparing two different time series by constructing non self joins, as per the first post example using stump with T_A and T_B, which I have working correctly and very well with 'single dimensional' observations for 2 distinct time series of different lengths.

are you interested in distance between T[i:i+m] and T[j:j+m]? In this case, you should ask yourself:

I think It would be T1[ i : (i + m)] and T2[ j : (j+m)] but either way I'm left with the point you made in your edit above @NimaSarajpoor -

"for this domain of data what is the most appropriate way of calculating pairwise distances to compute a 1 dimensional matrix profile."

Thank you all again for your input and ill take a look at the notes / code mentioned above.

[NimaSarajpoor]

@vade
Also, there is one more thing you can think about / try:

Let's say your data is T_A and you want to treat each of its subseq as a query. (each query is a multi-dim subsequence, where each single point in your subsequence has d features.) And, you want to find the best-match to the query in a second time series T_B. One idea can be to perform mstumped on T_B first (self-join on T_B). Here, you are trying to first find the good features and throw out the garbage ones to reduce the dimension d. This can be time consuming. So, maybe just ignore it (for now) and assume all d features are important. Then, you can start sliding over the time series T_A (by only considering the selected features) and then for each Query, use stumpy.match which I think is able to perform pattern matching in multi-dim data. Just set the param max_matches=1 and you should be almost good. You may need to set other parameters but I think you will get the idea when you read the docstring and start using it.