René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"QAPage","mainEntity":{"@type":"Question","name":"STIMP result mechanism","text":"
Hi community,
\nI have a question about the class STIMP which is implementation of SKIMP algorithm. For better understanding, I used the dataset in the official SKIMP Dataset page: Termitte DNA https://sites.google.com/view/pan-matrix-profile/datasets
\nThe dataset length is 16326
\n
\n
I used stimp and get pmp results.
\n
I checked the length of outputs respectively. len(pmp.M_) len(pmp.PAN_). They have same length that is what I expected.
\n
I want to control the M_values. pmp.M_[:100].  The largest value is 13060. Why this value is greater than half of the dataset, it is closed to the length of the dataset.
\n
\n
My other question is, let's say I picked the first element of M_ array (M_[0]. As I understood, this is subsequence length. Then I investigate it in PAN_. I would expect to see the length of 1 values in the related PAN_ array as M_[0]*2.
\n
\n
","upvoteCount":1,"answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"
\n
I mean based on the 1000 length T that you have provided, why pmp._PAN[0] differs from m = 401 stump = stumpy.stump(T, m) stump[:,0]
\n
\n
I think I see the problem. pmp._PAN stores the matrix profiles ordered from smallest m to largest m. However, the matrix profiles are NOT computed in this order and, instead, are computed in \"breadth-first-search\" (BFS) order. Thus, in order to retrieve the correct BFS order, we must follow the procedure in the pan transformation function (shown in the permalink above) but this example should work:
\n
import numpy as np\nimport stumpy\n\nT = np.random.rand(1000)\npmp = stumpy.stimp(T, percentage=1.0)\npmp.update()\n\n# retrieve the BFS indices for the matrix profiles that have already been processed\nidx = pmp._bfs_indices[: pmp._n_processed]\n\n# retrieve the specific raw matrix profiles from the pan matrix profile and make a copy\n# Note that `_PAN` is a private variable and should NEVER be accessed directly as there is no guarantee that it will not change!!\nprocessed_mps = pmp._PAN[idx].copy()  # Don't do this!!\n\n# Compute the first matrix profile by hand\nmp = stumpy.stump(T, pmp.M_[0])\n\n# Compare `mp[:, 0]` and `processed_mps[0, :len(T)-pmp.M_[0]+1]`\n
\n
I think what you've helped me identify is that it is somewhat difficult to retrieve the raw matrix profiles from the pmp object. I will take a look at whether there is a more convenient way to do this. Perhaps, it would make sense to add another class attribute for something like pmp.P_ that represents the raw matrix profiles but ordered by BFS?
","upvoteCount":0,"url":"https://github.com/stumpy-dev/stumpy/discussions/1043#discussioncomment-11177428"}}}