BUG: Can't reconstruct with reconstruct_tracker

[larsoner]

I have a paradigm with TrackerDealer having two TrackerUD instances, and when I try to use reconstruct_tracker I get this error:

  File "/home/larsoner/Documents/python/larsoner/adaptivetempcoh/behavioral.py", line 18, in <module>
    trackers = reconstruct_tracker(fname_tab)
  File "expyfun/io/_parse.py", line 154, in reconstruct_tracker
    tr[-1].respond(r)
  File "expyfun/stimuli/_tracker.py", line 219, in respond
    raise RuntimeError('Tracker is stopped.')
RuntimeError: Tracker is stopped.

@maddycapp27 is this use case intended to be supported by reconstruct_tracker, and if so, this is a bug, right?

[maddycapp27]

I have a few thoughts. Since we recently changed trackers to stop at n trials (or reversals) rather than n + 1 trials (or reversals) I could see this being an issue for data collected before the fix and reconstructed after the fix. My other thought is that if you're using a TrackerDealer you may want to run reconstruct_dealer instead -- there's no reason that reconstruct_tracker wouldn't work though (particularly since reconstruct_dealer calls reconstruct_tracker).

[rkmaddox]

Ahhh, yeah the reconstruction code will need to be updated to reflect the additional arguments.

[rkmaddox]

And the new arguments will need to be written to the init parts of the expyfun log.

[larsoner]

I could see this being an issue for data collected before the fix and reconstructed after the fix.

I'm pretty sure the code was run after the fix.

My other thought is that if you're using a TrackerDealer you may want to run reconstruct_dealer instead

Same error.

the reconstruction code will need to be updated to reflect the additional arguments.

Which arguments? Do you mean the new arguments / new stopping behavior? If so I'm not sure why it wasn't caught by some test.

[larsoner]

And the new arguments will need to be written to the init parts of the expyfun log.

I just used the default for repeat_limit, though

[maddycapp27]

Hmm if you ran the original trackers after we changing the stopping criteria for the trackers then the length of the responses wouldn't be the source of the issue...

The new arguments shouldn't be an issue. It just makes a dict from the log file to fill in kwargs.

[rkmaddox]

I just used the default for repeat_limit, though

The default repeat_limit is a change in behavior from how it used to work (it now counts them as reversals).

If so I'm not sure why it wasn't caught by some test.

The tests didn't hit the limit.

[larsoner]

But the code in both places (during original run, during reconstruction) just uses the defaults as @maddycapp27 says, right? So I'm not sure where the change of behavior should become a problem.

[rkmaddox]

Ohhhhhh. Right. So your code used the new reversal_limit behavior. And the reconstruction code also does. That is trickier. Is it stopping one trial/reversal short? Wonder if it's something with the reversal counting that we changed.

[rkmaddox]

Can you send the data files our way?

[maddycapp27]

@larsoner looks like your tracks have 11 reversals when stop_reversals is 10. The code is hitting the error when you get to 10 reversals as well. We've been doing stop_trials instead of stop_reversals so I'll have to run a test track to see if that's a consistent bug and if so how to fix it.

@rkmaddox you are right about the logging in TrackerUD being incomplete though--I do need to submit a PR to add repeat_limit to the dict.

[larsoner]

I'll ask folks to double-check the version asserted in the experimental file (I don't have access to it ATM) but it should have been version '1cee0b2' which was the merge commit for #323. But if it's hitting the error even with 10 trials, then there must be at least some bug to fix in the meantime.

[larsoner]

... I'm also going to open a PR to record the expyfun version used to run the experiment, I can't believe we don't already do it!

[maddycapp27]

if it's hitting the error even with 10 trials

It's hitting the error right after 10 reversals. When I generated the plot it looked like it should with the behavior from #323.

[larsoner]

So is there some bug to be fixed, or do you think it's likely that we ran with an older version (we still need to check...)?

[maddycapp27]

I'm guessing that the version issue is significantly more likely. If you confirm that that is not the case I'm more than happy to go bug hunting though.

[larsoner]

Oops, error at our end. We ran on d27b7c8. Really sorry to waste your time :(

[maddycapp27]

It's alright! Glad it turned out to not be a bug :)

[larsoner]

Okay we collected more data, this time with version '79e05c6' (current master as of today). On all three subjects when doing reconstruct_dealer I get Tracker ######## has not stopped. All Trackers must be stopped. Is this expected? If not, any interest in having a look at the .tab file to see what the problem is?

[larsoner]

@maddycapp27 any ideas / time to look this week?

[maddycapp27]

Sorry, just looking at this right now. The TrackerUD must not be hitting line 301 where it prints out all the info???
That should happen at the end of td.respond() for the last trial. I didn't think that changing the stopping criteria in stop_here() would do that?? But that doesn't mean I didn't make a mistake there.
I'm happy to look at the tab file--it'll just be tomorrow before I'm able to get to it. That ok?

[larsoner]

Yeah that's fine, will email a couple of failing files.