Tracker improvements

[tdurand]

One good thing of having this nice live view + path view is that we can see all the errors of the tracker 😉, I'll list them in this issue and let's see if we can improve: https://github.com/tdurand/node-moving-things-tracker

First one is something I noticed since a long time but didn't create an issue to track it yet.

Ghost re-targeting on the other lane:

Video: https://youtu.be/vFde4ZqP7Ao

"Frame by frame"

As the bounding box of the left car going down has some area overlap area with the new bbox of the car appearing on the opposite lane, it is retargeted and keep "living" as the same ID.

Potential solutions:

• add a mode in the tracker that prevent abrupt change of directions of the velocity vector , and makes a new ID in this case ... tricky to reason just on the next frame as a bounding box can seems to have changed direction on one frame just because YOLO isn't accurate... Would need to be computed over 3-4 frames , and this would still cause a wrong tracking for some frame before re-assigning a new ID.

• this generally happens at the edges of the frames because YOLO in this case stretches the bbox as the car is partially occluded... Try to add a mode that add a rule which consider that if the velocity vector is pointing toward leaving the frame and is very close the border, just stop the tracking and ignore the bboxes there.... This might be a better solution.

[tdurand]

Other improvement would be to add this idea of "tracker accuracy" , right now it seems everything is well tracked but it would be important to have a way to see which part of the frame is well tracked and do not have much id-reassigments.

Data wise, naive algo idea is that tracker should divide the frame is several areas and count how much reassigments are happening in each one of them and then tag each bbox with this "accuracy" score when they are in this part of the screen.

Visually @b-g had this idea of "blurry" borders .. would love to see some example of this ?

[tdurand]

Another bug found is on stationary object (traffic jam), to make sure that they aren't counted several time. If they are id reassign I think we can't do much but to make sure we at least have a rule in the counter that doesn't count the same id on the same counting line.

It's more an improvement to do to the counting algorithm and not the tracker

[tdurand]

I'm also looking into integrating some ideas of an iteration of the original IOU Tracker paper, the V-IOU: Extending IOU Based Multi-Object Tracking by Visual Information , which formalize some of the things we are already doing with node-moving-things-tracker , but also add some new idea, more here: bochinski/iou-tracker#7

[tdurand]

I have made two improvements: (cc @b-g )

• Counting logic: In case of red light / traffic jam, when cars are stationnary, sometimes as the bbox can go a bit backward we were counting several times the same object, this is not the case anymore, we only count the same tracked item one time for each crossing area

• Tracker Display : I didn't fix the isssue of the ghost re-targeting the other lane yet... as it's pretty tricky to do.. but I at least fixed the display, now when we count an object, we change the background color of it for 1 second, and then it return to initial visual state, this avoid seeing lots of recycled tracked items counted around.

[tdurand]

cc @b-g , some work on the idea of tracker accuracy: https://www.youtube.com/watch?v=me4_f5DwTUE

Note for future self picking up work on this:

• Need to evaluate accuracy on a "rolling period" , for now the POC is doing it from the beggining so if we move the camera / the scene change, it won't affect it immediatly
• See if those Ghost are the only way to evaluate tracker accuracy, or if there are others meaninfull

[b-g]

@tdurand Thanks! I think we should discuss this in our next call.

Basically from my own projects I don't have good experiences with splitting one big thing into tiles. Normally you (or at least me) never get it right in terms of representing with the tiles the problem accurately. Hence what about something more relative e.g. closest neighbors to the tracked object/host?

[tdurand]

Good points ! let's discuss this on friday...

[tdurand]

• heat map
• reverse penalty way of doing it, when it is re-assign it lose opacity

[tdurand]

@b-g , worked on this idea of a heatmap today, let me know what you think, here is a demo, I'm quite happy with the quick prototype:

https://youtu.be/QNJNqWYO_8o

If we go for something like this, we would need those improvements:

• Only draw the heatmap based the last XX seconds / minutes of tracking , so if you change the camera viewpoint or if the scene changes significantly, it reflect the "live" view

• Improve perfs : Related the previous point, for now it is not really performant as with time it draws too much things as it never reset

• Maybe add a modal explaining what the heapmap means ?

[b-g]

Hi @tdurand! Many thanks for the neat prototype! I agree! I think it is already working in principle and it would be great feature!

Only draw the heatmap based the last XX seconds / minutes of tracking ...

Yes! And I would rather vote on a short time window seconds and not minutes.

Improve perfs

What about using an additional small canvas element of the same aspect ratio to draw the detection errors, and not a full blown heatmap lib? Something like:

// every nth frame
// low res e.g. 1/10 -> 192 x 108
function drawErrorMap(erros) {
  // paint canvas white every iteration with low opacity
  background(255, 255, 255, 5);
  // draw errors with low opacity
  for errors {
    fill(255, 0, 0, 5);
    rect(error.x, error.y, 1, 1);
  }
}

Maybe add a modal explaining what the heapmap means?

Yes. Now maybe is the time to introduce a generic ℹ️ or ⍰ element?

Also:
What I currently don't like: I think the high definition of the heatmap suggest a very high accuracy of the "error map" ... which is probably not the case. Hence the viz should IMO rather look like a pixelated heatmap.

[tdurand]

• Yes, seconds indeed, I'll also put this in the config.json file so people can tweak it.

• Yes for perfs I don't think the heatmap library will be an issue, I tried with a very simple one: https://github.com/mourner/simpleheat , which maybe give us a better look and feel than doing it ourself .. The perfs problem are because I never reset and after 1 min I draw thousands of data points on each frame, which will be solve by have a smaller buffer

• For the high definition, I can increase the bluryness, I'll give it a shot you can play with look and feel here: https://mourner.github.io/simpleheat/demo/

• General help : not sure, I think it might be better to have a modal dialog showing up the first time you toggle it on....

[tdurand]

cc @b-g , some progress update screencast:

https://youtu.be/Ax5snJ1rOjc

And more:

• WIP Documentation of tracker accuracy display settings: https://github.com/opendatacam/opendatacam/blob/feature-trackeraccuracy/documentation/CONFIG.md#tracker_accuracy_display

• help modal example, very much to improve, but I think it is enough and we still can avoid this help toggle..
  

TODO behind the hoods:

• Release a new version of node-tracker-moving-thing with a simple way to include the zombie Status in the output
• Think about a better name than zombie ?

[b-g]

Hi @tdurand, Sorry I'm a bit behind with everything due to my NY trip. Sorry!

Basically the heatmap feature is all great! I would just change the color gradient to something a lot simpler and that visualizes better what is actually going on: e.g. just two steps orange and red, as every cluster stands already for "bad" tracking areas.

I would entirely try to avoid anything "green" as the heatmap just accumulate bad tracking areas. This is should imo also be added to the hint modal: transparent = good tracking, heatmap = bad tracking areas

[tdurand]

Good points will do ! and no problem 😉

[tdurand]

A basic orange to red config works much better indeed:

Still todo:

• Release a new version of node-tracker-moving-thing with a way to include the zombies in the output
• Think about a better name than zombie ?

[tdurand]

Ok feature tracker accuracy with heatmap merged