Mouth Tracking Using Mean Shift and Covariance Trackers

Matlab implementation ( Meanshift and Covariance Trackers )
For academic reason, files are protected by password. Ask for the password by using your university email (yahoo.com, gmail.com, hotmail.com, etc. will not be considered)

Experiments

The videos we use in this experiment are self-taken in a normal office environment by using the Sony DCR-SR42 camera. Original images are then compressed to the size of 288 by 368 by 3 using Vegas Pro[1]. The sampling rate is 25Hz. The mean shift tracker [2] uses quantified color (16 bins per color) as the feature vector while the covariance tracker [3] utilizes both spatial and color information. For both trackers, a 50 by 50 bounding box is used by default and its initial position is set manually to catch the target of interest. In the next section, we will compare two trackers' performances in different task scenarios.

Discussion

1) Accuray and efficiency

In this experiment, the tracking performance is evaluated based on two criterions – accuracy and efficiency. A video of target moving smoothly is used for the tracking test, and results are shown at [meanshift] and [covariance]. Based on visual judgment, two trackers perform equally well in terms of accuracy in this case. To measure the computational complexity of each tracker, we record the amount of time each tracker took to process a single frame. We conclude that mean shift tracker is about 50 times faster than covariance tracker.

2) Moving speed of target

In a tracking task, moving speed of the target plays a role in the performance. In this experiment, we tested how movement speed affects the tracking performance. For comparison, we took two videos of a target moving in similar patterns but with different speed. The tracking results using mean shift tracker are shown at [msSpeed1,msSpeed2] and those of covriance tracker are [covSpeed1,covSpeed2]. We found that both trackers did a reasonable job in tracking the target at different speed. However, in fast moving conditions, the mean shift tracker is sensitive to the initial position of the tracking window.

3) Patch size and convergence criterion for mean shift tracker

We notice that mean shift tracker is sensitive to several parameters such as the patch size and convergence criterion. In this experiment, we test the tracker's performance using different patch sizes and convergence criterions. The first video has a patch size of 50 by 50 and convergence error of 1 [p50c1]. In the second video we reduce the patch size to 40 by 40 [p40c1]. From above we summarize that the mean shift tracker can successfully track the target using patches of different sizes. However, it fails if we set the convergence criterion to be 2 [p40c2]. Therefore, a small convergence error is needed to closely follow the target when the patch size is small.

4) Occlusion

In this experiment, we are curious to see whether trackers can still follow the target when it is partially or totally occluded by other objects. For this purpose, we designed three tasks to test the tracker's performance. In the first task, the target will be partially occluded by an object in different color. In the second task, the occluding object has a similar color instead. In the third task, the target will be totally occluded during certain time. The results are shown at [msOcclusion1, msOcclusion2, msOcclusion3] and [covOcclusion1, covOcclusion2, covOcclusion3]. We conclude that if the target is partially occluded by an object with a totally different color, the mean shift tracker may "cross" the occluder and still follow the target. Similarly, the covariance tracker may "jump over" the occluder and catch the target. Neverthless, in this example, it loses track because of its relatively small search region. Understandably, an occluding object with similar color will pull the bounding box off the right track and a complete occlusion results in the loss of the target.

References

[1] Vegas pro can be found at www.sonycreativesoftware.com/vegaspro.

[2] Fatih Porikli, Oncel Tuzel and Peter Meer, “Covariance tracking using model update based on means on Reimannian manifolds”, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2006.

[3] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, May 2003.