NNT: ~0.5% false positives across all SNR values tested.
Mosaic: ~2% false positives when SNR > 3, but > 20% when SNR < 3.
ICY: ~5% false positives across all SNR values.
The two windows above are displaying the same experimental video but the video on the left has been analyzed with Mosaic, one of the most commonly used particle tracking softwares, while the video on the right has been analyzed with the automated Neural Net Tracker (NNT). Software like Mosaic rely on a human user to set subjective parameters on their videos prior to tracking particles. The Neural Network Tracker however, uses statistics and machine learning to accurately identify particles automatically without any input from a human user, thus generating objective results. As the frames in the videos above progress, phantom particles become especially apparent for Mosaic, resulting in false positives. The NNT however, has refrained from generating phantom particles while not increasing the amount of false negatives. In fact, the false positive rate for the NNT is 0.5% on average.
NNT: <20% false negatives for all SNR > 2.5.
Mosaic and ICY: >40% false negatives for all SNR > 2.
Above are the same videos used in the first example, however here the tunable parameters of Mosaic have been altered in an attempt to reduce the amount of false positives recorded. We can see that the amount of false positives have been reduced, however, this comes at the cost of an increase in false negatives. While select false negatives can be seen in the NNT analyzed video as well, there are fewer compared to Mosaic and without generating an increase in false positives. These examples illustrate that subjective softwares like Mosaic can only identify a subset of particles from given tuned parameters whereas the NNT essentially captures all of the particles present without generating additional phantom particle data and is able to do so objectively.
NNT Does well under low SNR conditions by making fewer predictions, and the number of predictions made are more accurate than Mosaic and ICY. NNT results are reasonably agreeable to theoretical max at higher SNR