The following data has been published in PNAS by Newby, et al.

https://doi.org/10.1073/pnas.1804420115
Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D
Jay M. Newby, Alison M. Schaefer, Phoebe T. Lee, M. Gregory Forest, Samuel K. Lai


Sensitivity analysis for randomized 2D synthetic test videos results showing the percentage of false positives

NNT: ~0.5% false positives across all SNR values tested.

Mosaic: ~2% false positives when SNR > 3, but > 20% when SNR < 3.

ICY: ~5% false positives across all SNR values.


Mosaic

NNT



The two windows above are displaying the same experimental video but the video on the left has been analyzed with Mosaic, one of the most commonly used particle tracking softwares, while the video on the right has been analyzed with the automated Neural Net Tracker (NNT). Software like Mosaic rely on a human user to set subjective parameters on their videos prior to tracking particles. The Neural Network Tracker however, uses statistics and machine learning to accurately identify particles automatically without any input from a human user, thus generating objective results. As the frames in the videos above progress, phantom particles become especially apparent for Mosaic, resulting in false positives. The NNT however, has refrained from generating phantom particles while not increasing the amount of false negatives. In fact, the false positive rate for the NNT is 0.5% on average.



Sensitivity analysis for randomized 2D synthetic test videos results showing the percentage of false negatives

NNT: <20% false negatives for all SNR > 2.5.

Mosaic and ICY: >40% false negatives for all SNR > 2.


Mosaic

NNT



Above are the same videos used in the first example, however here the tunable parameters of Mosaic have been altered in an attempt to reduce the amount of false positives recorded. We can see that the amount of false positives have been reduced, however, this comes at the cost of an increase in false negatives. While select false negatives can be seen in the NNT analyzed video as well, there are fewer compared to Mosaic and without generating an increase in false positives. These examples illustrate that subjective softwares like Mosaic can only identify a subset of particles from given tuned parameters whereas the NNT essentially captures all of the particles present without generating additional phantom particle data and is able to do so objectively.



Sensitivity analysis for randomized 2D synthetic test videos results showing the predictions per frame vs. SNR with ground truth of 100 particles.

NNT Does well under low SNR conditions by making fewer predictions, and the number of predictions made are more accurate than Mosaic and ICY. NNT results are reasonably agreeable to theoretical max at higher SNR


Methods




https://doi.org/10.1073/pnas.1804420115
Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D
Jay M. Newby, Alison M. Schaefer, Phoebe T. Lee, M. Gregory Forest, Samuel K. Lai