Trajectory evaluation and behavioral scoring using JAABA in a noisy system - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 222

of tracking data. However, imperfect conditions will apply for a majority of behavioral obser-
vation systems in nature. Even in many lab situations, experimenters often have to work with
such conditions to collect relevant data. Our methods offer the potential for investigators to
more successfully work with such data.
Our simple TABU tracking algorithm, by making a few realistic assumptions about the per-
sistence of flies across frames and within blobs, greatly reduces the uncertainty of the initial
image processing data from the algorithm of Ardekani et al. [ 6 ] . It allows us to count flies on
patches with more certainty, and reduces the apparent degree of fly movement on and of of
patches. Error rates are still nonzero, but it is clear that subsequent application of any of the
ML methods we tested here has the potential to increase correct allocation of flies among blobs
from around 90% to over 98%.
Among the algorithms we evaluated, there is no clear winner among the ML methods in
terms of performance. However, for ease of implementation, and robustly high performance,
the GentleBoost algorithm natively implemented in JAABA represents a reasonable choice
for future work. It performed well for the Multifly classifier, and consistently well for Sex
and Chase. We emphasize that use of JAABA (and all the behavioral annotation algorithms
we tested) requires fly tracking data as input, thereby necessitating pre-processing using an
algorithm such as TABU before use. Such a pre-processing algorithm needs to be able to
construct tracks successfully in nonideal conditions, and when the number of objects being
tracked is unknown, a problem that is known to be extremely challenging [ 1 ] .
In the test videos, we were able to integrate multiple classifiers, such as painted marker
color and Sex to improve our ability to score accurately. We were also able to integrate Sex
and Chase in order to obtain richer biological data than was available from individual classi-
iers. We recapitulated the experimental sex ratio conditions (i.e., higher rates of observation
of female prevalence correspond with higher female sex ratios) in the untrained videos. Im-
portantly, we detected interesting paterns in behavioral diferences among the trials related
to sex ratio—there were trends in the frequency of male-male Aggression, but not, interest-
ingly, Courtship. It might be that as males spend more time fighting, they have less time for
opportunity to court. The ability to detect such meaningful behavioral trends, in naturalistic
setups, will be of great interest to ethologists.
In sum, our methods produce improved performance both in terms of accurate identiica-
tion of the number of flies in a blob (and, therefore, the number of flies in a frame at any giv-
en moment), and in terms of generation of tracks for individual flies. Both of these types of
information are crucial for analysis of fly (and other animal) group behavior. Flies are social
animals that actively aggregate and interact in groups. The sizes of these groups is therefore
a key diagnostic of the behavior of those flies, and varies with factors such as genotype, sex
ratio, etc. Therefore, the methods we present here provide the opportunity for researchers to
use automated methods to generate large quantities of such data in an experimental context.
A more difficult remaining challenge is to automatically recognize interactions between flies,
such as courtship and acts of aggression. Methods (including JAABA) are being developed to
attack this problem. Creating, and error correcting, fly trajectories is a necessary first step in
taking advantage of this work.
References
[1] Branson K, Robie AA, Bender J, Perona P, Dickinson MH. High-throughput ethomics
in large groups of Drosophila . Nat Methods. 2009;6:451-457.