Trajectory evaluation and behavioral scoring using JAABA in a noisy system - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 217

Sex classifier, integration was performed by applying the Sex classifier to frames where the
color of the painted marker is uncertain.
In order to understand the biological significance of the Chase classifier, we need to under-
stand the sex of the individuals involved in the behavior. Courtship and aggression are known
to be important components of fly social behavior [ 3 ] . These are usually male initiated, often
characterised by chasing, and are directed at females and males respectively. Females rarely if
ifever chase other flies. Taking Chase together with Sex and Multifly, we created two composite
behaviors: Aggression and Courtship. In the case where Multifly = 0, Sex = 1, and Chase = 1,
Aggression was defined as a male chasing another male, and Courtship was defined as a male
chasing a female. To compare our test videos, we created a composite behavior profile for
each. That profile comprised of the percentage of frames that (a) contained Multifly blobs; (b)
contained at least one female; (c) contained Aggression; and (d) contained Courtship.

3 Results
We begin by evaluating the performance of the basic blob-recognition algorithm from
Ardekani et al. [ 6 ] , and the change in accuracy after processing the data with TABU, for the
basic task of recognizing fly number and for joining and leaving events. The empirical “real”
results are obtained from manual annotation. Results are shown in Table 1 . Let e be the es-
timated number of flies in a frame for a given method, n be the actual (manually annot-
ated) number, and τ be the total number of frames. We estimate overall counting error, E , as

(where the denominator is n + 1 to avoid division by zero). This represents an
approximate per-fly probability of being miscounted. Directionality, D , is calculated similarly,

, and demonstrates the chances of being consistently over- or under-counted.
Joining or leaving events, “Jump,” are reported as the per-frame probability of either a change
in blob number, or a trajectory starting or ending. Results are shown for five separate videos
(Rep).

Table 1
Performance of the Blob Algorithm Output (Blob), and TABU Trajectory Output

Rep Blob E TABU E Blob D TABU D Real Jump Blob Jump TABU Jump

1 0.177 0.106 − 0.145 0.074 0.013 0.085 0.031

2 0.138 0.197 − 0.101 0.192 0.009 0.116 0.017

3 0.156 0.106 0.077 0.036 0.011 0.023 0.014

4 0.11 0.09 0.025 0.048 0.003 0.023 0.015

5 0.123 0.124 − 0.074 0.098 0.012 0.127 0.042

Mean 0.141 0.125 − 0.044 0.090 0.010 0.075 0.024

Counting error ( E ), and directionality ( D ) bias in counting is shown. Empirical (Real) and estimated fly patch-joining
or leaving rates (Jump) are also shown for raw blob data and processed trajectories.
By using TABU to create our trajectories, we have obtained more accurate data than was
provided by the raw blob counts ( Table 1 ). We have also greatly reduced the correlation