Database Reference
In-Depth Information
The following formulas are a little complicated and dificult to read in lisp notation. However,
there's a good overview of this process, with formulas, on the Wikipedia page for Algorithms to
calculate variance (
http://en.wikipedia.org/wiki/Algorithms_for_calculating_
variance
). In order to somewhat simplify this example, we'll only calculate the mean
and variance.
Getting ready
For this, we'll need to have easy access to the
reducers
library and the Java
Math
class:
(require '[clojure.core.reducers :as r])
(import '[java.lang Math])
How to do it…
For this recipe, we'll irst deine the accumulator data structures and then the accumulator
functions. Finally, we'll put it all together.
1. We need to deine a data structure to store all of the data that we want to
accumulate and keep track of:
(def zero-counts {:n (long 0), :s 0.0,
:mean 0.0, :m2 0.0})
2.
Now, we'll need some way to add data to the counts and accumulation.
The
accum-counts
function will take care of this:
(defn accum-counts
([] zero-counts)
([{:keys [n mean m2 s] :as accum} x]
(let [new-n (long (inc n))
delta (- x mean)
delta-n (/ delta new-n)
term-1 (* delta delta-n n)
new-mean (+ mean delta-n)]
{:n new-n
:mean new-mean
:s (+ s x)
:m2 (+ m2 term-1)})))
