Database Reference
In-Depth Information
6.
Now that everything's in place, here is the function that controls the process:
(defn main [data-files agent-count]
(let [agents (map agent (repeat agent-count {}))]
(dorun
(map #(send %1 read-file-amounts %2)
(cycle agents)
data-files))
(apply merge-with + (map force-val agents))))
And we can see this in action:
User=> (def data-files ["data/campaign-fin/pacs90.txt"
"data/campaign-fin/pacs92.txt"
"data/campaign-fin/pacs94.txt"
"data/campaign-fin/pacs96.txt"
"data/campaign-fin/pacs98.txt"
"data/campaign-fin/pacs00.txt"
"data/campaign-fin/pacs02.txt"
"data/campaign-fin/pacs04.txt"
"data/campaign-fin/pacs06.txt"
"data/campaign-fin/pacs08.txt"
"data/campaign-fin/pacs10.txt"])
user=> (def contribs (main data-files 5))
user=> (contribs "|N00026349|")
280
user=> (contribs "|N00001845|")
134121
How it works…
Except for
force-val
, all of the agent-related code is in
main
. Let's walk through the lines
that are of interest:
F
We deine the number of agents that we want to use. Each agent is initialized to a
vector of zeroes of the same length as the number of ields:
(let [agents (map agent (repeat agent-count {}))]
F
Next, after reading the input CSV ile to a sequence of maps and partitioning them
into chunks of equal size, we send each agent the
read-file-amounts
function.
We cycle through the agents until all of the iles are assigned to an agent:
(dorun
(map #(send %1 read-file-amounts %2)
(cycle agents)
data-files))
