Database Reference
In-Depth Information
All was confusion until a vice president came out of the corner office and con-
fronted our questing consultant with a stern (but whispered), “Shut up with the
questions!”
It seems that the company had surreptitiously set up a foreign factory to which they
shipped partially finished work in unmarked railcars for final processing. Said factory's
production data was spread across the four factories via secret nightly processing rou-
tines. Through a distinct lack of foresight, this allocation process was tied to the nightly
data constraints and when those routines were turned off, the plant identifier mani-
fested itself in the data.
This rather convoluted subterfuge was because the fifth plant was union-free in a
company that was otherwise heavily unionized. Did I mention that the collective bar-
gaining agreement included the right to organize all new plants and that a breach of the
agreement was cause for industrial action? Whoops. I wonder what it is like to have your
very own case history with the national Labor relations Board.
no, this unique professional footnote did not make our sadder but wiser pro-
tagonist happy. nor did it give his consulting manager a toothsome smile. nor, for
that matter, did it make the owner of the consulting firm particularly cheery. In
fact, it made them all very, very sad because they were dragged into a lengthy and
costly legal controversy that was not of their making simply for doing their good data
duty. I told you that bad data is not fair. In conclusion, I pose this question: Was this
really bad data or was it simply data used badly? This was a matter to be settled in
litigation.
2.2.7.4 What Have We Learned from These Amusing/Instructive/Disturbing Anecdotes?
he
first lesson is easy: bad data kills everything it touches, regardless of cause, be that com-
plexity, consultant hubris, or secrecy. There are real consequences to getting data wrong
and as you are the responsible Essbase developer, that means
your
job,
your
engagement,
and
your
business are all vulnerable.
The second lesson is that you, the Essbase developer, must create systems with
good
data that eschew complexity and use tools fit for purpose. As you may have noticed
in the above, I do not consider Essbase Load rules a proper place to do EtL and they are
certainly not fit for that purpose. But, exactly why do I so horribly denigrate doing EtL
with Load rules? Three reasons stand out:
1. Load rules are slow.
2. Load rules have limited functionality.
3. EtL should all be done in one place, not partially in the extraction and partially
within Essbase, which makes multiple transforms much harder to debug.
That is two out of the three dealt with, what about that last “tale,” the one about
secrecy? Ironically, that organization actually had the right idea about data quality,
although lying to their employees bit them in the you-know-where, and rightfully so,
because what everyone thought was good data, really was not. This brings us right back
to the quality issue. As all of this topic's readers are forthright and honest Essbase devel-
opers who only want to do good, they will never fall into that trap. honesty, in data
as in life, is always the right policy. So, the third lesson is that you, the ethical Essbase
developer, must understand your data sources so that you validate them correctly. In the
end remember that you were the last one to touch it, so whatever is in the cube is your
responsibi lit y.
