Databases Reference
In-Depth Information
One day a new staff member spent most of his day re-creating a chart when a simi-
lar chart already existed, but couldn't be found. In a staff meeting a manager asked if
there was some way that the charts could be loaded into a database and searched.
Storing charts in a relational database would've been a multimonth-long task.
There were hundreds of chart properties and multiple chart variations. Even the pro-
cess of adding keywords to each chart and placing them in a word document would've
been time consuming. This is an excellent example showing that high-variability data
is best stored in a NoSQL system.
Instead of loading the charts into an
RDBMS
, the charts were loaded into an open
source native
XML
document store (eXist-db) and a series of path expressions were
created to search for various chart types. For example, all charts that had time across
the horizontal x-axis could be found using an XPath expression on the x-axis descrip-
tor. After finding specific charts with queries, chart keywords could be added to the
charts using XQuery update statements.
You might find it ironic that the
XML
-based charting system was the preferred solu-
tion of an organization that had hundreds of person-years experience with
RDBMS
s in
the department. But the cost estimates to develop a full
RDBMS
seriously outweighed
the benefits. Since the data was in
XML
format, there was no need for data modeling;
they simply loaded and queried the information.
A search form was then added to find all charts with specific properties. The chart
titles, descriptions, and developer note elements were indexed using the Apache
Lucene full-text indexing tools. The search form allowed users to restrict searches by
various chart properties, organization, and dates. After entering search criteria, the
user performed a search, and preview icons of the charts were returned directly in the
search results page.
As a result of creating the chart search service, the time for finding a chart in the
chart library dropped from hours to a matter of seconds. A close match to the new tar-
get chart was usually returned within the first 10 results in the search screen.
The company achieved additional benefits from being able to perform queries over
all the prior charts. Quality and consistency reports were created to show which charts
were consistent with the bank's approved style guide. New charts could also be vali-
dated for quality and consistency guidelines before they were used by a business unit.
An unexpected result of the new system was other groups within the organization
began to use the financial dashboard system. Instead of building custom charts with
low-level C programs, statistical programs, or Microsoft Excel, there was increased use
of the
XML
chart standard, because non-experts could quickly find a chart that was sim-
ilar to their needs. Users also knew that if they created a high-quality chart and added
it to the database, there was a greater chance that others could reuse their work.
This case study shows that as software systems increase in complexity, finding the
right chunk of code becomes increasingly important. Software reuse starts with find-
ability. The phrase “you can't reuse what you can't find” is a good summary of this
approach.
