When to Build, When to Buy, When to Outsource - Data Just Right: Introduction to Large-Scale Data and Analytics - page 187

Despite the advantages of using open-source solutions for solving data challenges,
many nonobvious costs arise in the course of building-out solutions. One is simply
the time necessary for members of your organization to research and deploy a robust
solution.
The f lexibility and cutting-edge features found in many open-source data-software
packages means that your engineering teams can move quickly to solve problems for
which commercial product solutions are either nonexistent or prohibitively expensive.
The major takeaway is to strive to keep your organization honest about the total costs
of developing in-house solutions using open-source technology, including development
time, salaries, and the cost of using resources that can better be used somewhere else.
Everything as a Service
Organizations are quickly embracing cloud technologies to solve a wide range of
problems. Cloud-based solutions for CRM and accounting software and business pro-
ductivity programs for tasks such as email and word processing are becoming more
commonplace. Many businesses are starting to view cloud-based business productivity
tools as a primary platform, falling back to more traditional in-house deployments as a
secondary option.
Avoiding physical infrastructure investments is one thing, but having someone
manage scalable software services can be even better. As organizations figure out
the best ways to solve data challenges, companies are emerging to provide repeatable
solutions for new customers. There are already many examples of this phenomenon
happening right now. Cloud-based services, such as Amazon's Redshift and Google's
BigQuery, provide completely managed services for tasks that until recently were
handled by software deployed using expensive in-house appliances. Hosted batch-
processing services are also being developed, as companies are exploring how to
reduce the administrative barriers to frameworks such as Hadoop.
In the future, Internet speeds will inevitably get faster, commodity hardware will
get cheaper, and more organizations will find business opportunities in solving repeat-
able problems. Where will all this lead? The most likely scenario is that, eventually,
a majority of common data-processing tasks will be done using tools that provide
various types of data analytics as a service. As the growth of utility computing contin-
ues, this trend should make large-scale data collection, processing, and analysis more
accessible.
Summary
As organizations find new ways to derive value from their data, the number of com-
mercial and open-source data technologies is rapidly growing to provide solutions. The
most accessible innovations in data software have come from the vibrant and f lexible
open-source community. However, the organic growth of open-source data solutions
has resulted in a variety of tools that address overlapping use cases. Some tools are