Database Reference
In-Depth Information
Combining Big Data and SQL Server Tools for Better
Solutions
As with many platforms, both Hadoop and SQL Server have strengths and
weaknesses. By using the tools together, you can help leverage the strengths
of each platform while mitigating the weaknesses.
The Hadoop platform's strengths derive from its capabilities to scale out
easily to accommodate growing amounts of data. It also can handle a wide
variety of data formats with little up front transformation effort. However,
it has few restrictions or validations on the data it stores, and it doesn't
have the same end-user capabilities and ecosystem that SQL Server has
developed.
SQL Server, however, handles enforcing data quality and consistency much
better than Hadoop, and a wide variety of tools and clients enable you
to perform different analytic, reporting, and development tasks with SQL
Server. Challenges with SQL Server include scaling it to handle massive
amounts of data and also support for easily storing loosely formatted data.
Why Move the Data?
When your environment includes both Hadoop and SQL Server, you will
sometimes need to move the data between the two systems. One example of
this is importing data into Hadoop from an online transactional processing
(OLTP) system using SQL Server, such as sales data from an
order-processing system.
There are a number of reasons to move data from SQL Server into Hadoop:
•
Archival and data retention
: Hadoop storage can be less expensive
than SQL Server, and the costs of scaling up are generally less. In
addition, keeping extensive, infrequently accessed historical data in an
OLTP database creates overhead that negatively impacts performance.
Moving this data to Hadoop can save the physical resources on your
SQL Server for processing transactions quickly and efficiently.
•
Analytics
: Hadoop offers tools like Mahout for data mining and
machine learning that can be applied to your Hadoop data. These tools
leverage Hadoop capabilities for processing large amounts of data.
Moving data from SQL Server to Hadoop enables the use of these tools.
