Database Reference
In-Depth Information
CHAPTER 2
Downloading Spark and Getting Started
In this chapter we will walk through the process of downloading and running Spark
in local mode on a single computer. This chapter was written for anybody who is new
to Spark, including both data scientists and engineers.
Spark can be used from Python, Java, or Scala. To benefit from this topic, you don't
need to be an expert programmer, but we do assume that you are comfortable with
the basic syntax of at least one of these languages. We will include examples in all
languages wherever possible.
Spark itself is written in Scala, and runs on the Java Virtual Machine (JVM). To run
Spark on either your laptop or a cluster, all you need is an installation of Java 6 or
newer. If you wish to use the Python API you will also need a Python interpreter
(version 2.6 or newer). Spark does not yet work with Python 3.
Downloading Spark
The first step to using Spark is to download and unpack it. Let's start by downloading
a recent precompiled released version of Spark. Visit
http://spark.apache.org/down
loads.html
,
select the package type of “Pre-built for Hadoop 2.4 and later,” and click
“Direct Download.” This will download a compressed TAR file, or
tarball
, called
spark-1.2.0-bin-hadoop2.4.tgz
.
Windows users may run into issues installing Spark into a direc‐
tory with a space in the name. Instead, install Spark in a directory
with no space (e.g.,
C:\spark
).
