Databases Reference
In-Depth Information
Build Scalding Apps with Gradle
For the example in an upcoming section, we will need to incorporate extensions to
Cascading, and the
scald.rb
script will not work for that. Instead, let's look at how to use
Gradle to build what is called a “fat jar.” In other words, create a JAR file that includes
all the class dependencies for Scalding that Apache Hadoop would not normally provide.
Note that Gradle version 1.3 or later is required for Scala support.
Starting from the “Impatient” source code directory that you cloned in Git, connect into
the
part8
subdirectory. Next, we'll use a Gradle
build.gradle
script to build a Scalding
app:
apply
plugin:
'scala'
archivesBaseName
=
'impatient'
repositories
{
mavenLocal
()
mavenCentral
()
mavenRepo
name:
'conjars'
,
url:
'http://conjars.org/repo/'
}
dependencies
{
// Scala compiler + related tools, and standard library
scalaTools
'org.scala-lang:scala-compiler:2.9.2'
compile
'org.scala-lang:scala-library:2.9.2'
// Scalding
compile
(
'com.twitter:scalding_2.9.2:0.8.1'
)
// in case you need to add JARs from a local build
compile
fileTree
(
dir:
'lib'
,
includes:
[
'*.jar'
]
)
compile
(
'cascading:cascading-core:2.0.2'
)
compile
(
'cascading:cascading-hadoop:2.0.2'
)
}
jar
{
description
=
"Assembles a Hadoop-ready JAR file"
doFirst
{
into
(
'lib'
)
{
from
configurations
.
compile
}
}
manifest
{
attributes
(
"Main-Class"
:
"com.twitter.scalding.Tool"
)
}
}
