Spark SQL - Learning Spark - page 167

to query it via HiveContext.sql or SQLContext.sql . You do so using the SchemaRDD's
registerTempTable() method, as in Examples 9-9 through 9-11 .

Temp tables are local to the HiveContext or SQLContext being
used, and go away when your application exits.

SchemaRDDs can store several basic types, as well as structures and arrays of these
types. They use the HiveQL syntax for type definitions. Table 9-1 shows the sup‐
ported types.

Table 9-1. Types stored by SchemaRDDs

Spark SQL/HiveQL type Scala type Java type Python

Byte / byte int / long (in range of -

128 to 127)

TINYINT Byte

Short / short int / long (in range of -

32768 to 32767)

SMALLINT Short

Int / int int or long INT Int

Long / long BIGINT Long long

Float / float

FLOAT Float float

Double / double DOUBLE Double float

DECIMAL Scala.math.BigDeci

mal

Java.math.BigDeci
mal

decimal.Decimal

STRING String String string

BINARY Array[Byte] byte[] bytearray

Boolean / boolean BOOLEAN Boolean bool

TIMESTAMP java.sql.TimeStamp java.sql.TimeStamp datetime.datetime

list , tuple , or array

ARRAY<DATA_TYPE> Seq List

MAP<KEY_TYPE,
VAL_TYPE>

Map Map dict