Database Reference
In-Depth Information
cores
14/01/30 20:26:17 INFO cluster.SparkDeploySchedulerBackend:
Granted executor ID app-20140130202617-0001/0 on hostPort
ip-10-34-137-45.eu-west-1.compute.internal:57119 with 1
cores, 2.4 GB RAM
14/01/30 20:26:17 INFO client.Client$ClientActor: Executor
updated: app-20140130202617-0001/0 is now RUNNING
14/01/30 20:26:18 INFO spark.SparkContext: Starting job:
reduce at SparkPi.scala:39
...
Feel free to experiment with your cluster. Try out the interactive console in Scala, for ex-
ample:
> ./bin/spark-shell --master
spark://ec2-54-227-127-14.compute-1.amazonaws.com:7077
Once you've finished, type
exit
to leave the console. You can also try the PySpark con-
sole by running the following command:
> ./bin/pyspark --master
spark://ec2-54-227-127-14.compute-1.amazonaws.com:7077
You can use the Spark Master web interface to see the applications registered with the
master. To load the Master Web UI, navigate to
ec2-54-227-127-14.compute-1.amazonaws.com:8080
(again, remember to
replace this domain name with your own master domain name). You should see something
similar to the following screenshot showing the example you ran as well as the two con-
sole applications you launched:
