Database Reference
In-Depth Information
Jobs: Progress and metrics of stages, tasks, and more
The jobs page, shown in
Figure 8-2
, contains detailed execution information for
active and recently completed Spark jobs. One very useful piece of information on
this page is the progress of running jobs, stages, and tasks. Within each stage, this
page provides several metrics that you can use to better understand physical
execution.
The jobs page was only added in Spark 1.2, so you may not see it in
earlier versions of Spark.
Figure 8-2. The Spark application UI's jobs index page
A common use for this page is to assess the performance of a job. A good first step is
to look through the stages that make up a job and see whether some are particularly
slow or vary significantly in response time across multiple runs of the same job. If
