Monitoring MapReduce Jobs
A MapReduce job is a unit of processing (query or transformation) on the data stored within a Hadoop cluster. You can view information about the different jobs that have run in your cluster during a selected time span.
- The list of jobs provides specific metrics about the jobs that were submitted, were running, or finished within the time frame you select.
- You can select charts that show a variety of metrics of interest, either for the cluster as a whole or for individual jobs.
You can use the Time Range Selector or a duration link ( ) to set the time range. (See Time Line for details).
Note: Activity Monitor treats the original job start time as immutable. If a job is resubmitted
due to failover it will retain its original start time.
You can select an activity and drill down to look at the jobs and tasks spawned by that job:
- View the children (MapReduce jobs) of a Pig or Hive activity.
- View the task attempts generated by a MapReduce job.
- View the children (MapReduce, Pig, or Hive activities) of an Oozie job.
- View the activity or job statistics in a detail report format.
- Compare the selected activity to a set of other similar activities, to determine if the selected activity showed anomalous behavior. For example, if a standard job suddenly runs much longer than usual, this may indicate issues with your cluster.
- Display the distribution of task attempts that made up a job, by different metrics compared to task duration. You can use this, for example, to determine if tasks running on a certain host are performing slower than average.
- Kill a running job, if necessary.
Note: Some activity data is sampled at one-minute intervals. This means that if you run a very short
job that both starts and ends within the sampling interval, it may not be detected by the Activity Monitor, and thus will not appear in the Activities list or charts.
Page generated May 18, 2018.
<< Monitoring Activities | ©2016 Cloudera, Inc. All rights reserved | Viewing and Filtering MapReduce Activities >> |
Terms and Conditions Privacy Policy |