Configuring the MapReduce Service to Save Job History in HDFS
Normally job history is saved on the host on which the JobTracker is running. To enable Map Reduce job history to be saved to HDFS:
- Create a folder in HDFS to contain the history information. When creating the folder in HDFS, set the owner and group to mapred:hadoop with permission setting 775.
- In the Cloudera Manager Admin Console, select the MapReduce service.
- Select .
- Expand the JobTracker (Default) category and click the Paths subcategory.
- Set the Completed Job History Location property to the location that you created in step 1.
- Click Save Changes.
- Restart the MapReduce service.
To change the retention duration:
- Expand the Service-Wide category and click the Advanced subcategory.
- In the MapReduce Service Configuration Safety Valve for mapred-site.xml, specify the mapreduce.jobhistory.max-age-ms property value in milliseconds. For example, to set the value to 60 days, add the following:
<property> <name>mapreduce.jobhistory.max-age-ms</name> <value>5184000000</value> </property>
- Restart the MapReduce service.
<< |