Using the LZO Parcel
This section describes how to install and use the LZO parcel in Cloudera Manager.
The Repository
Add the appropriate repository to Cloudera Manager’s list of parcel repositories. The HADOOP_LZO parcel will then become available on the parcel management screen. If required, the repository can be mirrored in the same way as the CDH repo.
The public customer repositories can be found at: http://archive.cloudera.com/gplextras/parcels/.
You must choose the specific version of the HADOOP_LZO parcel for each
Impala version. Starting with Cloudera Manager 4.8, only Impala versions 1.2.1 or
later are supported, so you must use the LZO parcel versions according to the
following table:
Impala Version | LZO Parcel Version |
---|---|
1.2.1 | HADOOP_LZO-0.4.15-1.gplextras.p0.33 |
1.2.2 | HADOOP_LZO-0.4.15-1.gplextras.p0.37 |
1.2.3 | HADOOP_LZO-0.4.15-1.gplextras.p0.39 |
Activation
The HADOOP_LZO parcel can be downloaded, distributed, and activated in the same way as the CDH parcel. Once activated, it will be necessary to reconfigure and restart services that intend to use LZO functionality.
MapReduce
- Add the following entries to the MapReduce Client Environment Safety valve:
- Under the Configuration > View and Edit tab, search for "MapReduce Client Safety".
- In the MapReduce Client Environment Safety Valve, enter the following two lines:
- HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
- JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
- Add the LZO codecs to the io.compression.codecs property under the MapReduce Service:
- Under the > tab, search for "io.compression".
- In the Compression Codecs property, click in the field, then click the + sign to open a new value field.
- Add the following two codecs:
- com.hadoop.compression.lzo.LzoCodec
- com.hadoop.compression.lzo.LzopCodec
- Save your configuration changes.
- Restart MapReduce.
- Redeploy MapReduce Client Configuration.
Oozie
- Go to /var/lib/oozie on each Oozie server and symlink the Hadoop LZO JAR.
- /opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
- Restart Oozie.

HBase
- Restart HBase.
Impala (1.0 or later)
- Restart Impala.
Hive
- Restart the Hive server.
Sqoop
- Add the following entries to the Sqoop Service Environment Safety valve:
- HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
- JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
- Restart the Sqoop service.

<< |