Using Parcels
Cloudera Manager, as of version 4.5, supports parcels as an alternate form of distribution for CDH and other system packages. Among other benefits, parcels provide a mechanism for upgrading the packages installed on a cluster from within the Cloudera Manager Admin Console with minimal disruption. Your cluster can continue to run on the existing installed components while you stage a new version across your cluster, without impacting your current operations. You can then cut over to the new version using a rolling restart that will help avoid disruptive downtime of your operations.
You can upgrade individual parcels, or multiple parcels at the same time -- for example, upgrading CDH and Impala together.
Upgrading or Adding Software on Your Cluster
To enable upgrades or additions with minimal disruption, parcels are handled in three steps: Download, Distribution, and Activation.
Downloading a parcel downloads the appropriate software to a local parcel repository on the Cloudera Manager server, where it is available for distribution to the other nodes in any of your clusters managed by this Cloudera Manager server. You can have multiple parcels for a given product downloaded to your Cloudera Manager server. Once a parcel has been downloaded to the server, it will be available for distribution on all clusters managed by the server. A downloaded parcel will appear in the cluster-specfic section for every cluster managed by this Cloudera Manager server.
The section at the top of the page labeled Downloadable shows you the parcels that are available to download from remote repositories (either from the standard Cloudera repositories or custom repositories).
Distributing a parcel copies the parcel to the member hosts of a cluster and unpacks it. Note that distributing a parcel does not actually upgrade the components running on your cluster; the current services continue to run unchanged. You can have multiple parcels distributed on your cluster.
Under each cluster you can see parcels that have been downloaded to a local repository on the Cloudera Manager server, and are available to be distributed to hosts in that cluster. You will also see the parcel(s) that have been Activated on that cluster.

Activating a parcel causes the Cloudera Manager to link to the new components, ready to run the new version upon the next restart. Activation does not automatically stop the current services or perform a restart — you have the option to restart the service(s) after activation, or you can allow the system administrator to determine the appropriate time to perform those operations.
Cloudera Manager detects when new parcels are available. The parcel
indicator in the Admin console navigation bar ( ) indicates when parcels are available for downloading or
distribution. If no parcels are available, or if all parcels have been activated, then this
indicator will be zero. Note that you can configure Cloudera Manager to download and
distribute parcels automatically, if desired.

Downloading a Parcel
- Click the parcel indicator in the top navigation bar. This takes
you to the Hosts page, Parcels tab. By default, any parcels available for download are shown in the Available Remotely section of the Parcels page. Parcels
available for download will display a Download
button.If the parcel you want is not shown here — for example, you want to upgrade to version of CDH that is not the most current version — you can make additional remote parcel repositories available through the Administration parcel settings page.
- Click Edit Settings near the top right of the Parcels page. From there, you can add URLs for remote parcel repositories that will then appear on the Parcels page as available for download. See Parcel Configuration Settings below for more information.
You can also configure the location of the local parcel repository, and other settings, through the Administration Settings page in the Parcels section. You can also access these properties from the Administration tab, select Settings, then Parcels.
- Click Download to initiate the download of the parcel from the remote parcel repository to your local repository.
When the parcel has been downloaded, the button changes to say Distribute.

The parcel download is done at the Cloudera Manager server, so with multiple clusters, the downloaded parcels are shown as available to all clusters managed by the Cloudera Manager server. However, distribution (to a specific cluster's member nodes) must be selected on a cluster-by-cluster basis.
Distributing a Parcel
Parcels that have been downloaded can be distributed to the nodes in your cluster, available for activation.
- From the Parcels tab, click the Distribute button for the parcel you want to distribute. This starts the distribution process to the nodes in the cluster.
Distribution does not require Internet access; rather the Cloudera Manager agent on each cluster member downloads the parcel from the local parcel repository hosted on the Cloudera Manager server.
If you have a large number of nodes to which the parcels should be distributed, you can control how many concurrent uploads Cloudera Manager will perform. You can configure this setting on the Administration page, Properties tab under the Parcels section.
You can delete a parcel that is ready to be distributed; click the triangle at the right end of the Distribute button to access the Delete command. This will delete the downloaded parcel from the local parcel repository.
Note that distributing parcels to the nodes in the cluster does not affect the current running services.
Activating a Parcel
Parcels that have been distributed to the nodes in a cluster are ready to be activated.
- From the Parcels tab, click the Activate button for the parcel you want to activate. This will update Cloudera Manager to point to the new software, ready to be run the next time a service is restarted.
- A pop-up warns you that your currently running process will not be affected until you restart, and gives you the option to perform a restart. If you do not want to restart at this time, click Close.
If you elect not to restart services as part of the Activation process, you can instead go to the Services tab and restart your services at a later time. Until you restart services, the current software will continue to run. This allows you to restart your services at a time that is convenient based on your maintenance schedules or other considerations.
Activating a new parcel will also automatically Deactivate the previously active parcel (if any) for the product you've just upgraded.

Deactivating a Parcel
You can deactivate an active parcel; this will update Cloudera Manager to point to the previous software version, ready to be run the next time a service is restarted.
To use the previous version of the software, go to the Services tab and restart your services.

Parcel Configuration Settings
You can configure settings for parcels, on the Administration Settings page, under the Parcels section. You can also access these properties by clicking the link Edit Settings button near the top of the
page.The Local Parcel Repository Path defines the path on the Cloudera Manager server host where downloaded parcels are stored.
The Remote Parcel Repository URLs is a list of remote repositories that Cloudera Manager should check for parcels. Initially this points to the default repository at archive.cloudera.com (http://archive.cloudera.com/cdh4/parcels/latest/) but you can added your own repository locations to the list.
- In the Remote Parcel Repository URLs list, click a plus to open an additional row.
- Enter the path to the repository you want.
You can use this mechanism to add Cloudera repositories that are not listed by default, such as older versions of CDH, or the Sentry parcel for CDH4.3. You can also use this to add your own custom repositories.
You can find the locations of the previous CDH4 parcels at http://archive.cloudera.com/cdh4/parcels/. If you are using CDH4.3 and want to add Sentry, you can find the separate Sentry parcel at http://archive.cloudera.com/sentry/parcels/latest/). (Sentry is included with CDH4.4 or later parcels.)
- Save Changes to have your additions take effect.
You can also:
- Set the frequency with which Cloudera Manager will check for new parcels
- Configure a proxy to access to the remote repositories
- Configure whether downloads and distribution of parcels should occur automatically whenever new ones are detected.
- Control which products can be downloaded if automatic downloading is enabled.
If automatic downloading/distribution are not enabled (the default), you must go to the Parcels page to initiate these actions.
You can configure the bandwidth limits and the number of concurrent uploads, to tune the load that parcel distribution puts on your network. The defaults are up to 50 concurrent parcel uploads and 50 MiB/s aggregate bandwidth.
- The concurrent upload count (Maximum Parcel Uploads) doesn't matter, theoretically, if all nodes have the same speed ethernet. In general, 50 concurrent uploads is an acceptable setting in most cases. However, in a scenario where the server has more bandwidth (say 10Gbe while the normal nodes are using 1Gbe), then the count is important to maximize bandwidth, and would need to be at least the difference in speeds (10x in this case).
- The bandwidth limit (Parcel Distribution Rate Limit) should be your ethernet speed (in MiB/seconds) divided by approximately 16. You can use a higher limit if you have QoS set up to prevent starving other services, or if you are willing accept a higher risk of higher bandwidth load.
For individual hosts (or all hosts) you can "blacklist" selected parcels; this will prevent those parcels from being distributed to or activated upon those hosts.
To blacklist a parcel:
- Go to the Configuration tab for a host (or for All Hosts) and click Edit.
- Under the parcels category, enter the parcel(s) you want to blacklist. Enter the name as it appears on the Parcels page — for example, CDH 4.3.0-1.csh4.3.0.p0.22 — and click Save.
If a parcel you blacklist has already been distributed to the host, it will be removed from that host. If it is already running on the host, it will continue to run until the next restart, when it will not be restarted.
Parcel File Locations
The default location for the local parcel repository is /opt/cloudera/parcel-repo on the Cloudera Manager server host. This is configurable as described above.
The default path for the distributed parcels on the member nodes is /opt/cloudera/parcels. This is configurable in the config.ini of the Cloudera Manager agent on each node.
Managing the Software Running on Your Cluster
Through the Parcels interface in Cloudera Manager, you can determine what versions of CDH and other packages are running across your cluster.
You can access the Parcels page by clicking the parcel indicator in the
Admin console navigation bar ( ) or by clicking the Hosts tab, then the Parcels tab.
The parcels page is divided into several sections. The top section, labeled Downloadable, shows you all the parcels that are available for download from the Cloudera parcel repositories, or from any other custom repositories you have configured. (See the section below, Parcel Configuration Settings for information on configuring additional repository locations.)
Below the Downloadable section, each cluster managed by this Cloudera Manager server has a section that shows the parcels that have been downloaded, distributed, or activated on that cluster.
Note that when you download a parcel, it appears under every cluster, if you are managing more than one. However, this just indicates that the parcel is available for distribution on those clusters — in fact there is only one copy of the downloaded parcel, residing on the Cloudera Manager server. Only after you distribute the parcel to a cluster will copies of it be placed on the hosts in that cluster.
Viewing Parcel Usage
The Parcel Usage page, available as of Cloudera Manager 4.7, shows you which parcels are in current use in your clusters. This is particularly useful in a large deployment where it may be difficult to keep track of what versions are installed across the cluster, especially if some nodes were not available when you performed an installation or upgrade, or were added later.
Note that this page only shows the usage of parcels, not components that were installed as packages. If you select a cluster running packages (for example, a CH3 cluster) the cluster is not displayed, and instead you will see a message indicating the cluster is not running parcels. If you have individual hosts running components installed as packages, they will appear as "empty."

You can view parcel usage by cluster, or by product (CDH, SOLR, and IMPALA).
You can also view just the hosts running only the active parcels, or just hosts running older parcels (not the currently active parcels) or both.
The "host map" at the right shows each host in the cluster with the status of the parcels on that host. If the host is actually running the processes from the currently activated parcels, the host is indicated in blue. A black square indicates that a parcel has been activated, but that all the running processes are from an earlier version of the software. This can happen, for example, if you have not restarted a service or role after activating a new parcel.
Move the cursor
over the icon to see the rack to which the hosts are
assigned. Hosts on different racks are displayed in separate rows.
A host that is running multiple product versions is shown with the
four-square icon ( ).
To view the exact versions of the software running on a given host, you can click on the square representing the host. This pops up a display showing the versions installed on that node.

For CDH4.4, Impala 1.1.1, and Solr 0.9.3 or later, it the list of roles running on the selected host that are part of the listed parcel. Clicking a role takes you to the Cloudera Manager page for that role. It also shows whether the parcel is Active or not.
If a node is running a mix
of software versions, the square representing the host is shown by a four-square icon
. When you move the cursor over that host, both the active and
inactive components are shown. For example, in the image below the older CDH parcel has
been deactivated but only the HDFS service has been restarted.

<< Previous: Performing a Rolling Upgrade on your Cluster | Next: Upgrading to the Latest Version of CDH4 in a Cloudera Manager Deployment >> |