Upgrading to CDH4.1.3 or Later Using the Upgrade Wizard

If you want to upgrade to CDH4.1.3 or later, you can do the upgrade using parcels from within the Cloudera Manager Admin Console.

To upgrade Impala from the beta version to version 1.0 or later:

  Note: If you are not running a beta version of Impala, you should skip this step.

You must point the Impala parcel repo beta URL to the released Impala parcel.

  1. From the Administration tab, select Properties.
  2. Go to the Parcels category.
  3. Under the Remote Parcel Repository URLs property, find the entry http://beta.cloudera.com/impala/parcels/ and replace it with http://archive.cloudera.com/impala/parcels/.
  4. Save your change.

Now you can proceed to upgrade your installation.

Step 1. Download, Distribute, and Activate the CDH4 (and Impala) Parcels.

  1. In the Cloudera Manager Admin Console, click the Parcels indicator in the top navigation bar ( or ) to go to the Parcels page.
  2. In the parcels page, click Download for the version(s) you want to download. If you want to run both CDH and Cloudera Impala, you should download both the CDH and Impala parcels.
    If the parcel you want is not shown here — for example, you want to upgrade to version of CDH that is not the most current version — you can make additional parcel repos available through the Administration parcel settings page.
    • From the Administration tab, select Settings, then Parcels. From there, you can add URLs for additional parcels that will then appear on the Parcels page as available for download.
    You can find the locations of the previous CDH4 parcels at http://archive.cloudera.com/cdh4/parcels/. If you are upgrading to CDH4.3 and want to use Sentry, you can find the separate Sentry parcel at http://archive.cloudera.com/sentry/parcels/latest/). Sentry is included with CDH4.4 or later parcels.

    If your Cloudera Manager server does not have Internet access, you can obtain the required parcel file(s) and put them into the local repository. See Creating a Local Parcel Repository for more details.

  3. When the download has completed, click Distribute for the version you downloaded.
  4. When the parcel has been distributed and unpacked, the button will change to say Activate.
  5. Click Activate. This will display a pop-up that will offer to restart your services. DO NOT RESTART services at this point – click Close to remove the pop-up.
      Important:

    If you are upgrading between major Hive versions (i.e from CDH4.0 to 4.1 or 4.2, or from CDH4.1 to 4.2) DO NOT restart the services – you must upgrade your Hive metastore before you restart Hive.

Step 2. Upgrade the Hive Metastore

If you are upgrading from CDH4.2 to CDH4.3 or later, you do not need to perform this step. If you are upgrading from an earlier version of CDH to CDH4.2 or later, you DO need to do this.

  1. (Strongly recommended) Make a backup copy of your Hive metastore database.
  2. Run the metastore upgrade script. The script you run depends on whether you are upgrading to parcels or packages.
    • If you are upgrading to packages, the upgrade script is at /usr/lib/hive/scripts/metastore/upgrade/
    • If you are upgrading to parcels, then the upgrade script is located at /opt/cloudera/parcels/<parcel_name>/lib/hive/scripts/metastore/upgrade/<database>.

      <parcel_name> should be the name of the parcel to which you have upgraded.

      <database> is the type of database you are running (i.e. mysql, postgres, etc.)

      For example, if you are installing a CDH4.2.0 parcel using the default location for the local repository, and using the default database (PostgreSQL) the script will be at: /opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10-e16.parcel/lib/hive/scripts/metastore/upgrade/postgres
    • You must cd to the directory the scripts are in.
    • Execute the script in the appropriate DB command shell. Note that there are multiple scripts in each directory. You nust run the one that corresponds to the versions of Hive you are upgrading between. For example, if you are upgrading with MySQL from Hive 0.9 to 0.10, the command would be similar to:
      mysql -u hive1 -phive1 hive1 < upgrade-0.9.0-to-0.10.0.mysql.sql

      (with the appropriate substitutions for username, etc.).

      If your upgrade spans multiple versions of Hive (for example, upgrading from Hive 0.8 to Hive 0.10) you must run all the relevant scripts in the proper order.

      Important:

    You must know the password for the Hive metastore database; if you installed Cloudera Manager using the default (embedded PostgreSQL) database, the password was displayed on the Database Setup page during the Cloudera Manager installation wizard. If you do not know the password for your Hive metastore database, you can find it as follows:

    • cat /etc/cloudera-scm-server/db.properties This shows you Cloudera Manager's internal database credentials.
    • Run the following command:
      psql -p 7432 -U cm cm -c "select s.display_name as hive_service_name, s.name as  hive_internal_name, c.value as metastore_password from CONFIGS c, SERVICES s where attr='hive_metastore_database_password' and  c.service_id = s.service_id"
    • Use the password from com.cloudera.cmf.db.password. This will output the passwords for the hive service metastore as follows:
       hive_service_name | hive_internal_name | metastore_password
      -------------------+--------------------+--------------------
       hive1             | hive1              | lF3Cv2zsvI
      (1 row)
  3. If you have multiple instances of Hive, run the upgrade script(s) on each metastore database.

Step 3. (If Upgrading to CDH 4.2) Upgrade the Oozie Sharelib

  1. In the Cloudera Manager Admin Console, select Oozie from the Services tab.
  2. From the Actions button, choose Stop.
  3. When the service has stopped, from the Actions button choose Install Oozie Sharelib. The commands to perform this function are run.

Step 4. Restart the Services

  1. In the Cloudera Manager Admin Console, select All Services from the Services tab.
  2. Click the top Actions button that corresponds to the cluster and choose Restart. The Command Details window shows the progress of starting services.

Step 5. Deploy the new client configuration files

  1. From the top Actions button that corresponds to the cluster and choose Deploy Client Configuration....
  2. Click the Deploy Client Configuration button in the confirmation pop-up that appears.

Step 6. Remove the previous CDH version packages.

If your previous installation of CDH 4 (4.0.x or 4.1.x) was done using packages, you must remove those packages and refresh the symlinks so that clients will run the new software versions.

  1. If Hue is configured to use SQLite as its database, back up the desktop.db to a temporary location before deleting the old Hue Common package. The location of the database can be found in the Hue service Configuration tab under Service > Database > Hue's Database Directory.
      Important: Removing the Hue Common package will remove your Hue database; if you do not back it up you may lose all your Hue user account information.

    Make sure the new Hue service is running before you remove the old packages.

  2. To uninstall the CDH packages (not including Impala):

    On each host:

    Operating System Command

    RHEL

    $ sudo yum remove hadoop hue-common bigtop-jsvc bigtop-tomcat

    SLES

    $ sudo zypper remove hadoop hue-common bigtop-jsvc bigtop-tomcat

    Ubuntu or Debian

    $ sudo apt-get purge hadoop hue-common bigtop-jsvc bigtop-tomcat
  3. To uninstall the CDH packages (including Impala):

    On each host:

    Operating System Command

    RHEL

    $ sudo yum remove hadoop hue-common impala-shell 'bigtop-*'

    SLES

    $ sudo zypper remove hadoop hue-common impala-shell 'bigtop-*'

    Ubuntu or Debian

    $ sudo apt-get purge hadoop hue-common impala-shell 'bigtop-*'

Step 7. Update symlinks for the newly installed components.

Restart all the Cloudera Manager agents to force an update of the symlinks to point to the newly installed components.

To restart the Cloudera Manager agents:

On each host:

$ sudo service cloudera-scm-agent restart