Cloudera Manager Requirements

Cloudera Manager interacts with a diversity of entities such as operating systems, databases, and browsers. Cloudera provides information about which major release version and minor release version is supported, where available. In some cases, such as some browsers, a minor version may not be provided. After installing each element, upgrade to the latest patch version and apply any other appropriate updates. Note that the available updates may be specific to the operating system on which it is installed.

For example, you might be using CentOS in your environment. You could choose 6 as the major version and 2 as the minor version. These choices would mean you would be using CentOS 6.2. After installing this operating system, you would then apply any and all relevant CentOS 6.2 upgrades and patches.

The following sections describe the OS, browser, database, and other requirements for Cloudera Manager:

For information on CDH4 requirements, see "Supported Operating Systems for CDH4" in the CDH4 Installation Guide.

For information on CDH3 requirements, see "Supported Operating Systems for CDH3" in the CDH3 Installation Guide. (Please note that CDH3 has reached End of Maintenance (EOM) as of June 20th, 2013. Cloudera will not support or provide patches for any CDH3 releases.)

Supported Operating Systems for Cloudera Manager

Cloudera Manager supports a range of operating systems including:

  • Red Hat-compatible systems
    • Red Hat Enterprise Linux 5.7 and CentOS 5.7, 64-bit
    • Red Hat Enterprise Linux 6.2 and 6.4, and CentOS 6.2 and 6.4, 64-bit
    • Oracle Enterprise Linux 5.6 and 6.4 with Unbreakable Enterprise Kernel, 64-bit
  • SLES systems
    • SUSE Linux Enterprise Server 11, 64-bit. Service Pack 1 or later is required. Also, the SUSE Linux Enterprise Software Development Kit 11 SP1 is required on cluster hosts running the Cloudera Manager Agents (not required on the Cloudera Manager Server host); you can download the SDK here.
  • Debian systems
    • Debian 6.0 (Squeeze), 64-bit
  • Ubuntu systems
    • Ubuntu 10.04 (Lucid Lynx), 64-bit
    • Ubuntu 12.04 (Precise Pangolin), 64-bit
  Note: For Hadoop to work properly, using the same version of the same operating system on all cluster hosts is strongly recommended.

Supported Browsers for Cloudera Manager Admin Console

The Cloudera Manager Admin Console, which you use to configure, manage, and monitor CDH, supports the following browsers:

  • Firefox 11 or later
  • Google Chrome
  • Internet Explorer 9
  • Safari 5 or later

Supported Databases for Cloudera Manager

Cloudera Manager requires several databases. The Cloudera Manager server stores information about configured services, role assignments, configuration history, commands, users, and running processes in a database of its own. The Activity Monitor, Service Monitor, Report Manager, and Host Monitor also each use a database to store information.

The database you choose to use must be configured to support UTF8 character set encoding. The embedded PostgreSQL database that is installed using Path A automatically provides UTF8 encoding. If you install a custom database, you may need to enable UTF8 encoding. The commands for enabling UTF8 encoding are described in each database's section under Installing and Configuring Databases.

After installing a database, upgrade to the latest patch version and apply any other appropriate updates. Note that the available updates may be specific to the operating system on which it is installed.

Cloudera Manager and its supporting services can use the following database systems and releases:

  • MySQL:
    • 5.0
    • 5.1
    • 5.5
  • Oracle
    • 10g Release 2
    • 11g Release 2
  • PostgreSQL
    • 8.1
    • 8.3
    • 8.4
    • 9.1

Other Requirements

Cloudera Manager supports a variety of services and depends on resources being available.

CDH Version Support

  • Cloudera Manager 4.6 supports CDH3 Update 1 (cdh3u1) or later and CDH4.0 or later. CDH3 Update 2 or later is strongly recommended.
  Warning: The latest released versions of CDH3 and CDH4 are strongly recommended.
  • If you want to use Cloudera Manager to manage Oozie, CDH3 Update 2 or later is required.
  • Cloudera Manager uses Python. Python is part of the default installation for all operating systems that Cloudera Manager supports, so there is no need to complete any installation tasks to make Python available. Cloudera Manager is tested with the default installation. Modifying the Python installation available on systems on which you install Cloudera Manager is not supported.
  • Cloudera Manager supports Impala 1.2.1 or later with CDH4.1 or later.

Resources

Cloudera Manager requires sufficient resources of the following types:

Disk Space:

On the Cloudera Manager server:
  • 5 GB on the partition hosting /var.
  • 500 MB on the partition hosting /usr.
  • For parcels, the space required will depend on the number of parcels you download to the Cloudera Manager server, and distribute to agent nodes. You can download multiple parcels of the same product, of different versions and builds.

    Note that even if you are managing multiple clusters, there will be only one parcel of any given product/version/build/distro downloaded on the Cloudera Manager server — not one per cluster.

    In the local parcel repo on the Cloudera Manager server (in /opt/cloudera/parcel-repo by default) the approximate sizes of the various parcels (as of Cloudera Manager 4.6) are as follows:

    • CDH4.3 — slightly over 700MB per parcel.
    • Impala — approximately 200MB per parcel.
    • Cloudera Search (Solr) — approximately 400MB per parcel.

On Agent Nodes:

On the agent nodes, when a parcel has been distributed to a cluster, each unpacked parcel will require about three times the space of the downloaded parcel on the Cloudera Manager server. By default unpacked parcels are located in /opt/cloudera/ on each cluster member.

RAM:

4GB is appropriate for most cases, and is required when using Oracle databases. 2GB may be sufficient for non-Oracle deployments involving fewer than 100 hosts.

JDK

The Cloudera Manager Installation program installs Oracle JDK 1.6 (1.6.0_31) if a JDK is not already installed. However, Cloudera Manager 4.5.1 and later also support JDK 1.7.

Networking and Security

  • Cluster hosts must have a working network name resolution system. Properly configuring DNS and reverse DNS meets this requirement. If you use /etc/hosts instead of DNS, all hosts files must contain consistent information about host names and addresses across all nodes. For example, /etc/hosts might contain something of the form:
    127.0.0.1 localhost.localdomain localhost
    192.168.1.1 cluster-01.domain cluster-01
    192.168.1.2 cluster-02.domain cluster-02
    192.168.1.3 cluster-03.domain cluster-03 
  • In most cases, the Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard. This does not apply if you install Cloudera Manager using Installation Path B - Installation Using Your Own Method . For further information, see Permission Requirements.
      Note: You must log in using a root account or an account that has password-less sudo permission. For authentication during the installation and upgrade procedures, you will need to either enter the password or upload a public and private key pair for the root or sudo user account. If you want to use a public and private key pair, the public key must be installed on the cluster hosts before you use Cloudera Manager. Authentication is not supported for accounts that have password-protected sudo permission. Cloudera Manager uses SSH only during the initial install or upgrade. Once your cluster is set up, you can safely disable root SSH access or change the root password. Cloudera Manager does not save SSH credentials and all credential information is discarded once the installation is complete.
  • The Cloudera Manager Agent runs as root so that it can make sure the required directories are created and that processes and files are owned by the appropriate user (for example, the hdfs user and mapred user).
  • No blocking by Security-Enhanced Linux (SELinux).
  • Disable Ipv6 on all machines.
  • No blocking by iptables or firewalls; make sure port 7180 is open because it is the port used to access Cloudera Manager after installation. Cloudera Manager communicates using specific ports, which must be open. For additional information, see Configuring Ports for Cloudera Manager.
  • For RedHat/CentOS operating systems, make sure the/etc/sysconfig/network file on each system contains the hostname you have just set (or verified) for that system. (This does not apply to Debian/Ubuntu or SLES).
  • Cloudera Manager and CDH use several user accounts and groups to complete their tasks. The set of user accounts and groups varies according to which components you choose to install. Do not delete these accounts or groups and do not modify their permissions and rights. Ensure no existing systems obstruct the functioning of these accounts and groups. For example, if you have scripts that delete user accounts not in a white-list, add these accounts to the list of permitted accounts. Cloudera Manager and CDH create and use the following accounts and groups:

    Account

    Type

    Product

    Comment

    cloudera-scm

    User and group

    Cloudera Manager

     

    mapred

    User and group

    CDH3 and CDH4

    MapReduce

    hdfs

    User and group

    CDH3 and CDH4

    Distributed file system

    zookeeper

    User and group

    CDH3 and CDH4

    Distributed system coordination service

    yarn

    User and group

    CDH4

    MapReduce2.0 or MRv2

    httpfs

    User and group

    CDH4

    HTTP gateway to HDFS

    hbase

    User and group

    CDH3 and CDH4

    Hadoop database

    hive

    User and group

    CDH3 and CDH4

    Hadoop data warehouse

    hue

    User and group

    CDH3 and CDH4

    Web interface to hadoop

    oozie

    User and group

    CDH3 and CDH4

    Workflow coordination system

    flume

    User and group

    CDH3 and CDH4

    Log collection system

    hadoop

    Group

    CDH3 and CDH4

     

    impala

    User and group

    CDH4.1

    Interactive query tool

    sqoop

    User and group

    CDH3 and CDH4

    Sqoop

    sqoop2

    User

    CDH4.2 and later

    Sqoop2 server

    solr

    User and group

    CDH4.3 and later

    Solr

For additional port information, see Configuring Ports for Cloudera Manager.