Troubleshooting Installation and Upgrade Problems

Use the information in this section to troubleshoot installation problems. For information on known issues, see Known Issues and Work Arounds in Cloudera Manager 4.

Symptom

Problem

What to Do

"Failed to start server" reported by cloudera-manager-installer.bin. /var/log/cloudera-scm-server/cloudera-scm-server.log contains a message beginning Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver...

You may have SELinux enabled.

You can disable SELinux by running
 sudo setenforce 0 

on the Cloudera Manager Server host. To disable it permanently, edit /etc/selinux/config.

Installation interrupted and installer won't restart.

You need to do some manual cleanup.

See Uninstalling Cloudera Manager.

Cloudera Manager Server fails to start. Server is configured to use a MySQL database to store information about service configuration.

Tables may be configured with the ISAM engine.

Make sure that the InnoDB engine is configured, not the MyISAM engine. To check what engine your tables are using, run the following command from the MySQL shell: mysql> show table status;

Agents fail to connect to server. Error 113 ('No route to host') in cloudera-scm-agent.log.

You may have SELinux or iptables enabled.

Check /var/log/cloudera-scm-server/cloudera-scm-server.log on the Server system and /var/log/cloudera-scm-agent/cloudera-scm-agent.log on the Agent system(s). Disable SELinux and iptables.

Some cluster hosts do not appear when you click Find Hosts in install or update wizard.

You may have network connectivity problems.

  • Make sure all cluster hosts have SSH port 22 open.
  • Check other common causes of loss of connectivity such as firewalls and interference from SELinux.

"Access denied" in install or update wizard during database configuration for Activity Monitor, Report Manager, or Service Monitor.

Hostname mapping or permissions are incorrectly set up.

  • For hostname configuration, see information on Configuring Network Names in the CDH3 Deployment on a Cluster topic in CDH3 Installation Guide.
  • For permissions, make sure the values you enter into the wizard match those you used when you configured the databases. For more information, see Checking Database Hostnames.

Activity Monitor, Report Manager, or Service Monitor databases fail to start.

MySQL binlog format problem.

Set binlog_format=mixed in /etc/my.cnf. For more information, see this MySQL bug report. See also Installing and Configuring Databases.

You have upgraded the Cloudera Manager Server to 4.5, but now cannot start services.

You may have mismatched versions of the Cloudera Manager Server and Agents.

Make sure you have upgraded the Cloudera Manager Agents on all host machines to 4.5. (The previous version of the Agents will heartbeat with the new version of the Server, but you can't start HDFS and MapReduce with this combination.)

Cloudera services fail to start.

Java may not be installed or may be installed at a custom location.

See Using Custom Java Home Locations for more information on resolving this issue.

The Service Monitor, Activity Monitor, or Host Monitor display a status of BAD in the Cloudera Manager Admin Console. The log file contains the following message: ERROR 1436 (HY000): Thread stack overrun: 7808 bytes used of a 131072 byte stack, and 128000 bytes needed. Use 'mysqld -O thread_stack=#' to specify a bigger stack.

The MySQL thread stack is too small.

Update the thread_stack value in my.cnf to 256KB. The my.cnf file is normally located in /etc or /etc/mysql.

Restart the mysql service:
$ sudo service mysql restart

Restart the failed service using the Cloudera Manager Admin Console.

The Service Monitor or Activity Monitor agents fail to start. Logs contain the error read-committed isolation not safe for the statement binlog format.

The binlog_format is not set to mixed.

Modify the mysql.cnf file to include the entry for binlog format as specified in Installing and Configuring a MySQL Database.

Attempts to reinstall older versions of CDH or Cloudera Manager using Yum fails.

It is possible to install, uninstall, and reinstall CDH and Cloudera Manager. In certain cases, this does not complete as expected. If you install Cloudera Manager 4 and CDH 4, then uninstall Cloudera Manager and CDH, and then attempt to install CDH 3.7 and Cloudera Manager 3.7, incorrect cached information may result in the installation of an incompatible version of the Oracle JDK.

To resolve this issue, you must clear information in the yum cache. Clear cache information as follows:

Connect to the CDH host.

Execute either of the following commands: $ yum --enablerepo='*'clean all or $ rm -rf /var/cache/yum/cloudera*

After clearing cache information, proceed with installing CDH 3.7 and Cloudera Manager 3.7.