Configuring Federated Nameservices
Cloudera Manager supports the configuration of multiple Nameservices managing separate HDFS namespaces, all of which share the storage available on the set of DataNodes. These Nameservices are federated, meaning each Nameservice is independent and does not require coordination with other Nameservices. See HDFS Federation for more information.
It is simplest to add a second Nameservice if High Availability is already enabled. The process of enabling High Availability creates a Nameservice as part of the enable High Availability workflow.

Converting a non-Federated HDFS Service to a Federated HDFS Service
You must have one Nameservice in place before you can add a second (or additional) Nameservices. Follow the instructions below to convert your current NameNode/SecondaryNameNode setup to a Federated Setup with a single Nameservice.
- Click the Services tab and select your HDFS service.
- Select .
- Search for "nameservice". This will show you the Nameservice properties for your NameNode and SecondaryNameNode.
- In the NameNode Nameservice field, type a name for the Nameservice. Note that this name must not include the underscore character.
- In the Mountpoints field, change the mount point from "/" to a list of mount points that are in the namespace that this Nameservice will manage. (You can enter this as a comma-separated list — for example, "/hbase, /tmp, /user" or by clicking the plus icon to add each mount point in its own field.) You can determine the list of mount points by running the command hadoop fs -ls / from the CLI on the NameNode host.
- In the SecondaryNameNode Nameservice field, type the name of the Nameservice. This must be the same as you provided for the NameNode Nameservice property.
- Save your changes.
- Return to the HDFS Service page, and click the Instances tab. You should now see the Federation and High Availability section with your Nameservice listed.
- You can use the Edit command under the Actions menu to edit the list of mount points for this Nameservice. In the Mountpoints field, change the mount point from "/" to a list of mount points that are in the namespace that this Nameservice will manage.
Adding a Nameservice
The instructions below for adding a Nameservice assume that a Nameservice is already set up. The first Nameservice cam be set up either by converting a simple HDFS service as described above (see Converting a non-Federated HDFS Service to a Federated HDFS Service or by enabling High Availability.
- Click the Services tab and select your CDH4 HDFS service.
- Click the Instances tab. At the top of this page you should see the Federation and High Availability section.
Note
: If this section does not appear, it means you do not have any Nameservices configured. You must have one Nameservice already configured in order to add a second. You can either enable High Availability, which will create a Nameservice, or you can convert your existing HDFS service. See Converting a non-Federated HDFS Service to a Federated HDFS Service above for instructions. - Click the Add Nameservice button.
- Enter a name for your new Nameservice. This name must be unique.
- Enter at least one mount point for this Nameservice. This defines the portion of HDFS that will be managed under the new Nameservice. (Click the + to the right of the Mount Point field to add a new mount point). You cannot use "/" as a mount point; you must specify HDFS directories by name.
- Note that the mount points must be unique for this Nameservice; you cannot specify any of the same mount points you have used for other Nameservices.
- You can specify mount points that do not yet exist, and create the corresponding directories in a later step in this procedure.
- If you want to use a mount point previously associated with another Nameservice you must first remove that mount point from that service. You can do this using the Edit command from the Actions menu for that Nameservice, and later add the mount point to the new Nameservice.
- After you have brought up the new Nameservice, you will need to create, in the new namespace, the directories that correspond with the mount points you specified.
- If a mount point corresponds to a directory that formerly was under a different Nameservice, you will also need to move any contents of that directory, if appropriate. Instructions for doing this are at the end of this procedure — you must have created the Nameservice before you can create directories in its namespace.
- If an HBase service is set to depend on the federated HDFS service, make sure to edit the mount points of the existing Nameservice to reference:
- the HBase service's HBase root directory (default value /hbase)
- the MapReduce System Directory (default value /tmp/mapred/system) and
- the MapReduce JobTracker Staging Root Directory (default value /user).
- If you want to configure High Availability for this Nameservice, leave the Highly Available checkbox checked.
- Click Continue.
- Select the hosts on which the new NameNode and Secondary NameNodes will be created. (Note that these must be hosts that are not already running other NameNode or SecondaryNameNode instances, and their /dfs/nn and /dfs/snn directories should be empty if they exist. Click Continue.
- Enter or confirm the directory property values (these will differ depending on whether you are enabling High Availability for this Nameservice, or not).
- Uncheck the Start Dependent Services checkbox if you need to create directories or move data onto the new Nameservice. Leave this checked if you want the workflow to restart services and redeploy the client configurations as the last steps in the workflow.
- Click Continue. If the process finished successfully, click Finish. You should now see your new Nameservice in the Federation and High Availability section in the Instances tab of the HDFS service.
- You must now create the directories you want under the new Nameservice. You need to do this in the CLI.
- To create a directory in the new namespace, use the command hadoop fs -mkdir /nameservices/ <nameservice name> / <directory> where <nameservice name> is the new nameservice you just created, and <directory> is the directory that corresponds to a mount point you specified.
- If you need to move data from one Nameservice to another, use distcp or manual export/import. dfs -cp and dfs -mv will not work.
- Verify that the directories and data are where you expect them to be.
- Restart the dependent services.

Nameservice and Quorum-based Storage
With Quorum-based Storage, JournalNodes are shared across Nameservices. So, if JournalNodes are present in an HDFS service, all Nameservices will have Quorum-based Storage enabled. To override this:
- The dfs.namenode.shared.edits.dir configuration of the two NameNodes of a High Availability Nameservice should be configured to include the NFS mount, or
- The dfs.namenode.edits.dir configuration of the one NameNode of a non-High Availability Nameservice should be configured to include the value of the dfs.namenode.name.dirs setting.

When this happens, configure the new NameNodes (via the safety valve) with a QuorumJournal URL that has a different journal name from the original Nameservice, and then manually perform the rest of the steps in the "Add Nameservice" workflow.
<< | ||
Terms and Conditions Privacy Policy |