Cloudera Enterprise 5.15.x | Other versions

Upgrading Cloudera Manager Using Packages

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

This page describes how to upgrade Cloudera Manager from any 5.x version to a higher 5.x version, including minor and maintenance releases, using packages. When you upgrade Cloudera Manager using packages, you run operating system package commands from the command line, and then complete the upgrade using Cloudera Manager. The Cloudera Manager upgrade process does the following:
  1. Upgrades the database schema to reflect the current version.
  2. Upgrades the Cloudera Manager Server and all supporting services.
  3. Upgrades the Cloudera Manager agents on all hosts.
  4. Redeploys client configurations to ensure that client services have the most current configuration.

Upgrading Cloudera Manager does not upgrade the version of CDH used in clusters managed by Cloudera Manager. See Upgrading CDH Using Cloudera Manager for more details.

Fill in the following form to create a customized set of instructions for your Cloudera Manager upgrade.

Loading Filters ...

  Important:
  • Collect information about your environment.
    1. Log in to the Cloudera Manager server host using ssh. 
      ssh my_cloudera_manager_server_host
    2. Run the following commands:

      For operating system information:

      lsb_release -a
      For database information: 
      cat /etc/cloudera-scm-server/db.properties
      ...
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost:port
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=SOME_PASSWORD
    3. Log in to the Cloudera Manager Admin console and find the following:
      1. The version of Cloudera Manager used in your cluster. Go to Support > About.
      2. The version of the JDK deployed in the cluster. Go to Support > About.
  • Any content not applicable to your environment will not be shown.
  • Rollback of your software and data is not supported.
  • Embedded database is not supported in production environment because downtime is required.
  • If you are upgrading from Cloudera Manager 5.5.0 or lower to Cloudera Manager 5.5.1 or higher, A cluster re-start is required which will require downtime for the cluster.
  • For upgrades other than from Cloudera Manager 5.5.0 or lower to Cloudera Manager 5.5.1 or higher. Downtime for the cluster is not required to complete the Cloudera Manager upgrade.
  • Downtime for the cluster is not required to complete the Cloudera Manager upgrade; for services that contribute to Cloudera Navigator audits, lineage, or search, restarting the service is recommended but can be scheduled independently of the upgrade.
  • Cloudera Manager 6.x supports upgrading from Cloudera Manager 5.7 or higher.
  • Cloudera Manager 6.x must use JDK 1.8.
  • Cloudera Manager 6.x only supports Redhat/Centos 6 or 7.
  • Cloudera Manager 6.x only supports SLES 12.
  • Cloudera Manager 6.x only supports Ubuntu 16.04 (Xenial).
  • Cloudera Manager 6.x only supports Debian 8 (jessie).

Continue reading:

  1. Before You Begin
  2. Backup Databases
  3. Backup Cloudera Navigator Data
  4. Establish Access to the Software
  5. Upgrade the Cloudera Manager Server
  6. Upgrade the Cloudera Manager Agents
  7. Perform Post Upgrade Steps
  8. Upgrade Cloudera Navigator Encryption Components
  9. (Optional) Upgrade CDH
  Important: If you encounter problems during an upgrade, see the following:

Before You Begin

  1. You must have SSH access to the Cloudera Manager server hosts and be able to login using the root account or an account that has password-less sudo permission for all hosts.
  2. Review the Requirements and Supported Versions.
  3. Review the Release Notes:
  4. Review the Cloudera Security Bulletins.
  5. You should have either Oracle JDK 7 or JDK 8 installed on all the hosts. More Information. If you plan to use Spark 2.2, you must install Oracle JDK 8 and configure the Java Home Directory property for all the hosts in the cluster. See Java Development Kit Installation.
  6. If you have previously installed Kafka 1.2, and are upgrading from Cloudera Manager 5.4 or lower, remove the Kafka CSD:
    1. Determine the location of the CSD directory:
      1. Select Administration > Settings.
      2. Click the Custom Service Descriptors category.
      3. Retrieve the directory from the Local Descriptor Repository Path property.
    2. Delete the Kafka CSD from the directory.
  7. Check the Package Dependencies. A Cloudera Manager upgrade can introduce new package dependencies. Your organization may have restrictions or require prior approval for installation of new packages.

Backup Databases

  1. Stop the Cloudera Management Service.
    1. Login to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Stop.
  2. Log in to the Cloudera Manager server host using ssh. 
    ssh my_cloudera_manager_server_host
  3. If you have not already done so, collect database information by running the following command: 
    cat /etc/cloudera-scm-server/db.properties
    ...
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost:port
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=SOME_PASSWORD
  4. Back up the Cloudera Manager server database – Run the following command. (The command displayed below depends on the database you selected in the form at the top of this page. Replace placeholders with the actual values returned from the db.properties file):
    MySQL
    mysqldump --databases scm --host=database_hostname --port=datbase_port -u scm -p > $HOME/scm-backup-`date +%F`.sql
      Note: If the db.properties file does not contain a port number, omit the port number parameter from the above command. .
    PostgreSQL/Embedded
    pg_dump -h database_hostname -U scm -W -p database_port scm > $HOME/scm-backup-`date +%F`.sql
    Oracle
    Work with your database administrator to ensure databases are properly backed up.

    For more information about backing up databases, see Backing Up Databases.

  5. Backup All other databases - For database information, go to Clusters > Cloudera Management Service > Configuration, select the Database category. You may need to contact your database administrator to obtain the passwords.

    These databases can include Reports Manager, Navigator Audit Server, Navigator Metadata Server, and Activity Monitor (Only used for MapReduce 1 monitoring).

    Run the following commands to backup the databases. (The command displayed below depends on the database you selected in the form at the top of this page. Replace placeholders with the actual values.):

    MySQL
    mysqldump --databases database_name --host=database_hostname --port=database_port -u database_username -p > $HOME/database_name-backup-`date +%F`.sql
    PostgreSQL/Embedded
    pg_dump -h database_hostname -U database_username -W -p database_port database_name > $HOME/database_name-backup-`date +%F`.sql
    Oracle
    Work with your database administrator to ensure databases are properly backed up.
  6. Start the Cloudera Management Service if you are NOT performing the Cloudera Manager server upgrade immediately.
    1. Login to the Cloudera Manager Admin Console
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Start.

Backup Cloudera Navigator Data

  Important: Upgrading from Cloudera Manager 5.9 (Navigator 2.8) and earlier can take a significant amount of time, depending on the size of the Navigator Metadata storage directory. When the Cloudera Manager upgrade process completes and Cloudera Navigator services restart, the Solr indexing upgrade automatically begins. No other actions can be performed until Solr indexing completes (a progress message displays during this process). It can take as long as two days to upgrade a storage directory with 60 GB. To help mitigate this extended upgrade step, make sure to clear out all unnecessary metadata using purge, check the size of the storage directory, and consider rerunning purge with tighter conditions to further reduce the size of the storage directory.
  1. Make sure a purge task has run recently to clear stale and deleted entities.
    • You can see when the last purge tasks was run in the Navigator console, Administration > Purge Settings.
    • If a purge hasn't run recently, run it by editing the Purge schedule on the same page.
    • Set the purge process options to clear out as much of the backlog of data as you can tolerate for your upgraded system. See Managing Metadata Storage with Purge.
  2. Identify the location of the Storage Directory.
    • Go to Clusters > Cloudera Management Service > Instances.
    • Click the Configuration tab.
    • Select Scope > Navigator Metadata Server.
    • The Navigator Metadata Server Storage Dir property stores the location of the directory.
  3. Stop the Navigator Metadata Server.
    • Go to Clusters > Cloudera Management Service > Instances.
    • Select Navigator Metadata Server.
    • Click Actions for Selected > Stop.
  4. Backup the Cloudera Navigator Solr storage directory.
  5. Start the Navigator Metadata Server.
    • Go to Clusters > Cloudera Management Service > Instances.
    • Select Navigator Metadata Server.
    • Click Actions for Selected > Start.
  6. Ensure that Navigator Metadata Server Java heap is large enough to complete the upgrade. You can estimate the amount of heap needed from the number of elements and relations stored in the Solr storage directory.
    • Go to Clusters > Cloudera Management Service > Instances.
    • Select Navigator Metadata Server.
    • Select Log Files > Role Log File.
    • Search the log file for "solr core nav_elements" and note the number of element documents.
    • Search the log file for "solr core nav_relations" and note the number of relation documents.
    • Multiply the total number of documents by 200 bytes per document and add to it a baseline of 2 GB: 
      ((num_nav_elements + num_nav_relations) * 200 bytes) + 2 GB
      For example, if you had 68813088 elements and 78813930 relations, the recommended Java heap size is ~30 GB: 
      ((68813088 + 78813930) * 200) + 2 GB = 13762617600 bytes = ~29.5 GB + 2 GB = ~ 31.5 GB
    • Set the heap value in the Java Heap Size of Navigator Metadata Server in Bytes property in Clusters Cloudera Management ServiceConfiguration.
  7. If you are using an Oracle database for audit, in SQL*Plus, ensure that the following additional privileges are set: 
      GRANT EXECUTE ON sys.dbms_crypto TO nav;
  GRANT CREATE VIEW TO nav;
    where nav is the user of the Navigator Audit Server database.

Establish Access to the Software

Cloudera Manager needs access to a package repository that contains the updated software packages.

If the Cloudera Manager server host has internet access, you can use the publicly available repositories from https://archive.cloudera.com. Fill in the form at the top of this page to generate the repository file for your system and the version of Cloudera Manager you want to upgrade to. The file created allows you to upgrade to the latest maintenance version for that minor version.

If the Cloudera Manager server host does not have internet access, configure a local package repository hosted on your network, and enter the repository URL below to generate the repository file. For example: http://MyHostname:port/cm

Package Repository URL:

  1. Log in to the Cloudera Manager server host. 
    ssh my_cloudera_manager_server_host
  2. Redhat / CentOS

    Create a file named cloudera_manager.repo with the following content:

    [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1
    SLES

    Create a file named cloudera_manager.repo with the following content:

    [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1
    Debian / Ubuntu

    Create a file named cloudera_manager.list with the following content:

    # Packages for Cloudera Manager
deb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
deb-src https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib

    The repository file, as created, specifies an upgrade to the most recent maintenance release of the specified minor release. If you would like to upgrade to an specific maintenance version, for example 5.15.1, replace 5.15 with 5.15.1 in the generated repository file shown above.

  3. Backup the existing repository directory.
    Redhat / CentOS
    sudo cp -rf /etc/yum.repos.d $HOME/yum.repos.d-`date +%F`
    SLES
    sudo cp -rf /etc/zypp/repos.d $HOME/repos.d-`date +%F`
    Debian / Ubuntu
    sudo cp -rf /etc/apt/sources.list.d $HOME/sources.list.d-`date +%F`
  4. Remove any older files in the existing repository directory:
    Redhat / CentOS
    sudo rm /etc/yum.repos.d/cloudera*manager.repo*
    SLES
    sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
    Debian / Ubuntu
    sudo rm /etc/apt/sources.list.d/cloudera*.list*
  5. Copy the repository file created above to the repository directory:
    Redhat / CentOS
    sudo cp cloudera_manager.repo /etc/yum.repos.d/
    SLES
    sudo cp cloudera_manager.repo /etc/zypp/repos.d/
    Debian / Ubuntu
    sudo cp cloudera_manager.list /etc/apt/sources.list.d/

Upgrade the Cloudera Manager Server

  1. Log in to the Cloudera Manager server host as the root user. 
    ssh root@my_cloudera_manager_server_host
  2. If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These can include:
    • Hive service and all services such as Impala and Hue that use the Hive metastore
    • Oozie
    • Sentry
    • Sqoop
  3. Back up the server and the agent directories: 
    sudo cp -r /etc/cloudera-scm-server $HOME/cloudera-scm-server-`date +%F`
sudo cp -r /etc/cloudera-scm-agent $HOME/cloudera-scm-agent-`date +%F`
  4. Stop the Cloudera Management Service.
    1. Login to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Stop.
      Important:

    Not stopping the Cloudera Management Service at this point might cause management roles to crash or the Cloudera Manager server may fail to restart.

    If there are replication jobs, snapshot jobs, or other commands running when you stop Cloudera Manager Server, Cloudera Manager Server may fail to start after the upgrade.

  5. Ensure that you have disabled any scheduled replication or snapshot jobs and wait for any running commands from the Cloudera Manager admin console to complete before proceeding with the upgrade.
  6. Stop the Cloudera Manager Server.
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-server
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server stop
  7. Stop the Cloudera Manager Embedded Database if you are using the embedded PostgreSQL database.
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-server-db
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server-db stop

    If you are not using the embedded PostgreSQL database and you attempt to stop it, you may see a message indicating that the service cannot be found. If you see a message that the shutdown failed, the embedded database is still running, probably because services are connected to the Hive metastore. If the database shutdown fails due to connected services, issue the following command:

    RHEL-compatible 7 and higher, Ubuntu 16.04 
    sudo service cloudera-scm-server-db next_stop_fast
    sudo service cloudera-scm-server-db stop
    All other Linux distributions 
    sudo service cloudera-scm-server-db fast_stop
  8. Stop the Cloudera Manager Agent.
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-agent
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent stop
  9. Upgrade the packages. Include the cloudera-manager-server-db-2 package in the command only if you are using the embedded PostgreSQL database.
      Note: If you receive the following error message when running these commands: [Errno 14] HTTP Error 404 - Not Found, make sure the URL in the cloudera-manage.listcloudera-manager-repo file is correct and is reachable from the Cloudera Manager server host.
    Redhat / CentOS
    sudo yum clean all
sudo yum upgrade cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    SLES
    sudo zypper clean --all
sudo zypper up cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    Debian / Ubuntu
    sudo apt-get clean
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    You might be prompted about your configuration file version: 
    Configuration file '/etc/cloudera-scm-agent/config.ini'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
    You might receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.
  10. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.

  11. Verify that you have the correct packages installed.

    rpm -qa 'cloudera-manager-*'
    cloudera-manager-server-5.15.0-..cm...
cloudera-manager-agent-5.15.0-..cm...
cloudera-manager-daemons-5.15.0-..cm...
cloudera-manager-server-db-2-5.15.0-..cm...
    dpkg-query -l 'cloudera-manager-*'
    Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                   Version                Description
+++-======================-======================-============================================================
ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
ii  cloudera-manager-serve 5.15.0-0.cm...~sq The Cloudera Manager Server
  12. If you are using the embedded PostgreSQL database, start the database :
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-server-db
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server-db start
  13. Start the Cloudera Manager Agent.
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-agent
    You should see no response if there are no errors starting the agent.
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent start
    You should see the following: 
    Starting cloudera-scm-agent: [ OK ]
  14. Start the Cloudera Manager Server.
    Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-server
    You should see no response if there are no errors starting the Cloudera Manager server.
    Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server start
    You should see the following: 
    Starting cloudera-scm-server: [ OK ]
  15. Log in to the Cloudera Manager Admin Console. It can take several minutes for Cloudera Manager Server to start, and the console is unavailable until the server startup is complete. The Upgrade Cloudera Manager page displays. Continue with the steps below to upgrade the Cloudera Manager Agents.
  Note: If you have problems starting the server or the agent, such as database permissions problems, you can use the server log /var/log/cloudera-scm-server/cloudera-scm-server.log or the agent log /var/log/cloudera-scm-agent/cloudera-scm-agent.log to troubleshoot the problem.

Upgrade the Cloudera Manager Agents

  1. You can upgrade the agents using one of the two options below.

    Option 1. Upgrade the Agents using Cloudera Manager (Recommended)

    After the Cloudera Manager server starts and you log in to the Cloudera Manager Admin Console, the Upgrade Cloudera Manager page displays. (It may take several minutes for the server to start.) This page displays the current status of your upgrade. A green check icon indicates that an upgrade step has been successfully completed.

    The status of Agent upgrades is displayed in one or more groups. Because you have only upgraded the Cloudera Manager agent on the Cloudera Manager server host, the first group shows that host as having upgraded agent. If the hosts manged by Cloudera Manager have different operating systems, a group for each operating system displays the Agent upgrade status for those hosts.

    After the Cloudera Manager server starts and you log in to the Cloudera Manager Admin Console, the Upgrade Cloudera Manager page displays. (It may take several minutes for the server to start.)

    1. If there are more than one group of hosts that require agent upgrades, select the group from the drop-down list labeled Upgrade Cloudera Manager Agent Packages running on:. If there is only one group that requires upgrades, this drop-down list does not appear.

      In Cloudera Manager 5.15 or higher, Cloudera Manager can upgrade agents even they are running on different operating systems or versions (One OS group at a time).

    2. Click Yes, I would like to upgrade Cloudera Manager Agent packages now.
    3. Click Continue.

      The Upgrade Cloudera Manager Agent Packages page displays.

    4. Click Upgrade Cloudera Manager Agent packages

      The Upgrade Cloudera Manager Agent Packages page displays.

    5. If you are using a local package repository instead of the public repository at https://archive.cloudera.com, select the Custom Repository option when installing the Cloudera Manager Agent packages and enter the Custom Repository URL:
      Redhat / CentOS

      Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

      SLES

      Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

      Debian / Ubuntu

      Use the entire deb url contrib line from the cloudera-Manager.list file as the Custom Repository. Use the url/archive.key as the Custom GPG Key URL.

    6. Click Continue.

      The Accept JDK License page displays.

    7. Select Install Oracle Java SE Development Kit (JDK 7) if you want to install JDK 7 on all hosts.
    8. Click Continue.

      The Enter Login Credentials page displays.

    9. Specify credentials and initiate Agent installation:
      1. Select root for the root account, or select Another user and enter the username for an account that has password-less sudo permission.
      2. Select an authentication method:
        • If you choose the All hosts accept same password option, enter and confirm the password.
        • If you choose the All hosts accept same private key option, provide a passphrase and path to the required key files.

        You can modify the default SSH port if necessary.

      3. Specify the maximum Number of Simultaneous Installations to run at once. The default and recommended value is 10. You can adjust this based on your network capacity.
    10. Click Continue.

      The Cloudera Manager Agent packages and, if selected, the JDK are installed.

      When the installation completes, the Upgrade Cloudera Manager page displays the status of the upgrade.

      If there are additional groups of hosts that require Agent upgrades, select the next group from the Upgrade Cloudera Manager Agent Packages running on: drop-down list, and repeat the agent installation steps.

    11. Click Run Host Inspector to run the host inspector

      The Host Inspector opens in a new tab and runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then rerun the inspector.

      When you are satisfied with the inspection results, return to the tab where the upgrade is running and click the link to go back to the Home Page.

      The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.

    12. Click Continue.

      The Cloudera Manager Agent packages and, if selected, the JDK are installed.

      The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then click Run Again to rerun the inspector.

    13. Click Continue.

      The Review Changes page displays and suggests configuration changes you may need to make.

    14. Make any necessary changes and click Continue.

      The Upgrade Wizard restarts the Cloudera Manager Management Service.

    15. Click Finish.
    Troubleshoot
    If the Upgrade Cloudera Manager page does not display after you upgraded the packages on the Cloudera Manager server host, open the following URL in your web browser:
    • Cloudera Manager 5.15 or higher: 
      https://my_cloudera_manager_server_host:port/cmf/upgrade
    • Cloudera Manager 5.14 or lower: 
      https://my_cloudera_manager_server_host:port/cmf/upgrade-wizard/welcome
    Option 2. Upgrade the Agents using the Command Line
    1. Assume your Cloudera Manager agents (exclude the Cloudera Manager Server) are running on 
      host1.example.com
...
hostn.example.com
    2. SSH to each Cloudera Manager host. 
      ssh host1.example.com
      You could also multiplex the same set of commands to all hosts by using csshX, pdsh or pssh.
    3.   Tip: If you have a mixed operating system environment, adjust the Operating System filter at the top of the page for each operating system. The guide will generate the repo file for you automatically here.
      Redhat / CentOS

      Create a file named cloudera_manager.repo with the following content:

      [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1
      SLES

      Create a file named cloudera_manager.repo with the following content:

      [cloudera-manager]
# Packages for Cloudera Manager
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/5.15
gpgkey=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/RPM-GPG-KEY-cloudera
gpgcheck=1
      Debian / Ubuntu

      Create a file named cloudera_manager.list with the following content:

      # Packages for Cloudera Manager
deb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
deb-src https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
    4. Backup the existing repository directory.
      Redhat / CentOS
      sudo cp -rf /etc/yum.repos.d $HOME/yum.repos.d-`date +%F`
      SLES
      sudo cp -rf /etc/zypp/repos.d $HOME/repos.d-`date +%F`
      Debian / Ubuntu
      sudo cp -rf /etc/apt/sources.list.d $HOME/sources.list.d-`date +%F`
    5. Remove any older files in the existing repository directory:
      Redhat / CentOS
      sudo rm /etc/yum.repos.d/cloudera*manager.repo*
      SLES
      sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
      Debian / Ubuntu
      sudo rm /etc/apt/sources.list.d/cloudera*.list*
    6. Copy the repository file created above to the repository directory:
      Redhat / CentOS
      sudo cp cloudera_manager.repo /etc/yum.repos.d/
      SLES
      sudo cp cloudera_manager.repo /etc/zypp/repos.d/
      Debian / Ubuntu
      sudo cp cloudera_manager.list /etc/apt/sources.list.d/
    7. Back up the agent directory. 
      cp -r /etc/cloudera-scm-agent $HOME/cloudera-scm-agent.bak
    8. Stop the Cloudera Manager Agent.
      Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
      sudo systemctl stop cloudera-scm-agent
      Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
      sudo service cloudera-scm-agent stop

    9. Upgrade the agent packages.

      Note: Only add cloudera-manager-server-db-2 if you are using the embedded PostgreSQL database.
      Redhat / CentOS
      sudo yum clean all
sudo yum repolist
      sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
      SLES
      sudo zypper clean --all
      sudo zypper up cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
      Debian / Ubuntu
      sudo apt-get clean
sudo apt-get update
sudo apt-get dist-upgrade
      sudo apt-get install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
      You might be prompted about your configuration file version: 
      Configuration file '/etc/cloudera-scm-agent/config.ini'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
      You might receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.
    10. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.

    11. Verify that you have the correct packages installed.

      rpm -qa 'cloudera-manager-*'
      cloudera-manager-agent-5.15.0-..cm...
cloudera-manager-daemons-5.15.0-..cm...
cloudera-manager-server-db-2-5.15.0-..cm...
    12. If you are using the embedded PostgreSQL database, start the database :
      Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
      sudo systemctl start cloudera-scm-server-db
      Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
      sudo service cloudera-scm-server-db start
    13. Start the Cloudera Manager Agent.
      Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
      sudo systemctl start cloudera-scm-agent
      You should see no response if there are no errors starting the agent.
      Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
      sudo service cloudera-scm-agent start
      You should see the following: 
      Starting cloudera-scm-agent: [ OK ]
    14. In Cloudera Manager 5.15 or higher, you can monitor the progress at https://my_cloudera_manager_server_host:port/cmf/upgrade.
    15. Run the Run Host Inspector: Select Hosts > All Hosts and click Inspect All Hosts.

      The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then rerun the inspector.

      The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.

  2. If you are upgrading from Cloudera Manager 5.5.0 or lower to Cloudera Manager 5.5.0 or higher:
    1. Stop the cluster:
      1. On the Home > Status tab, click to the right of the cluster name and select Stop.
      2. Click Stop in the confirmation screen. The Command Details window shows the progress of stopping services.

        When All services successfully stopped appears, the task is complete and you can close the Command Details window.

    2. Hard restart the agent on all hosts to update and restart the supervisord process:
      Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
      sudo service cloudera-scm-agent next_stop_hard
sudo service cloudera-scm-agent restart
      Other Linux Distributions:
      sudo service cloudera-scm-agent hard_restart
    3. Start the cluster:
      1. On the Home > Status tab, click to the right of the cluster name and select Start.
      2. Click Start that appears in the next screen to confirm. The Command Details window shows the progress of starting services.

        When All services successfully started appears, the task is complete and you can close the Command Details window.

  3. Start the Cloudera Management Service and adjust any configurations when prompted.
    1. Login to the Cloudera Manager Admin Console
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Start.

Perform Post Upgrade Steps

  1. If you upgraded the JDK, do the following:
    1. If the Cloudera Manager Server host is also running a Cloudera Manager Agent, restart the Cloudera Manager Server:
    2. Restart the Cloudera Manager Server.
      Redhat 7, SLES 12, Debian 8, Ubuntu 16.04
      sudo systemctl restart cloudera-scm-server
      Redhat 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
      sudo service cloudera-scm-server restart
    3. Open the Cloudera Manager Admin console and set the Java Home Directory property in the host configuration:
      1. Go to Home > All Hosts > Configuration
      2. Set the value to the path to the new JDK.
      3. Click Save Changes.
    4. Restart all services:
      1. On the Home > Status tab, click next to the cluster name, select Restart and confirm.
  2. If Cloudera Manager reports stale configurations after the upgrade, you may need to restart the cluster services and redeploy the client configurations. If you are also upgrading CDH, this step is not required.

    Stale configurations can occur after a Cloudera Manager upgrade when a default configuration value has changed, often required to fix a serious problem. Configuration changes that result in Cloudera Manager reporting stale configurations are described the Cloudera Manager release notes.

    1. On the Home > Status tab, click next to the cluster name, select Restart and confirm.
    2. On the Home > Status tab, click next to the cluster name, select Deploy Client Configuration and confirm.
  3. If upgrading from Navigator 2.6 (Cloudera Manager 5.7) or lower:
    1. Start and log into the Cloudera Navigator data management component UI. The Upgrading Navigator page displays. Depending on the amount of data in the Navigator Metadata Server storage directory, the upgrade process can take three to four hours or longer.

    When the upgrade is complete, click Continue. The Cloudera Navigator landing page is displayed.

  4. If you disabled any backup or snapshot jobs before the upgrade, now is a good time to re-enable them.
  5. The Cloudera Manager upgrade is now complete. If Cloudera Manager is not working correctly, or the upgrade did not complete, see Troubleshooting Installation Problems.

Upgrade Cloudera Navigator Encryption Components

Upgrade any Cloudera Navigator Encryption components deployed in your cluster:

  • Cloudera Navigator Key Trustee Server
  • Cloudera Navigator Key HSM
  • Cloudera Navigator Key Trustee KMS
  • Cloudera Navigator Encrypt.

If you are still using Key Trustee Server 5.4, and you are upgrading to Cloudera Manager 5.10 or higher, you must upgrade the Key Trustee Server to a more recent version.

You can upgrade other Cloudera Navigator components at any time. You do not have to perform these upgrades when upgrading Cloudera Manager or CDH.

See Upgrading Cloudera Navigator Components.

(Optional) Upgrade CDH

See Upgrading CDH Using Cloudera Manager.

Page generated May 18, 2018.