This post takes you through the process of upgrading a MapR cluster, beginning with what to include in a cluster upgrade plan, how to perform pre-upgrade testing, and then to upgrading MapR and patching core software, ecosystem components, and MapR clients.
Plan the Upgrade
Identify Upgrade Methods
Software Releases and Patches
Each MapR release has a version number written in the format X.Y.Z
Upgrade the software to move from one version to the next
Patch the software to update to the current version. It is advisable to install patches after upgrade, when the patch is installed the MapR software version doesn’t change.
Select an Upgrade Method
- Upgrades can be performed Offline/Rolling and Manual/Scripted. Choosing an Upgrade method will involve assessing the risks and availability requirements of the cluster.
- An Offline upgrade poses least risk since it’s simpler where the complete cluster is offline and upgrade are performed manually or scripted in a short time.
- Rolling upgrades are complex and performed in stages (a few nodes at a time) while the cluster remains available; which requires Highly Availability features so that critical services running on a node that is taken offline will automatically startup in an another node.
- Group nodes for upgrades so that critical services are available
- Plan when there is very low cluster activity
- Cluster users should expect performance segregation and its best to limit cluster operations to only critical functions during rolling upgrade.
Community Edition and Rolling Upgrades
- You can perform Rolling Upgrade on Community Edition with single services.
- Expect interruptions when nodes running the only copy of a service are upgraded
- Minimize cluster access
- With 10 or fewer nodes, an offline upgrade probably makes the most sense
Supported Upgrade Methods (to v5.2)
* Only supported for versions of MapR that were installed using the MapR Installer
Method One: Offline Upgrade
An offline installer follows the following steps.
MapR Installer cannot stop non-MapReduce jobs, so
mapr admin should perform following steps :
- Stop non-MapReduce jobs
- Start MapR Installer (if using)
Administrator or Installer:
Remaining steps completed by Installer or Manually by adminstrator are :
- Stop MapReduce jobs
- Stops cluster services
- Upgrades packages on all nodes
- Brings the cluster back online
- Upgrades ecosystem components
Method Two: Scripted Rolling Upgrade
- Upgrades MapR software one node at a time
- Cluster remains operational
- MFS service on each node goes down while packages are upgraded
- Does not raise the under-replication alarm since time required for upgrade is short enough.
- No ecosystem components are upgraded
- Process can be monitored from MCS from any node that is not currently being upgraded.
- After upgrade, Warden restarts services
- Including ecosystem components if compatible with the new release.
Method Three: Manual Rolling Upgrade
- Upgrade MapR software one node at a time
- Cluster remains operational
- Possible to upgrade some nodes in parallel
- Plan carefully!
- Services on nodes being upgraded are unavailable
Develop a Plan and Prepare for Upgrade
Plan: Determine What to Include
* MEPs were introduced in MapR 5.2
Plan: Develop a Test Plan
- Run tests before and after each upgrade step
- Ecosystem components
- Test basic functionality
- Verify cluster access and volumes
- Use maprcli, hadoop fs, MCS
- Test jobs and queries
- Based on the components you use
Plan: Create an Upgrade Schedule
Select an Upgrade Method
- Review release notes
- Behaviour changes?
- New features to enable?
- Download the installer, packages, etc.
- Verify node specifications
- Upgrade on a test cluster
- Document surprises
- Prepare configuration files for production cluster
- Remind cluster users of upcoming upgrade
- Verify cluster health
- Run tests and record results
- Back up critical data
Day of Upgrade
- Verify cluster health and clear alarms
- Terminate jobs and clear the pipeline
- Stop cross-cluster operations
- Volume mirroring
- Table replication
- Upgrade, then verify and test
- Update applications to use new binaries
- Enable and test new features
- Put the cluster back into full production mode
Summarise the Upgrade Process
Included During an Upgrade of MapR Core
- Ecosystem components
- MapR Clients
- Edge nodes that are not of the cluster
- Exception: ecosystem components are upgraded if using the Installer
New Files and Folders
During the upgrade process :
- Product binaries are replaced with new versions
- Additional folders may be created (if required for upgrade)
New default configuration files created when new features are added during upgrade.
Important! Merge required changes into active configuration files.
Following steps are performed when upgrade includes a new Hadoop Common.
Upgrade MapR Core
Prepare to Upgrade
High Level Overview
Upgrade MapR Core Software
- Update Configuration Files
- Observe and merge changes or new configurations
- Manually update configuration files on all nodes
- Save time by doing this step during upgrade on test cluster
- Update License Files
- First upgrade to MapR 5.1 or later requires update to base license on all nodes during to the changes implemented in licensing.
Verify and Validate
Upgrade Ecosystem Components and MapR Clients
Upgrade Ecosystem Components
- You don’t need to upgrade all ecosystem components together during upgrade.
- Determining what ecosystems requires upgrade is part of pre-upgrade planning
- MapR 5.2, introduces MapR Ecosystem Packs or MEPs.
- When you upgrade ecosystem components upgrade to versions that all include same MEPs, choose one of the MEPs available for a MapR version to ensure they work together well.
- As of MapR 5.2, must use ecosystem components that belong to the same MapR Ecosystem Pack (MEP)
- New MEPs released on a regular basis
- You may choose not to upgrade even if a new version is available for MEPs.
- Can wait unless they are incompatible with new version of MapR
- Follow pre * and post-upgrade steps in the MapR documentation
- Always upgrade ecosystem components after an upgrade from MapR 3.x
Minor and Major Upgrades
Ecosystem components may have
- Minor upgrades * package or patch update
- Major upgrades * take you to new version
Minor upgrade may just require a package update
- Verify repository is configured to get the latest packages
Determine package names
Major upgrade may require additional steps
There may be pre * and post * upgrade testing required depending on the component :
- Perform pre-upgrade steps (Ex: Hive * backup configuration)
- Perform post-upgrade steps (Ex: Hive * update the schema)
Once you complete the upgrade perform the same pre-upgrade tests and compare the results.
Upgrade MapR Clients
Types of Client Access
Upgrade or Migrate?
The Direct Access NFS functionality is provided in the mapr-nfs core package and does not need to be upgraded separately. So we need to upgrade/migrate the POSIX/MapR client
If you choose to upgrade your NFS loopback POSIX client follow these steps :
Note! Support for NFS loopback client will be removed from a future version of MapR.
- We recommend to migrate to FUSE client rather than upgrading NFS client.
- Platinum POSIX client requires a new license
Finish up the process by testing the new client after upgrade or migrate.
Additional Upgrade Considerations
Patching a Fresh Upgrade
After upgrading to new version of MapR check for minor patches that provide minor code release that are not included with base version upgrade.
Check for a patch at:
Considerations for Multiple Clusters
Mirroring Between Clusters
- Volumes must be mirrored to a cluster at the same, or higher, version
- Upgrade the destination cluster first
- Upgrade both clusters if mirroring both ways
Table Replication Between Clusters
- Clusters involved in table replication can be at different versions
Enable New Features
- Review release notes for automatically enabled features
- Anticipate behavior changes
- Recommended: wait to enable optional new features
- Make sure the upgraded cluster is stable before implementing new features