Cloudera Manager And The Truth About Monitoring Applications vs. The Cluster

Enterprises that are using big data applications to successfully accelerate their growth will quickly find themselves leaning on their solution more than they ever imagined. For managers and practitioners in the DevOps department, this business-wide reliance on big data can be both a blessing and a curse.

In a business environment where 0% downtime is now the expectation, efficient troubleshooting and correction of technical issues with software is critical. Because data is coming from multiple sources, there is an amalgam of technologies that come together to execute a big data application workflow, including multiple monitoring tools (like Cloudera Manager, for instance). As a result, troubleshooting performance issues can be especially difficult.

The natural response is to implement whatever tools get the data you need in order to move forward. Several software companies have developed the tools they think answer enterprise big data needs, but the Hadoop landscape is a very complex world, how do you know which software solution is right for your installation?

For the longest time, managers and DevOps teams have relied on monitoring things just at the cluster level rather than a deep dive into how the Hadoop applications are performing. Cloudera Manager has one of the best solutions for monitoring the cluster, but this only solves a specific set of questions. To scale your production implementation of Hadoop or Spark, you need a more in-depth look into application performance and go beyond cluster performance monitoring. Here’s how DRIVEN provides the application performance visibility you need than just relying on Cloudera Manager.

Comparing Cloudera Manager To DRIVEN Is ‘Apples to Oranges’ For Application Monitoring

It’s only natural to compare Cloudera Manager and DRIVEN for monitoring big data application performance. But there is no one size fits all for the needs of every organization and these two solutions just don’t do the same thing.

As the old idiom goes, comparing apples to oranges is an oversimplified – but accurate – way to describe the differences between Cloudera and DRIVEN. Although when you take into consideration how each software performs, the correct comparison might actually be comparing a magnifying glass to an electron microscope. While one of the tools (Cloudera) is perfect for looking at performance issues on the surface, the other (DRIVEN) gives you an extremely granular look, allowing your DevOps team to better pinpoint and correct performance issues at the source of the problem.

Cloudera Manager is built specifically for cluster performance monitoring while DRIVEN’s specific purpose is to monitor the performance of all your big data applications. Monitoring the cluster is important, and you will be able to learn when something has failed using Cloudera – but you will not be able to determine why it failed. With an application monitoring tool like DRIVEN, your team will be provided answers for where it failed, why it failed, who is responsible, its business priority, and the downstream impact of that failure.

The easiest way to think about these two is understanding what level of visibility each one offers:

• Cloudera = Macro level visibility and management
• DRIVEN = Micro level visibility for more proactive analysis

Is There Anything Cloudera Can Do That Driven Can’t Do?

Because cluster and application monitoring are so different from one another, DRIVEN is the best way to know how your big data applications are performing. It does not monitor node health, CPU utilization, I/O utilization and more among other cluster monitoring factors.

Another thing that DRIVEN was not designed to do is act as a cluster deployment solution. Tools which already come with your particular distribution are designed to deploy their technology. For cluster performance management, Cloudera Manager or any other similar solution could fill this need. It is these differences between application and cluster performance monitoring that Cloudera Manager in combination with DRIVEN would provide next level visibility and complement each other quite well.

Hadoop Performance Management: The Cluster Or The App? Which Is More Important To You?

Cloudera Manager continues to be one of the market leaders for Apache Hadoop big data management space. However, it was not designed to be an appropriate tool for monitoring application performance.

When application performance fails completely (or even suffers), lack of visibility into the problem can be terrible for DevOps professionals. In order to pinpoint a problem in your big data application and optimize it efficiently, DRIVEN is the best software on the market to understand and improve the performance of your application.

To learn more about how DRIVEN compares to other Hadoop performance monitoring tools, download the full guide.