The Importance of Continuous Monitoring in DevOps Pipeline

As a database professional, you don't have a moment to rest. You're under pressure to quickly deploy new software and updates to your production applications so you can stay ahead in competitive markets and keep pace with rapidly evolving technology. To meet this challenge, more and more organizations are turning to DevOps — a set of practices that emphasize collaboration and communication between development, operations and other functional areas to enable the building, testing and release of software rapidly and reliably.

What Is DevOps?

Often, DevOps is described as a set of processes, such as continuous integration (CI), continuous deployment (CD), or as set of tools which enable faster time to market. However, these processes and tools are means to an end, not the end itself.

DevOps is best understood as a business initiative: to improve communication and collaboration among stakeholders and increase the speed and quality of software deployment. To achieve these goals, DevOps requires cultural changes, automated processes and investments in the right technologies (see Figure 1).

Figure 1. DevOps applied automation and performance monitoring throughout the development lifecycle enables shorter development cycles and higher quality releases.

The most important terms in DevOps are:

  • Agile Development — development methodology for releasing smaller sets of changes into production more frequently
  • Continuous Integration — process in which check-in of code changes triggers an automated build process to test and QA
  • Continuous Delivery — process for releasing code into production automatically.
  • Continuous Monitoring — process for continuously measuring the performance of the environment and the application.

What Does DevOps Offer?

The successful implementation of a DevOps culture offers a wealth of benefits for your IT team and your organization, including:

  • More frequent software deployments
  • Faster time to market for new features
  • Shorter times between software fixes
  • Better application quality and performance
  • Increased operational efficiency
  • Reduced risk

DevOps And You

Although DevOps is a relatively new culture, as a DBA you have always been part of DevOps because you're deeply involved in development aspects (SQL, performance tuning, Object Analysis, reporting) as well as operations aspects (server configuration, backup and restores, OS/network/storage tuning, etc.). The main change in recent years is that DevOps has hit the database team, demanding more automation and collaboration to accelerate application updates and delivery. It helps you efficiently and proactively identify application performance problems before they become a real issue.

The Role of Continuous Monitoring in DevOps

One of your most important tasks is to keep application databases performing well and, ultimately, keep your end users satisfied. In fact, according to recent research conducted by DBTA1, 66 percent of DBAs who responded said that performance is their most important responsibility when managing databases in the cloud, and 63 percent said that performance is the responsibility that takes up most of their time. These numbers emphasize the importance of database monitoring tools and the fact that database performance monitoring is closely related to DevOps culture.

DevOps is often about automation and automating database performance monitoring to ensure your databases operate at peak stability. Performance is an essential part of that; therefore, monitoring is an essential part of DevOps. Operations teams, development teams and DBAs can benefit from sharing a common set of performance data to support production application databases. The data can and should be relevant to the stakeholder's need, whether it is high-level or granular.

The Challenge of Performance Monitoring In Complex Database Environments

According to the DBTA survey, 64 percent of DBAs responsible for managing relational database management systems are also responsible for managing non-relational database management systems (i.e., NoSQL solutions). Moreover, 66 percent of respondents said that the number of databases they are responsible for is increasing. Adding to the challenge, many organizations use several tools for monitoring their database performance. This makes it difficult for you to identify and correlate issues. The learning curve for you and your IT organization increases which also results in operational and economical inefficiency.

Another common challenge in many IT teams is the "blame game" where application developers blame infrastructure teams for stability and performance issues and vice versa. Solving this challenge is one of the main goals of DevOps. The DevOps culture aims for collaboration and shared responsibilities, which should minimize or eliminate the blame game. However, it's not always easy to understand the root cause of an application performance or stability issue and correlate application or infrastructure changes to the actual impact on the database workload and the application performance.

How Foglight® For Cross-Platform Databases Can Help

Foglight for Cross-Platform Databases provides 24/7 cross-platform database visibility (see Figure 2). It provides DevOps teams a proactive approach for monitoring their entire database environment at a glance and allows them to understand whether there is any stability or performance problem before customers complain. It supports various database technologies including: Oracle, SQL Server, DB2, SAP ASE, PostgreSQL, MySQL, MongoDB and Cassandra. According to our customer, a DBA from a financial services company: "Our organization is using Foglight to monitor the enterprise data warehouse for realtime and historical usage patterns. This shortens the time between problem identification and fixing and as such, is the enabler of the DevOps culture."

Figure 2. Foglight for Databases — Cross Platform Visibility

Another common aspect of DevOps is leveraging test environments that simulate production environments. Foglight for Cross-Platform Databases allows comparing the performance and configuration across database instances which can be used in DevOps culture to compare development and testing environments. In addition, Foglight features a smart baseline algorithm which defines the expected environment workload profile at any point in time — hour in a day, day in a week, week in a month (see Figure 3). Your DevOps teams can leverage the tool to easily understand whether the database environment experienced abnormal activity.

Figure 3. Foglight for Databases Compare Tool

Tracking application and infrastructure changes plays a major role in DevOps culture. In many cases, database code, schema changes and infrastructure changes may impact the application performance which results in increasing host resource utilization and poor quality of service of the application. As previously mentioned, a big challenge which exists for IT teams is how to correlate between database changes, infrastructure changes and their associated impact on the environment workload. Foglight for Cross-Platform Databases can solve the "blame game" between development and operations by visually correlating database/application changes (schema, database configuration, execution plans) as well as infrastructure level (disk, CPU, memory) and show how these changes impacted the database performance (see Figure 4). By solving the "blame game" between development and operations, Foglight is the enabler of your DevOps culture.

Figure 4. Foglight for Databases Change Tracking Tool

Conclusion

With both the business environment and the technology landscape changing and growing rapidly, your organization needs to be nimble. DevOps helps your organization increase the speed and the quality of software deployments by improving communication and collaboration among stakeholders. Foglight provides visibility across multiple platforms through a single-console for visibility into database performance across your entire environment and enables development, DBAs, and application teams to share the same consistent performance metrics and understand the environment health at any given time. Foglight for Cross-Platform Databases helps you compare performance and configuration across production and test environments which is a significant part of DevOps. In addition, it can correlate database and infrastructure changes to the actual application performance. This reduces the gaps between your development and operations teams and by doing so, it enables DevOps culture.

To learn more about Foglight for Cross-Platform Databases, please visit www.quest.com/products/foglight-for-cross-platform-databases.

About Quest

At Quest, our purpose is to solve complex problems with simple solutions. We accomplish this with a philosophy focused on great products, great service and an overall goal of being simple to do business with. Our vision is to deliver technology that eliminates the need to choose between efficiency and effectiveness, which means you and your organization can spend less time on IT administration and more time on business innovation.