Digital Transformation Powered by AIOps

IT departments are central to all digital transformation strategies, but they are in need of help, due to the existing complexity of infrastructure and application footprints.

63% of organizations reported that time spent on correlating data from different sources is the key challenge for usability of IT performance monitoring tools1. This reflects more in business downtime and outage costs. Companies end up spending 72% of their budget just "keeping the lights on" instead of focusing on innovation and spending time increasing the value of their business.

The average organization contains at least five IT monitoring tools, all doing specific but different tasks, and each producing its own siloed data. This huge amount of data is superficially unrelated and disorganized. There is still no holistic view of the IT environment available in a context that businesses can understand.

To deal with the problem, we need the help of AIOps (Artificial Intelligence Operations).

AIOps platforms use big data, machine learning and other advanced analytics technologies to enhance IT operations (monitoring, automation and Service Desk) with proactive, customized and dynamic awareness. Instead of replacing all of your existing tools, which can potentially take years, AIOps takes over the pain of correlating disparate data to automate repetitive processes. This will give your organization free time to work on revenue generating IT projects and ensure that IT service delivery is optimized.

AIOps is defined as: "Software platforms that combine big data and machine learning functionality to enhance and partially replace a broad range of IT Operations processes and tasks, including IT monitoring, event correlation and analysis, IT service management, and automation."

So Why AIOps?

The emphasis of AIOps platforms is in the ability to collect all types of data from various IT management systems, in a scalable and rapid way. It then applies business context and smart automation on that data to enable your IT teams to be smarter, more responsive and proactive. The key benefits of AIOps are:

Prevent downtime and improve service delivery

  • Monitor proactively and resolve issues before they cause downtime

Move from siloed teams to IT business partners

  • Have your teams correlate issues and work together to pinpoint root-causes and business value

Automate tedious manual tasks

  • Resolve repetitive issues in the background without the need for manual intervention

Develop your IT maturity

  • Use best-in-class technology to supplement leading processes and skill sets which will let your teams focus on more strategic opportunities

The IT maturity journey

Achieving a successful digital transformation via an AIOps strategy can be a journey for most organizations. With an end goal of becoming an IT business partner, it is necessary to consider these three focus areas:

  1. Technology: IT Infrastructure and Operations teams need to be able to see high volumes of their complex technical data in a business context. This requires an AIOps solution that can handle large volumes of data and provide insight for users.
  2. Metrics: IT teams must have reliable and easy-to-measure metrics to benchmark against. In order to measure value over time, SLA reporting and set targets should be put in place.
  3. People & Processes: AIOps is part of a continuous improvement paradigm. All IT teams should be involved over time to reach higher IT maturity.

Implementing an AIOps solution

Savision advises a six step approach to implementing a successful AIOps solution. The end goal should be a holistic platform that integrates with all existing IT Management tools. To achieve this, it is important to start small by adding value with a single integration, provide improved visibility for one tool at a time, improve service monitoring and off er automation. Once that value is proven, the next phase of the deployment should be to add further data-sources.

1. Big Data Platform

AIOps platforms should be fully extensible to allow scalability in depth and breadth across your IT organization. The main requirements for a platform should include:

  • Data collection and storage of millions of records
  • Speed and responsiveness
  • Natural search and analytic capabilities
  • No vendor lock-in to underlying tools

Savision uses Elastic Stack to provide this big data platform. Built on open-source technology from Apache Lucene, this is a well-established enterprise class technology that provides the speed and scalability to manage huge data-volumes and remain flexible enough to handle data-sources from any tools that generate log fi les.

Data Normalization

The last piece needed to provide a great search facility for IT organizations is the ability to look at related data in a common way. Users will not want to interpret operational alerts in seven different ways because it came from seven different tools that each have a different format. There needs to be a consistent way of interacting with common data types such as: alerts, incidents, computers, websites, databases, etc.

Savision has over 10 years of experience in the IT Operations world, working with tools such as Microsoft SCOM, Solarwinds, and Nagios. By applying business context to alerts (coming from components and grouping them into business services), you can assess business impact and focus on issues that are affecting end-users.

Savision developed a universal data schema that can store any data point collected dynamically from an IT organization, but also normalize those data points so that they can be easily managed. For instance, Savision is able to present all computers and alerts in a common way even if they are managed from completely different IT management systems.

2. Dashboarding

A key feature of Savision's AIOps platform is to provide value to users from IT administrators, Helpdesk, or IT managers. Data visualization is critical in order to provide context on alert noise issues that come from a single monitoring tool.

Savision is able to quickly build custom dashboards to visualize your IT environment and share with stakeholders on any device. Technical dashboards are automatically created based upon standard business applications (e.g. Microsoft Exchange, Active Directory, CRM, Order Entry), together with high-level dashboards for any persona.

Savision Dashboarding features include:

  • Real-time dashboards of your environment
  • Performance widgets showing key trends
  • Dashboards that are customized for business users, technical admins and Service Desk

3. Service Monitoring

IT Operations run many business and custom applications over virtualized infrastructure. These are the tangible pieces that traditional monitoring tools are built to actively maintain. However, IT teams are ultimately responsible for delivering IT services such as Email, Telephony, CRM etc. These services, are relying on a set of components that all need to function in order for the service to be up and running. With a traditional mindset, you would need to monitor all the components individually to make sure they have a healthy state. But where's the relationship between these components?

This is where service-oriented monitoring comes into the picture. Instead of watching a single component, the complete service should be monitored with all its underlying components. This can be done within AIOps, where we can visualize these relationships. For your web shop to function properly, you need the following components to work:

  • The web site where the users navigate to create their order
  • The underlying databases where the orders are written
  • The servers hosting the above components
  • If any of these components were to fail, you would lose your orders and the money that came with it

When speaking of service-oriented monitoring, there are three layers that need to be defined:

  • End-user components
  • Application components
  • Infrastructure components

Once you have defined these layers, you can start monitoring the complete service.

Other important features include:

  • Dynamic service maintenance - Savision can quickly build Distributed Applications that are dynamic and stay up-to-date with advanced inclusion/exclusion rules. It lowers the time to create and maintain your monitoring environment even as changes happen.
  • Service Discovery - Discover services based upon standard templates. Savision provides out-of-the-box templates for common enterprise applications such as Microsoft O365, Microsoft Exchange and Active Directory. Or import your services directly from CMDB of your ITSM tool.

4. Automation

Now that we have created situational awareness from dashboards and service focused monitoring, the next step is to automate processes to save time for IT departments.

Savision is able to filter out alert noise and focus on real incidents to become proactive. The next step is to automate incident workflows to speed up MTTR (Mean-Time-To-Resolution).

  • Alert correlation to services and groups - multiple alerts are now correlated to the business service level automatically, meaning you are not overwhelmed with noise and can focus on service delivery.
  • Automatic incident creation and resolution - alerts are grouped into incidents with the potential to create an incident in your ITSM system. This will keep the full history of related alerts and when an incident is closed, the related alerts are closed out automatically.
  • Run PowerShell Scripts - incident creation can trigger a PowerShell Script to run some actions and restore services without any manual intervention needed.
  • Smartphone notifications - you will be notified of critical events that affect end-users. Take action on-the-go, directly from your mobile or tablet.

5. Unify IT Systems

Once a single integration has been deployed, the model can now be reproduced across all other IT management systems. The holistic view or single pane of glass can be rolled out throughout your multiple tools and IT teams. Integrate all of your IT monitoring, IT service management, public, and private cloud data into a single pane of glass. Integration packs for all major IT systems include:

  • Monitoring – add in your Infrastructure/APM and EUE monitoring tools into one central database. Savision can normalize the alert and component data by grouping them together into service views (regardless of the underlying monitoring tools) and will provide your teams with a single unified view of the IT environment.
  • Public clouds – bring in your public cloud applications IaaS information from AWS, Azure and VMware. Combining this with on-prem monitoring provides visibility of distributed cloud applications.
  • ITSM – integrating with your ITSM tools provides the ability to automatically create and close incidents based upon your monitoring. This will make your Help Desk more proactive in responding to issues before end users start logging tickets.

6. Machine Learning

The final part of an AIOps deployment can deliver great value to your business over time. With all data-sources ingested and normalized into a central database, it is now possible to make use of machine learning capabilities.

"Time Series Anomaly Detection" is a capability used by unsupervised machine learning. This approach builds a steady baseline of your IT operations overtime. When anomalies appear that are outside of normal operations, an incident is created automatically. Automation rules can then kick in if the anomaly matches a similar previous event by creating immediate remediation of the issue.

Conclusion

AIOps is a transformative technology for the enterprise today. The process is a journey and should be approached in a modular method. Focus should be put on building out value on every single IT Management tool. Over time an AIOps platform can spread throughout the organization, eventually with machine learning and automation taking over many of the manual tasks that administrators perform today. A successful implementation can assist a Digital Transformation strategy and increase the IT maturity of any enterprise to become an IT business partner.