The Future of The DBA in The Era of The Autonomous Database

Introduction

The database world is abuzz with the prospect of autonomous databases. Like vehicles, autonomous databases are said to be self-driving, with all the benefits of lower costs and reduced error rates that come from removing the human factor of database administrators (DBAs).

That means you.

Will your job go away as DBA responsibilities dwindle? The marketing messages around autonomous databases suggest that it will, and some DBAs (and IT directors and CIOs) are buying in. It's tempting to think that the database will run itself and labor costs will plummet. And if DBAs continue to regard their job in the same way even as companies adopt autonomous databases, then yes, many DBAs will indeed go away.

But are autonomous databases the real problem you should be focused on as a DBA? Which other factors are at work? How does the future look for DBA responsibilities in the world of the autonomous database — or the Internet of Things, the hybrid cloud, software-defined storage and all of the other breaking technologies begging for your attention? How can you position yourself to thrive in such a future?

This eBook examines autonomous databases from the perspectives of companies interested in adopting them and of the DBAs they will most affect. CIOs and IT directors can use it to inform their decisions about purchasing and hiring, and DBAs can use it to better understand their future in the era of the autonomous database.

When requirements change, somebody needs to know the internals well enough to modify the model. As a DBA, you're in a good position to play that role.

"Autonomous" Doesn't Mean "Unbreakable"

First, consider that nature is notoriously hard on autonomous systems, especially in their early stages of development and rollout. In "Star Trek

III: The Search for Spock," Scotty, the chief engineer on the starship Enterprise, aptly observed, "The more they overthink the plumbing, the easier it is to stop up the drain," which is how unpredictable conditions have occasionally led to tough outcomes with self-driving cars. Closer to the wallet, Bloomberg revealed that "hedge funds that use artificial intelligence and machine learning in their trading process posted the worst month on record in February [2018]."

Someone needs to understand how the autonomous trading program/ vehicle/database is put together and how to repair it when something goes wrong — in mid-flight, if need be. Even if the code is sound, the input data can change dramatically or run far outside of original assumptions. When requirements change, somebody needs to know the internals well enough to modify the model.

As a DBA, you're in a good position to play that role.

What Is the DBA of Today?

You're still a DBA, but you probably have far more responsibilities now than even five years ago. The trend toward multiple platforms is nudging Oracle DBAs, for instance, to learn SQL Server (or NoSQL or Hadoop or MongoDB), and vice versa, because it's what the business needs. Adoption of big data products and data analytics also leads to more demands on the time and talent of DBAs.

At the same time, database automation is gradually reducing the amount of work required for repetitive tasks like installation, configuration, storage management and workload management. Adaptive Tuning reconciles a variety of inputs, without human intervention, to optimize database performance.

Automation is here and more is coming, so you had better figure out what else to do with your time.

And your career.

The smart database administrator is actually evolving into the data administrator who sees that it's not all about the database; rather, it's all about the knowledge.

The paradigm shift that deserves your attention is that the smart database administrator is actually evolving into the data administrator who sees that it's not all about the database; rather, it's all about the knowledge that helps answer business questions like "Why is product X selling so well?" "How can I sell more of product Y?" "Why is product X selling better than product Y?"

In other words, knowledge is data in business context and objectives. The business knows the questions to ask and DBAs like you know where to find the answers.

The future belongs to DBAs who align those questions and answers to come up with knowledge about the customer based on captured data: where customers are, how they have behaved in the past and what they prefer. Companies use that data to differentiate themselves and they need professional DBAs to retrieve it.

Consider the business value you can add by mining data for knowledge about the customer:

  • Differentiation — Most companies are trying to figure out how they can do better than an existing global brand such as Facebook, Apple, Amazon, Netflix or Google (FAANG). It takes active use of knowledge about the customer to carve out a niche, and even more to avoid being steamrolled by a giant.
  • Automated Processes — Knowledge of the customer is the key to automating mundane tasks like package shipping and hotel check-in. It makes unremarkable processes simply disappear.
  • Customization — With enough of the right data it is possible to offer unparalleled, individualized service through targeted marketing and promotions that know where customers are, what they like and details about them down to their sleeve length or dress size.

On the customer side, the devices that generate customer data are ubiquitous and the data is easy to obtain, even with directives like the General Data Protection Regulation (GDPR).

On the company side, more production systems are customer-facing and downtime is unacceptably expensive. That has caused high performance, high availability and disaster recovery to evolve from nice-to-have's into absolute requirements.

DBAs already know that the databases are protected. The DBAs of the future also know what's in them.

And on the DBA side, fewer of the dull tasks associated with that data require human effort. According to a survey by DBTA, "in many cases, since the routine and mundane elements in administering databases can be eliminated, DBAs will be free to focus on higher-value activities." DBAs already know that the databases are protected.

The DBAs of the future also know what's in them.

System design, system tuning and system understanding: A case study

In the evolution from database administration to data administration, the three elements of system design, system tuning and system understanding are paramount, as author Clay Jackson describes:

"I worked for a company with a number of plants that took orders for product. Normally, the plants would run a program every day at 4:00 a.m. to generate pick lists for the day. They would send them to computers on their forklifts to tell the drivers where to locate the loads, which the drivers would start picking and loading at 6:00 a.m. Separately, order processors would come in around 9:00 a.m. and start entering orders they had received.

"Instead of picking just once per day, one of the plants decided to pick multiple times per day — at 6:00 a.m., noon and 3:00 p.m. — to keep drivers busy and to ship more product to customers faster. The first day they tried that, the order processors started complaining around noon that the entry process had slowed to a crawl and that they could not enter orders or serve their customers. After about 15 minutes that spontaneously repaired itself. We DBAs received several complaints about the database being slow; we looked at it and saw that it was locking because of the sharp increase in volume. We knew that the picking program was causing the locking, but we didn't know why.

"The same locking and order processing slowdown took place at the 3:00 p.m. cycle. It took us most of the afternoon to discover that there was a startup parameter in the inventory program that controlled the number of pieces of the inventory system that the picking process would lock before it started its processing. It turned out that the system designer had thought, 'Picking takes place at 6:00 a.m., when nobody is processing orders, so I'll lock the entire inventory table.' He'd added a parameter that locked the whole table while picking was going on, effectively locking everybody out of inventory so that order processors could not enter orders until the picking process had finished.

"Thus, two different systems were trying to access the same data, and an important startup parameter was poorly documented. So, to coexist with the order management system, we changed the parameter to lock only the current order and any other orders associated with the same products. That took care of the problem.

"The point is that an autonomous database won't tell you that. All it's going to tell you is that there's a locking problem it can't solve. That's where skill and experience as a DBA who understands the application will pay off."

Thriving As a Future-Proof DBA

Consider a couple of trends and do your own extrapolations into the future.

First, you can be thankful that certain tasks will start taking less of your time, as shown in Figure 1:

Figure 1: Tasks taking less of DBAs' time

The survey results indicate that database automation is getting verification and monitoring tasks off the plate of most DBAs. Figure 2, however, shows the kinds of tasks likely to replace those tasks:

Figure 2: Upcoming challenges for DBAs

Although those tasks may be less mundane and require more skill than the ones automation is taking over, they still revolve around database administration rather than data administration.

To truly embark on a future of building knowledge about the customer from data, you'll have to start getting out of your cubicle and getting around.

Get out into the business and see what's going on

First, know your business. Get to know your internal auditors, for example. They look for things you're not accustomed to looking for. If auditors are focused on a particular part of the business or set of issues, it's probably because they suspect fraud or illegitimate activity, but more likely, they feel it's an important part of the business. Follow the auditors and offer to help them find the data they want. Look for opportunities to automate data retrieval and reporting for them.

Next, know your internal customers and find out what keeps them awake at night. Start close to home: What keeps your CIO or database manager awake at night? They may wonder, "Can we truly rely on our backups?" and it's easy for DBAs to test them. Or they may wonder, "What will be our recovery time if we have a disaster?" That's a more difficult question for a DBA to answer, but it's more important, and you're as close as anybody in the organization to being able to answer it.

Your goal is for the business to come to you wanting to know why a particular item isn't selling and asking whether you have data to explain it. You'll open the data catalog, look at the item's location in the store, its quality, the amount of time on the shelf, the number of returns, and put something together to help find out why. It's unlikely that you'll know the answer right away, but you're in a position to know about other kinds of data that can lead to the answer.

More broadly, you're looking for the answer to business-oriented questions such as "What makes the business successful?" and "In a year from now, how can we determine whether we've been successful?"

Pour the answers you get elsewhere in the organization through that filter.

For example, getting out into the business is the only way that DBAs in companies without budget for monitoring tools can stumble onto pain points and database-related problems. Suppose somebody in Accounting happens to complain to you about having to pay overtime at the end of every month because it takes longer and longer to cut checks and issue payments to vendors. They don't have SQL tuning tools, but you do, and with a few lines of code that you could probably write in your sleep, you can slash the amount of time and money Accounting spends at month-end.

Your goal is for the business to come to you wanting to know why a particular item isn't selling and asking whether you have data to explain it.

Finally, know your technology and the applications that your internal customers depend on to get their work done. Suppose your lab manager's system keeps crashing in the middle of the night, with no way for anybody to know or do anything about it until the next morning when they have to call the help desk and request a system restart to get going on their work. Even if you can't diagnose and troubleshoot the main problem, you can put an alarm on the system to detect failure and restart automatically, without the need to contact the help desk. Knowing something about the applications that support the business — including the applications that aren't in your wheelhouse — is a big step on the way to greater visibility and greater value as a DBA.

Measure what you manage

Think about measuring success at three different levels:

  • Metrics of your business overall — Key performance indicators, or KPIs, such as bottom-line profit/loss and growth/decline tell you how your company is doing.
  • Metrics of your internal customers — Sales may want to increase its numbers by 10 percent and Operations may want to open five new sites in the next six months. What kind of data do they need to achieve those goals, and how can you help them pull it together so they can use it effectively?
  • Your own metrics — The traditional metrics of all DBAs still apply, because you're still on the hook for uptime and availability, so do what you must to meet your service-level agreements (SLAs) for different groups. Accounting may need access only between 8:00 a.m. and 5:00 p.m., but global development teams may need access to their code repositories almost 24/7. Another important traditional metric is response time, where autonomous databases and the service provider hosting them introduce new variables. If your users expect a response from you within 10 minutes of an incident, ensure that your provider's schedule aligns with that SLA.

Knowing something about the applications that support the business — including the applications that aren't in your wheelhouse — is a big step on the way to greater visibility and greater value as a DBA.

Once you've identified what you're going to measure, think about how you'll measure. The downside to giving up dull tasks is that you also give up control over them; you can't control service providers, for example, but you can put sensible SLAs in place, measure performance against them and hold providers responsible for any lapses. Since your internal customers have set their own metrics and goals for using a given database, make sure that your own metrics align with them, even if it means you have to spend money on tools for monitoring or managing the database.

Nothing motivates quite like a paycheck, so some measurements are best tied to compensation. Consider building process improvement for service providers around a cookbook of process documents. Then tie a bonus to the number and quality of process documents that DBAs create each year related to their respective databases. That will make a huge difference in how they set their priorities.

Use tools not only to manage your databases but also to automate the process of measuring how they're performing against your metrics.

Finally, use tools not only to manage your databases but also to automate the process of measuring how they're performing against your metrics. Assume that your autonomous databases will take care of themselves 99 percent of the time and that they will alert you when they cannot. That leaves you as DBA free from worry about mundane things like patching servers, rebuilding fragmented indexes and adding space for growing tables.

Somebody is going to be your company's next chief data officer. It may as well be you.

Conclusion: It's A Great Time to Be A Data Administrator

What, then, is the future of DBAs in the era of the autonomous database? What will the DBA of the future look like?

Autonomous databases are probably not going to eat your lunch — at least, not all of it. But smart DBAs look beyond the hype around autonomous databases, embrace the automation of many database administration tasks and position themselves for a data administration role in their organization.

Learn the business and show that you've learned it by getting out of your cube. Once you understand how your business works and how useful your knowledge can be to some business managers, you'll increase your value to the organization. It will be a long time before anyone figures out how to automate that.

Learn the technology that keeps your business running. That means more than learning how to write Java code; it means learning how the pieces interact and fit into the overall plumbing, how all of your company's database components work. Know what happens, for example, when you lock the inventory table and how that affects other business functions, so you don't send yourself and your fellow DBAs on a day-long hunt like the one described above.

Become a trusted advisor by asking questions in the terms your business managers understand: "Why do you need same-store sales by product by salesperson?" "Which products bring people in for the first time and which ones keep them coming back?" "Which data points don't you have that you wish you did have?"

Data is only growing — in both volume and importance — so every organization needs experts to make it available and turn it into knowledge of the customer.

In short, somebody is going to be your company's next chief data officer. It may as well be you.

About The Author

Clay Jackson is a Database Systems Consultant for Quest, specializing in Database Performance Management and Replication Tools. Prior to joining Quest, Jackson was the DBA Manager at Darigold. He also spent over ten years managing Oracle, SQLServer and DB2 databases and DBAs at Washington Mutual. While at WaMu, Jackson was the Enterprise Database Compliance Officer, with responsibility for Database Security and Disaster Recovery. He also worked at Microsoft and Starbucks, is a CISM and has a Masters in Software Engineering from Seattle University.

About Quest

At Quest, our purpose is to solve complex problems with simple solutions. We accomplish this with a philosophy focused on great products, great service and an overall goal of being simple to do business with. Our vision is to deliver technology that eliminates the need to choose between efficiency and effectiveness, which means you and your organization can spend less time on IT administration and more time on business innovation.