How to Start a DataOps Transformation at Your Organization

00:00:00

With that, you can take it away, Chris. Oh, thanks for the introduction, Beth. I'm going to turn now that you know what I look like, I'm going to turn off my video, and hopefully you can see my screen. So we're going to talk about taking an enterprise from zero to doing DataOps, and we're calling that the enterprise DataOps transformation because we're focused on enterprises to do it, bigger companies.

And so we're going to, for those of you who don't know DataOps, we're going to set up DataOps and then we're going to talk about transformation, then we're going to talk about six steps to do it and then have a conclusion. And so

if you've been in some of these webinars before, the term DataOps is a bit on a roll, like search volume's increasing. The Gartner analyst inquiry has definitely gone up 1,000%. It's sort of slowly moving up the hype cycle. There's more companies that have been in. We have 12,000 people that have signed the manifesto, so it's getting to be a bit of a thing, which I think is really good.

But like anything, that thing it's got different definitions and some people are saying DataOps is just DevOps for data, or it's the application of Agile to data science and engineering. Some people are rebranding their ETL and database stuff. Some people use DataOps and Model Ops and MLOps and DataGovOps, and so there's a little bit of confusion. So in order to do that, I'm just going to walk through our definition of DataOps, which is similar to Gene Kim's definition of DevOps, but it's a set of technical practices and cultural norms and architecture patterns that really focus on how fast you can experiment and try things out, because iterations lead to innovation.

And the second is how you can think of your systems end-to-end and lower error rates so your customers trust the data. And then how to collaborate when you've got data scientists and data engineers and self-service people all across the organization, and you've got lots of tech and environments. And then, of course, measuring and monitoring.

So,

in these slides, there's sort of source data on the left and data customers on the right, and we all love data and analytics, right? New oil, it's exciting. Big data, AI. I've been in the field now for a long time, and the problem is there's a lot of failure. A lot of projects sort of never get to production.

A lot of people calling themselves data-driven is down. A lot of analytics projects fail. A lot of projects have too many errors, and so it's sort of one of the dirty little secrets in data and analytics. And so I think there's a couple of reasons why that is and that DataOps addresses, and I think the first is really that if you look at how team and projects are run, they tend to run in a way that's more waterfall, where it's sort of labor-intensive and slow and takes weeks or months to deliver value to the customer. And a better way is this way that comes out of Agile or a more iterative way, where you deliver small bits of new insight to your customer, get value, get it in the bank, and then iterate and improve.

And it's sort of incremental and it's really focused on are you delivering value to your customer than are you delivering the project management artifacts that your waterfall process has. So it's really value delivery in increments that really matter and focusing on what your customer needs, not on kind of filling out your process. And I think the second thing is that

there's a lot of data mistrust, and one of the ways that we think the source of the data mistrust is because, yeah, there are data quality problems, the source data has problems, but there are also data processing problems where the transformation or the actions that happen to the data are wrong. And the both of those combine to having what I think of as errors, and that really means your data customer is finding a problem and it also causes people to sort of not really like going into work in the morning because they get that nasty email, which is a problem I experienced for many years.

And then the real fix to that, I think, is to apply principles from Deming and lean manufacturing and think of end-to-end, the work that you do across all your tools, across all your teams as a production line, and try to work that production line so you have low errors. And if you find problems in the data or find problems in the processing and the data, you get notifications.

And the last problem is that there's a lot of people who do data and analytics. There's data scientists and data engineers. We sometimes work for the same boss, we sometimes don't. Sometimes we have a production department who takes the work that we created and run it, and sometimes we push it ourselves. And so multi-team coordination is hard, just across the landscape of how big organizations work. And so I guess the fix of that, and that we're going to talk about it, and that's what

00:05:00

DataOps is, is how to collaborate without having more meetings. And can you build a system that enables collaboration and enables people to work where they don't have blinders on like a horse, and they can kind of see the world that they live in and get visibility to it. And so that's the third problem. And so how do you do this?

And I think one way that I'm going to talk about it is think in process cycles. And so if you look at your production process, where data comes in on one side and customers on the other, and there's a bunch of tools that it goes through. Maybe your data engineering tool is Informatica, maybe your data science tool is SaaS, maybe your data viz tool is Power BI, and you've got databases and cloud providers or on-prem.

But there's a process that it goes through from an event to a batch to deliver value to your customers. And when you run that process, think of it as a production line where you can lower errors. And across all your tools, all the code and configuration that drives those tools, all the data that is going into those tools And as that factory is running, can you focus on lowering errors and increasing customer data trust?

The second process that I think you have to think about is your deployment process. How do you get new things to those tools? And so perhaps there's a new data set that needs to go into production. Perhaps there's a change in how you're aggregating data or a change in a model. How do you deploy those from dev to production? How do you increase the cycle time at which you can make that happen?

And that cycle time of change is really the speed at which you can deliver new value, new charts and graphs, new data sets to your customer. And if you can do that faster, the ideas from Agile take place, you can learn more, and then you can maximize the amount of work you don't have to do, because then you don't spend three months doing something just to see the customer go, "Ah, that's not quite right." You've got to make a change.

And then the last cycle I'd like you to think about is your development cycle. And so your data engineering team, your data science team, your data visualization team, or self-service people, they all have cycles in which they do work. Data engineers may be doing a more documented IT-driven, we call it an SLDC, a software development life cycle. Your data scientists may be experimenting with Agile. Your data visualization people may be just pressing a button. But they all have a development process, and maybe they work together, maybe they're different, maybe they overlap.

But you've got these cycles where someone is creating something, a data engineer and a data scientist are collaborating. That gets deployed, and then it goes into production. And what we want to focus on in the development process is how these people work together, and how they can make sure that they don't step on each other, that they understand if I change something, if I'm a data engineer, did I break my data visualization? If I'm a data scientist, how am I adding value? And how do you just have less meetings and less bureaucracy?

And so all these things are what we're going to talk about, and really the steps to do DataOps are about automating these cycles. Automating the production process, automating the deployment process, adding automation to the development process so that you can lower errors and decrease cycle time and increase collaboration, and also just measure your process.

And we're going to talk about what that means, because these cycles are actually worthy of measurement. These cycles that happen in your organization, and you're doing these now, actually throw off data, and that data is very useful for you to help your transformation. So what exactly do I mean here? What is transformation? So what is that transformation?

Well, it requires kind of focus in four key areas, as I said, error rates, cycle time, collaboration, and process. And if you think of it, how well are you doing those now? Is your cycle time months or weeks to get something into production? Every day, are you having problems, or every week, are you having major problems?

Do you even know if you're having problems? Are you having the Hatfields and McCoy between your data scientist and data engineer, or all the people doing self-service analytics rolling their eyes at your data warehouse team? And are you actually measuring your process at all to find out if there's any problems? And so, we think that the reason most companies are at this low level, the result of them being at the low level is that you have actually higher costs and more unhappy customers.

And so how do you get from that situation where there's lots of errors and you're moving slow, how do you sort of take that graphic equalizer and push up on all those four bars? And so you can get to a level that's more like this, where you have less

00:10:00

costs, where people are more productive, and you have less happy customers. And that's, in some ways, the metaphor of transformation in that there aren't these trade-offs. And a lot of people who've been in the data and analytic field for a while feel that, yeah, if you go faster, you're going to break stuff. And so you can only push one of these up at a time. All the other ones, you can't.

And so, the idea of transformation is that it is possible to move all these things up at once. And in fact, best-in-class organizations do all these things well together, and that's how they achieve great results. So let's talk about what the keys are to doing transformation. So

the first part is that we would like you to think, in the spirit of Agile, in increments. And so avoid big bang approaches. Don't do a DataOps transformation project that takes a year to do to get out any value to your customer. Are there cases where you can demonstrate value of DataOps, where you can demonstrate changes to those cycle time error rates in the near term? And then can you iterate and improve upon it?

And so apply the spirit of Agile to your DataOps transformation. And if you can prove success, that can help generate change. Because iin some ways, DataOps is a bit of a religion, and it's a bit of a process change. And that means you have to not ignore the soft things, your social proof, how people push back, how sometimes people are defensive saying, "Oh, well, we do this already." And yeah, you may do it, but there are ways that you can do it better.

And that means that you've got to have some staff time focused on this transformation. And measurement is key, and if you can measure your process, you can argue, we've made x percentage improvement in our delivery time, x percentage improvement in our error rates. The amount of collaboration has gone up y percent. And those things can be arguments for doing it more.

And so avoid a big bang, remember the soft things, because it is about people. We're not just talking about plugging in another database that has an ODBC connection that just has to be faster. We are talking in some ways about changing the way your teams work. And so that means that there's some social aspect to it and measure the benefits. And so what are the six steps?

And so we've laid out, in working with customers to help them on this, we've laid out six steps that you can work. And so I'm going to walk through these steps right now, and then we're going to go into detail on them by groups of three. So the first part is that, I think realize that this idea of DataOps is a new idea, and you may have spent time reading about it or learning about it, but the person who was on your team next to you may know nothing about it.

And it's not common practice. So I think there's a lot of ways to help educate your team and to get the idea, the potential of DataOps across to them. We've written books and articles, there's analysts out there, and some customers, we've done a free one-hour Art of the Possible with DataOps to talk about why, in concept, why would you want to do DataOps?

And I think there's a copy of that video on our website if you're interested. And so at some point, there's got to be a group of people who start believing that this potential of DataOps to help. And then the next step is to say, well, can we find some first step? Can we take that first step to have value, and can we find a project?

And sometimes those discussions, I've found, are easy because people are open to change. They're like, "This DataOps thing sounds great. I don't like getting nasty emails every morning. Give me some help." And other organizations are more defensive. They're like, "Oh, we do this already. We have it covered." And maybe there's a little bit more discussions and therapy to help work through and get at the potential.

Because running a data and analytics organization now, it's hard. And a lot of people have a lot of stress from it, and there are organizations who respond to that by being defensive. But there are opportunities for improvement, and so seeing it as it's not you're dumb, you're not doing DataOps, it's DataOps in degree.

And so we're going to talk about something called a maturity model that may be helpful or working with organizations. And then the next step is how do you get a community of interest? How do you get people in your organization to start being that group that starts talking it up? And this is, is there a wiki or a website? And then can you start demonstrating this? Can you start working with more pilots, more ways, more expansion of DataOps step by step?

00:15:00

Again, not a big bang, but sort of the ground game at football. Instead of throwing an 80-yard pass, you're trying to get two to three yards over every run. And then iterate, keep taking those cases, keep improving. And then finally, how do you actually set up a system to expand? How do you institutionalize the change to DataOps?

And we're going to talk about the parallels to how companies have done it in manufacturing and in software, and how having teams dedicated to this to help organizations change. And we're going to talk about something called a dojo. So these questions on how to do DataOps transformation, how do I spread the word? How do I get executive sponsorship?

How do I formulate a long-term corporate strategy? And how do we assess organizational readiness and provide a roadmap? And so these are the things I'm going to talk about next. So the first is there's just a lot of stuff out there. As I said, there's a lot of research from analysts on DataOps. We've got our 18-point manifesto and 200-page book, and for DevOps people, we've got a section from The Unicorn Project.

So there's a lot of resources out there that we can help you with just to spread the word. And we've talked quite a bit about there, and there's more on our website, and there's more on other people's website. But I think there's a lot of people talking about DataOps, and even if you just Google search the term, the news articles, everyone's talking about DataOps nowadays, and that's good.

It's nice to see, considering four years ago, most people thought we were crazy when we were talking about it. And so the next step is how do you get-- If you're not an executive yourself, how do you get executive sponsorship on doing this? And so, I think the first thing is kind of start with your business customers or the problems that they notice.

Do they mistrust the data? Do they have a layer of consultants or kind of shadow teams doing the work on top of the data? Is your backlog of requests really long? Are you getting them rolling their eyes? And try to understand your customer. And a lot of times when I ran data and analytics teams, I sort of wanted to avoid the response from the customer. And it's sort of, you got to kind of run towards the fire and find out what the problems are. And because a lot of teams are not meeting their customers' needs, and either from mistrust or speed or the fact that the executives hire other people to do the work.

And there's huge companies out there that like Opera Solutions, Palantir, who just take data and do work. And then the next step is to sort of map the journey that data takes from source to value that a person's looking, and look for problems. Where are the problems? And where are the sources of friction?

Is your data warehouse working great, your data lake working great, but the teams that you've enabled with self-service having problems with the data? Is it the data warehouse having a problem? What team is it? Are the data scientists getting what they need? And how fast are you responding to problems, and what does it take to go in and look? And so when you have Go back and look at the top three problems you had in the last six months.

Look at who responded to them, how fast the response was, what the cause was. And when you start looking at that from a perspective, you start thinking about, "What's my downtime on data?" Not just my error rates, but how often is the data not available or wrong, and people have been taking a hard look at it.

So mapping that journey from data to value, I think is important in looking for problems. And then also look at the process that you have. How fast-- How long does it take you from when a data scientist creates an idea to getting it in production? Is that months or weeks? Why is that? How long does it take to create a development environment?

How up-to-date is it? And how well-governed and secure is that and to support self-service users or other users? There's a bunch of questions that you can start asking to start finding where there's problems and then taking that and kind of presenting it in more of a strategy. And I think we've seen kind of two artifacts. So one is, if you're an analytic leader saying, "I want to do the DataOps strategy.

Here's what it means." And it's sort of a strategy deck that a more senior leader is saying, "Okay, we're going to change the way that we work." And that often includes the agile way of working, if you don't have that already. How we're going to run our teams. Are we going to do Scrum or scale Agile? And we'll talk a bit about that.

And then what's the operating model? And then I think a second artifact we've seen is, well, what's the technology to support that? What does this mean practically on how we're going to do it? And so trying to get executive buy-in, you have to speak in the language of executives, which means you've got to deliver fancy PowerPoints.

And I think these are good ways to sort of frame the activity first and get the corporate strategy, do some research, understand where the problems are.

00:20:00

Almost voice of the customer research on the last side, and then building a presentation that says, "This is why we should do it." And then I think the question is how

ready are you? What's the roadmap to get to DataOps? And so, what we've developed, and there's a white paper on our website, is what we call a maturity model, which is a way of judging how far you are on that journey. And so, as an engineer, I tend to roll my eyes a bit at maturity models, but as a manager, I think they are useful. And so what we've done is try to think about how to judge an organization's maturity.

And it's hard to do this because there's often more than one analytic organization in a company. There's different teams in different parts of the company. But this can be done, and you can look at it on a team basis or an aggregate. And so the things that we look at to judge maturity, are how many errors are you having?

Data problems, processing problems, it's late, customer mistrust. How fast can you change things? What's the cycle time? How challenging is it to collaborate? Your data science, data engineering teams work, even a data engineer to a data engineer, how are they working now, even with COVID? Is it going better? Is it harder since people aren't next to each other to talk?

And then how well are you measuring your process? Not are you measuring whether your customers are satisfied, are you measuring your team productivity, are you measuring production problems? And then two that are a bit fuzzier, which is your culture. How value-driven is your team? Are they really focused on delivering value to your customer? Does that get them excited?

Or are they more focused on following your development process and creating the steps and artifacts that go and that they feel if they follow the process, it doesn't matter if the customer got value or not. And then what do your customers think? Are they happy? Do they want more insight? Do they trust your team to get the data and insights that they need? And so sometimes they're hard questions, but I think this is the way that we think about how to judge maturity. And I'm just going to go through the first four categories and sort of break it down a bit.

And the maturity models are, to a certain degree, fuzzy, right? They're not hard categories, and people are answering them verbally because a lot of times they don't have the data at hand. But the way we think about it is sort of level one to level five. And kind of the bookends on this, it's like if you're at level five, you're running production and there's no major production errors.

Things are running great. The data's right. It's never late. And you've got a lot of automated tests running in production, so you've proved that the data and the artifacts you create from the data are right across all your tools and pipelines and sources, and you send alerts. And then you're actually digging into the data in lots of interesting ways.

You're not just looking at line counts. You're looking at statistical ways, comparisons from last time. You're actually really digging into the data to see if it's right. And then level one tends to be the opposite, right? They're having production issues, they just don't know how many. Customers are finding problems in the data. And if there is problems, they don't get any alerts.

And in general, it's sort of they're in the land of hope. Data comes in, I hope it works, and I hope they like it. And you can judge where you are on a team, and there's questions in between. And the second is more around the cycle time, and the best companies are able to kind of deploy new data or new analytic code into production in minutes.

And they do it in a way where it's error-free and automated, and that they apply a whole set of tests, unit regression, functional integration, and most importantly, they take representative data from production and they pass it through, so they get a really very realistic view of how things work. And they can actually spin up a new development environment in minutes because being able to deploy quickly also means that you can patch quickly, and sometimes you have to patch your process based on some data provider forgot a column and you need to patch it.

And other companies struggle, right? It takes them months to get something in production, and the deployment process is either undocumented or manual or some combination, and it's really done with manual testing or hope that it's right. And the representative test data is not there, or it's three to six months old. Or even the worst is just development's done in production.

You're like, "Okay, I'm going to go do it, and I'm going to go change the SQL statement, and oh yeah, I'm going to do a rollback now." And that I think certainly is rapid, but it has a lot of risk.

00:25:00

And then well-coordinated. So this is talking about how teams work, and you could say the coordination happens between two people who are working on the same team or people who are suppliers and producers, a data engineering team and a data science team, or a data engineering team and self-service users. And so, I think there's different ways to think about this.

And so one case is, do you know what you're working on? Do you have all that work in version control in the same place? And do you have visibility into other people's work? And can you actually help collaborate on it, do things like code reviews or automated tests? And the same thing, does production have a view of the process that's coming, or the process that's in? And so it's a lot about getting visibility because your customer sees the end result.

You may have your Power BI workbook done by self-service users, and then your data science team happens to be using the latest Python libraries, and your data engineering team has just got a really cool data lake up or whatever the latest term is, and a problem goes on and, and you're not able to see across that. And then the finger-pointing starts and how do you get-- that's really about end-to-end visibility across the value chain. And a lot of times organizations are like this. The production releases are either infrequent or unreliable.

You only coordinate when customers yell because you just talk to your other data engineers, you have no idea if you make a change and it works. And there's a lot of war that happens. The centralized team and the self-service team, they just don't like each other, and they roll their eyes and people think everyone else is an idiot, and there's fiefdoms that go on. And I think that struggle, it just doesn't have to be that way. And, I think applying these DataOps principles can happen. And then the last thing I want to go into is measurement.

And we're going to talk a little bit more about measurement, but the idea here is how are you measuring your errors? Are you measuring how fast you can deploy? Are you looking at the amount of check-ins they do, the code configure it, the code metrics? And do you have dashboards that you're looking at yourself to measure, or do you have none of those?

And lastly, a maturity model is not there to blame people. It's not a shame and blame game. It's used to provoke a discussion. And where you are, it's good to know, because that gives you the opportunity to improve. And it's not, "You suck, here's the maturity model." It's really about provoking a discussion, because this is really an improvement activity, not an assignment of blame activity.

And so the second half of the DataOps transformation is how do you get this going? How do you find your use case? And once you get it, how do you start the ball rolling? How do you organize your teams and your staff? How do you structure a center of excellence or a dojo? And finally, how do you measure this?

And so there's lots of ways that people have talked about organizing agile teams. And the purpose of it is to have people work delivering things better and sooner and safer and happier. And there's techniques like Scrum and Kanban to manage an individual team. There's ways to manage teams of teams, like scaled agile or the Spotify model.

And there's lots of books and excellent articles on this and consulting teams who can help you organize your process. And then there's an emerging role of the data product manager who kind of acts as a proxy to the product owner, and kind of says, "This is what the product owner wants or not." And I think that also can be in the agile framework, a product owner is someone who comes in and is what the agile team works with. And remember, the challenge here is with these teams, and if you really believe in agile, and most agile organizations do, it is about giving these teams that are complete autonomous teams, and they're empowered to make changes, and in fact, they're empowered to make changes at their own control.

And that sort of autonomy also creates some risk. And that risk in the organization that people can make mistakes can be mitigated by a DataOps approach and the sort of testing and automation that goes into it.

And then the other question I hear quite a bit is, do I need to staff DataOps engineers? And so I think the question is, who's going to do the DataOps activities and who's doing it now? And a lot of the activities in these process cycles of orchestration and testing and monitoring production or automating the deployment from dev to production or the creation of environments or inter-team collaboration, they're done.

They're just done manually or they're done in people's spare time,

00:30:00

and they're seen as kind of not important. It's sort of like the footer on your Word document. It's like, eh, no, we only care about the Word document, not the footer. But if you look at what happened in software, those kind of operational processes actually gained a lot of credibility, and the role of a DevOps engineer actually became paramount because you want to spend time building your factory, not just building cars.

And that role of a DataOps engineer or maybe a data scientist or a data engineer fulfilling that role, I think is really important. And I think it has an opportunity for people to have kind of high visibility in an organization. So we think it's important to have people who do the DataOps engineer role, and more specifically, to spend the effort To think of 15% of your time working on the factory as opposed to 2% of your time or 3% of your time. And if you work on the factory, you're going to be able to produce better things faster.

And so once you've got it going, you've got it staffed, you've got a couple of demonstration projects going. They're in production. You want to kick this off, and maybe you have a team of 10 people, or maybe you have a team-- and in a bigger company, maybe you've got hundreds of people and multiple groups in the organization doing analytics.

And how do you actually make this change? And so, there's a couple of aspects that companies have done. And so, one way is to look at is from, okay, there's a set of shared tools that enable this, technical services, and that could be things like source control or bug tracking, project management systems like Jira, DataOps platforms like DataKitchen, that can be part of that. And then there could be sort of a team that actually helps organizations adopt these ideas.

And there's two ways of doing that, a COE, a center of excellence, and a dojo, and I'm going to talk about both models. So the first is that you've got a technical service. If you're going to make this change to working in an agile way, or if you're going to try to get people to automate deployment, do monitoring and testing, having some tools to help them.

So having a shared Jira instance, for instance, having

a DataOps platform can help. And this is very similar to what happened in companies who are trying to get their software teams to do DevOps, and they'd have a central CI and CD pipeline service. And that's good, right? Having shared tools is good. It saves expense, creates common standards. But that's not enough. You've got to talk about how to get a team to do it.

And there's sort of two models. One is a center of excellence that actually, where they do projects. They come in and they're the center of excellence is sort of a group that actually works in a DataOps way and does data and analytic projects. And I think that's good. And a lot of companies have done center of excellences around data lakes, and they're trying to get the organization to do it.

And really, it's sort of getting small wins working. But there's another way that I like, and it's called a dojo. And a dojo is really, think of it as a practice center, where you do practice. And they don't actually do projects. They invite people in from the organization to work for a short time to practice these skills, and then they can go back and apply those skills in their life. And like in Japan, it's a dojo. It's a safe environment where you can learn new skills.

And it's about gaining practical experience actually doing the work, but it's not work that is part of your day job. And so companies have, in the DevOps world, have really latched onto this in the past three or four years. So for instance, if you go to the Target dojo, dojo.target.com, there's actually a dojo consortium.

And this is all focused on transforming companies in their software and IT domain, not focused on analytics, but the principles apply. And there are a set of new skills and helping people go for four weeks, six weeks in this dojo to learn these new skills, get expert on it, and also kind of infect them a bit, help them get this mindset. Because really focusing on customer value as a focusing on your prospect is a mind change for some people. And it's harder in a lot of ways because you're given more autonomy and more responsibility to deliver value, but you've got to deliver value, and that can be scary.

But I think it's just a better way to work. And once you get in the habit and trying that out and trying the values that you do in a dojo, I think is good. And this is a small team that does it. And so lastly, how do I measure success? And so if we look at what we're trying to do with DataOps, there's sort of two aspects to it. One is your production.

And so if you want to have low errors, how do you measure errors?

00:35:00

And one way to look at it is looking at, do you have noisy data providers who are giving you crappy data? How much does that chart? How much does that cost you? Is the data I'm supposed to get today there? Because a lot of people may use syndicated data from IQVIA or Bloomberg. And then are your builds running? Has all the builds or systems running so your analyst can go look, and we've got things like a pulse report. Is everything right?

Or SLAs and timings. And then are your models still predictive right? And so there's ways to measure the system that's in production so it can tell you that, A, the data's right. A, you've run a whole set of production tests against it. You've been on time, and the data is good and right, and measure that and make sure across all your pipelines, you have a central place for people to go and get it.

And then that is really about what happens in production, and that's, I think, a good resource for people, and it's a good resource for you in transforming your organization because that data about errors and timing is a good source to show that DataOps is making your team awesome. And even then, some people have looked at utilization as well.

Are my charts and graphs being utilized by people? What's beneficial or not? And so measuring the production process and utilization, I think can also be helpful in this world. And then the second is, since we're also dealing with a development process change, how well is your team doing? What is their productivity? What is their amount?

Are they making the time on their project? And this is looking at it from kind of a very similar set of data, but pulling at it from a different dimension, the team that's working at it, the project that they're working on, how fast they're deploying, how well they're collaborating And there's things that come out of software and tools like Jira, like a burndown chart, or just even having a good list of tickets your team has worked on is actually really important and sometimes can be very helpful if you get an audit and just saying, "Here's what I worked on, and here's why I worked on it." I think just keeping track of very basic things can help you show that you're being successful in this transformation.

Because keeping track of tickets and the amount of work people do is a good way to measure their work, but it's also a good way to show your team has value and they've closed this many tickets and they've done this amount of work, and you can show that after six months or nine months, and show the increase in the amount of work that they've done.

And here's an example of these Tomato report and Pulse report that come out of our tools, and then the example project metrics that come out of our tools. And it's really about that idea. It's like on one hat, you're a manager, you're managing a team, you want them to do well, but on the other hat, you run a manufacturing line and you want that to work well, and measuring both those things. And also just the part of how do you get people in an organization who are doubters who've always worked this way, and it's so embedded in their mind that if you go fast, you break things.

And they say data's different, or data science is different, and you can't apply these agile methods because it doesn't apply. And I think one of the ways to get those doubters is to show them the data, is to show teams are successful doing it, and to actually start tracking their behavior and saying, "Well, have you written a whole bunch of automated tests? That's probably why you're having errors in production. Have you helped automate your process of deploying?" And data can help change people's behavior, in addition to make you as a leader look awesome and also help drive the transformation effort.

And so we're getting to the conclusion here, and I want to talk about a few things. So the first is that DataKitchen's a software company, but we've realized that people need help in trying to frame up how you do DataOps. And so we have a set of consulting services that we've set up around how to help you develop a DataOps strategy.

What are the steps that you can do to bring DataOps to your organization? And part of that is also a technical review on what tools you have, sometimes looking at how you want to do deployment and testing and version control, looking at your architecture. Those things can also help. And then, of course, doing this maturity model across your team and trying to adapt it to your environment and get the results.

And also helping you do a dojo and setting up a dojo and bringing staff in and doing training. And then also, second to last, developing metrics. And a lot of times, these metrics, if you want to get a 360 to review your team and project, it does mean doing work and integrating data from perhaps Jira or ServiceNow,

00:40:00

perhaps from a technology platform like DataKitchen, into a repository where you can actually see across. And so if you believe in data and analytics, then you should be pretty analytic about how you produce data and analytics. And so that can be part of the transformation services. And then finally, just helping your team be agile, and there's a lot of people who do this, and we can help, but how do you get people to start thinking in value and how to write a good ticket, how to size a good ticket?

And we've done sort of work over time with teams, a couple of hours every few weeks where we coach them, and it's more of a coaching service than a training service. And in some cases, the people on the teams, some people were very happy to write tickets, some people wouldn't write tickets, or they were always late because they thought it was dumb work, and trying to get people to think about how they are arranging their work and communicating it and the value of having a good queue of tickets as a way to negotiate and work with your end customers, I think is a good thing. And so how you end up giving sort of high-quality business value quickly. And so these are some examples of the kind of consulting services. And we're really doing this because we want people, of course, we're a software vendor and we want people to buy our software, to be blunt. But we also think that it's necessary to help people understand where DataOps is. And

we think that the benefits of DataOps, the lowering error rates, the cycle time, improving collaboration and productivity measurements can come when you've figured out a plan and when that plan can be best executed in some software. And so that's it in terms of my presentation, and I promised to be done at 1:45, and I made the time.

And so I know there's lots of questions, Beth. Is there any that you see that I can answer right away? Yes. Thanks, Chris. Yeah. Just encourage everyone to add their questions into the question box, and we'll go through as many as we can in the next few minutes. Here's a good one to start.

Are there any research reports on DataOps tools in the market?

Not any good ones, I guess, is the blunt term. Because there's a little bit of fud of what's in the market, and so I know Eckerson has done some good work, but there's not a magic quadrant or an RFP template.

So I guess the answer really is no, of a good understanding or comparison of the tools that are in the market. Not that I know of. Okay, great. Well, maybe that'll be coming soon as this market develops. So the next question is, how much is test-driven development important to DataOps? Can I leverage any of the DataOps principles into an MLOps framework?

Yeah. So there's two questions. So test-driven development means that you should work on your test at the same time you're working on the new thing that you're working on. And so there's sort of a doctrinaire test-driven development, means you write the test first, then you do your work. And I tend to be a little less doctrinaire on it.

As long as you are doing your work, and sometime during doing your work, you've written a test. I'm more like test next to development. And so, I tend to believe that at least some of the automated testing should be done by your data engineers. They should be doing production level testing. They should be working on development testing.

As well as there should be dedicated testers who look at the whole system end-to-end to see. And so test-driven development's really important, but not in a doctrinaire way. And then the question of DataOps and MLOps, and there's a lot of ops words out there, and I guess our feeling is that whether you call it DataOps or MLOps or Analytic Ops or Data Health Ops, these ops principles that we're putting under the generic term DataOps apply.

And these cycles that you're trying to work on, whether it's-- If I, let me go back to that cycle diagram.

They're really invariant of whether you're doing a data science code or a visualization or governance or data transformation or access. We all are processing, we all have tools that are driven by code that are acting on data, and some are a little bit more complicated, and yeah, the data science has got a slightly different workflow than a BI tool, which is a slightly different workflow from a data tool.

But I think the principles apply. And by focusing on these cycles, it really doesn't matter in my mind what you call it. We use the term DataOps across all these areas because I think looking at it end-to-end is the best. But the market is talking about Model Ops and MLOps as separate things.

00:45:00

But to me, I'm more focused on the sort of arguing and thinking from first principles and focusing on these process cycles, I think is the first principle that you need to focus on in adopting DataOps.

Okay, great. Thanks. In an ideal DataOps world, who should be on the DataOps team in terms of a mix of IT and business people?

Well, I think, in general,

Amazon's got this term two-pizza team. The team is sized, so it can have two pizzas feed everyone for lunch. And the idea is also to give, if you believe in the agile methodology that they're capable of doing all the work. So they'd have a data engineer and a data scientist and someone who does data visualization. They'd have a product owner who represented the business.

Maybe that's a data product manager or literally is a business person part-time. They'd have a part-time Scrum Master to help them, and maybe they'd have some part-time interactions with people who do

data architecture or other horizontal activities. And they would also have a DataOps engineer on, either full or part-time, to help build the system that enables them to deliver quickly. And so having autonomous teams that have representation of each discipline is, I think, the ideal structure. And of course, these teams, sometimes the work can be more data science than data visualization. Sometimes these people don't all work in the same department, and so there's some complications there.

But in general, the idea of a two-person team who can accomplish all the goals and focus solely on getting value to your customer and do the work and own the process, but also develop the tests and the automation, so that they don't create a lot of technical debt, that they don't create an ungoverned mess, and that they don't create a lot of chaos in your organization is essential.

So to me, it feels like if you're going to be agile, you need to have a framework, a technology framework that enables you to be agile or else you just are taking too much risk. And it has a potential to have too many problems. And that framework of DataOps focusing and automating and testing these processes is what enables those two-pizza teams to really work with maximum autonomy and velocity and quality with control.

Okay. Great. Thanks, Chris. How can we integrate data governance?

Well, yeah, actually, we had a very surprisingly popular DataGovOps presentation, and we just released the white paper, too. And in some ways, there's multiple aspects to data governance. And so the way that DataOps touches it is think of the data governance that you do as code. And what do I mean by that? Well, if I'm going to add a table into my database and join that table with another table, I've got a schema definition and some changes to a schema, and that may end up in a model and may end up in a report.

And there's two artifacts that most people do in data governance. One is my data dictionary, and you want to update that data dictionary as you update the schema. And then the other one is trying to look at what the sort of lineages of it. And I think DataOps can help with those because I think DataOps is the process lineage across the end-to-end, and we can also deploy as code the changes to the data dictionary along with the actual changes to the schema.

And so if you think of data governance as a code-driven activity, not as something that's done manually, and of course there's a lot of other things that data governance do. And really it's about managing this process, these cycles. And so there is a deployment of what data governance creates, and that's the metadata about schemas and data dictionaries and data lineage, and DataOps can help create that and deploy that.

Okay, great. So there are two related questions here, so I'll group them together. So you refer to this as an enterprise approach. Does this apply to smaller enterprises, so for example, less than a billion dollars in revenues? And then a similar question is, would the approach differ depending on how big the enterprise is?

That's a really good question. So, I think it's almost easier the smaller team that you have, right? If you've got 20 people who are doing data and analytics, you can start to make these changes much quicker, and maybe you don't need a dojo. Maybe you can start doing things in a more ad hoc manner.

Maybe you don't need all the reports that go with it. Maybe just having a spreadsheet to track your errors is good enough. You still need automated testing and automated deployment.

00:50:00

Those things apply, and the cultural change, I think, gets easier because you know people, and you can talk to them. In bigger organizations, people are busy and their calendar is filled up, and you've got a layer of indirection that you've got to go through. So I think it's easier on smaller teams, especially if you get people in the team sort of bought in and they can see over lunch, that this other group that's doing DataOps is having a lot of success, and who doesn't want to have success?

And then in larger, the billion dollars, it's really the size of the data and analytics function, right? If you've got 10 or 20 people in your analytics function, it's different than if you've got 400 people

across three lines of business across three continents. And so the complexity of a global multinational company and the ways in which they've tried to

reduce costs and spread best practices around the supporting organizations, like a data and analytics, make the enterprise, and that's been our experience with larger enterprise customers who are multinational corporations or have very large data and analytics functions. And that's sort of the perspective that we took here, and that's why I use the word enterprise.

And it does mean, generally bigger data and analytic teams this is opposed to. But the principles that I talk about, I think, apply. I think they just can be done easier and more quickly with smaller teams.

Great. Thanks, Chris. Are you ready for more questions? Because we've got quite a few more. I am. Okay. What is the timeframe we should look at when we assess our data maturity?

Oh, like how long does a DataOps maturity model take? Is that the question? I'm not sure if it's that or what timeframe do you look at when you're assessing these different elements of your maturity? What we tend to look at now, like the point in time is let's assess now or what's happened in the last three to six months, not over the last three years.

The maturity model itself can take, it depends on how many people want to be interviewed and how much data you have about these things on already. But the maturity model takes, you can sit down and it's quick, it's like 50, 70 questions, so you could fill it out yourself, and we've got surveys that do that, or the interview process can take a matter of weeks.

And we're not really assessing your data maturity or your data engineering maturity or any of the other functions that you do. There's plenty of maturity models out there to talk about your data governance maturity or your data management maturity, and we're really focused on the cycle times and we're on the DataOps maturity, and that's where we focus.

So we don't talk about what type of database you have or how you're doing your ETL process or how you're doing self-service. There's plenty of other vendors who can help you with that, and there's a whole host of data management, data science, data governance surveys out there, maturity models that you can choose from.

We're focused just on the DataOps part. Okay, great. Would you recommend a tiger team as an alternative to a center of excellence or a dojo to demonstrate the value of DataOps?

Part of me wants to answer, hey, if you've got any team of people focused on DataOps, awesome.

And so you call it a tiger team, which tends to be focused and short-term. I think it's good, and I think what really is about how you get a group of people to demonstrate that working in iterations, that building the infrastructure to support iterations and testing is valuable. And whatever way you get that done, tiger team, COE, pilots, is good, because once you see the value and you can demonstrate the value with actual numbers and actual reports of what your customers are getting and making your customers happy, that's the real coin of the realm, I think, is demonstrating success and value to your customers.

And once you have that, then how the best way to have organizational change, whether it's continuation of the tiger team, COE, dojo, I think is a discussion. But really, if you can convince people and start demonstrating success, and that is going to enable you to launch it farther.

Okay. How do we start with an agile approach when we are embedding data analytics into a process?

Yeah, I'm not sure I quite get that approach. So if you're embedding data and analytics, sometimes data and analytics ends up in, if the process is a software process where your data and analytics end up being part of

an app that you deliver to your customers.

00:55:00

And then the question then becomes how do you map a DataOps process with your DevOps process, if you have one? The other case is, how do you adapt DataOps to your existing data and analytics deployment process? And I think that's- If that's what the person asks by process, that becomes kind of a contingent question about how you would map the principles of Agile and what forms of Agile you would take and how to make that work with your team.

Okay. Could you elaborate on applying lean principles and statistical process control? I can understand how this applies to process measurement, like dev deploy processes, but is this also applied to DQ metrics? Yeah, I think it actually applies more to data quality metrics, because if you think about it, the units that you put in a manufacturing process like, the amount of tires or engine blocks you get when you're constructing a car and whether those are good or whether which supplier is giving you the good ones, it's very similar to data, right? You're getting data from different suppliers.

You get 100 rows one day, 100 rows the next, and then suddenly you have 1,000 rows and you have a process control break. And so I think the ideas of statistical process control apply very well to the production process of data and analytics because, in a lot of cases, we are having our customers look, at least in their mind, they're looking longitudinally at KPIs that they have in their head.

Number of customers, total sales in my region, looking at my sales bonus. And when it breaks bounds in their mind, they notice, and they notice super quick. And so you should have those tests embedded in your software so you notice the problem before they do. And maybe it's not actually a problem. Maybe it's that sales guy isn't going to make their bonus this year.

And it's good to release note it. That at least saves you the time of doing the deep dive and spending the next two hours digging in the data just to tell them, "Yeah, you're not going to get your bonus." And so I think the ideas of statistical process control or SPC apply really well to looking at the flow of data through your production process.

Okay. This one's more of a comment. Maybe you can comment on this, Chris. Your framework does not involve end user teams and their culture in terms of how to interact with technical teams and how to remove the Excel approach using analytics more as a data extraction tool.

I'm not sure I quite under-- Different organizations have different ways that they do analytics, and so some of them have a centralized data team, and then they have people who are customers of that data team and access, they're more of a data enablement function. And then people use Excel or use Tableau or Looker to extract data and do the work.

And our vision of DataOps is that those people who are doing Excel work, or doing basically what I consider low-code development work, that that should be as part of a DataOps process because they're building if-then else's the same way that a data engineer is building an if-then else. And so it needs to be tested and it's source control deployed, and they are part of this process, whether that tool happens to have a nice cool UI or even Excel, or whether they're using a very technical Scala and writing Spark jobs.

And so code is code, and it needs to be put in version control and tested and monitored and deployed. Okay, great. So we're up at the hour. I have two really easy questions for you, Chris, to end this. One, is DataKitchen pursuing federal government contracts? Yes. Okay. And two is how can I see a demo of the DataKitchen platform?

Go to the DataKitchen website and click, "I want to see a demo" and put your email address in and a guy named Nick will follow up that day. Great. Or else we'll get on Nick if he takes more than 24 hours. Awesome. All right. Well, we don't want to go over the hour, so I want to thank everyone for taking the time to join us today.

Hope you found this presentation really useful. Thank you so much, Chris, as always, doing a great job explaining everything. We will send out the recording and the slides to everyone within the next 24 hours, so be on the lookout for that in your email. If I didn't get to your question, I apologize, or if you have any additional questions, please don't hesitate to just reach out to Chris or I at DataKitchen directly, and if we didn't get to your question, we'll follow up with you directly.

So thanks everyone again, and have a great afternoon and evening.

Thank you. Bye.