DataOps Therapy with DataKitchen's Founders

00:00:00

Welcome, and thanks for joining us today. My name is Beth Pfefferle, and I'm the VP of marketing at DataKitchen, and I'll be the moderator for the discussion today with the three DataKitchen founders, Chris Bergh, Gil Ben-Gayot, and Eric Estabrooks. And today, they're going to provide some DataOps therapy. So we've been talking about doing this for a long time, so I'm excited that it is finally happening today.

So before we start, just a few housekeeping items. The webinar is being recorded, and that recording will be shared with all registrants after the webinar, so be on the lookout for that in your email. Also, if you have any questions during the webinar, just chat them in and we'll answer them at the end.

I highly encourage you to ask questions. We hope to get some good discussion going here today. And this is your chance to get some therapy, so definitely don't be shy. So with that, I'd like to kick it off and have Chris introduce the panelists. Chris, can you take it away? So welcome everybody, and thanks for that great introduction, Beth.

So, who are we, and why are we talking to you? So, actually I've worked now with Gil and Eric for over 15 years, and we all have kind of similar stories, and our similar stories are actually going to provide a background to the survey results that we took, and that we are going to share and use that as a forum to talk about what we're doing.

And so, I've worked with Eric. Actually, we all met at this company that did analytics for healthcare, and we all joined when it was really small, and we all kind of experienced the same problem because think of us as the operational team who had to make the data get out, had to make the insights come. And so, we learned a lot of lessons from that.

And so, both Eric and Gil and I kind of have similar backgrounds in that we spent a lot of time in the software industry building software, and then about the same time, about 2005, we joined this company that did data and analytics, and we all shared the same kind of pain. And then when that company was sold, we decided to found DataKitchen.

And as part of that, we realized that we weren't ever willing to go back to the way it was before. We weren't ever willing to give data or reports or anything to the customer where we didn't know it was right, where it wasn't tested. We wanted to work fast and have automation. And all three of us have a profound idea that we should make our customers very successful, so we're quite alike in that, in addition to being sort of white, middle-aged, data nerds. So, I'll give a chance for Gil to talk and then for Eric to talk. So Gil, you want to say hi?

Hey everybody. It's great to be here.

And Eric, you want to say hi?

Hey, everybody. Eric Estabrooks. Glad you could make it today. Yeah, and both of them actually today run... Gil and Eric both actually work directly with our customers. Gil's in charge of our product and helps run our product implementation team. Eric runs our managed service team, so in addition to this sort of shared experience, they're day in and day out interacting with our customers and kind of seeing the pain that our product solves and helping to understand where they came from.

So, thank you guys for joining me, and hopefully I'm not going to be speaking very much and you're going to hear Eric and Gil talk a lot more than me, so I'm looking forward to that. Keep our fingers crossed. Thanks, Chris. Welcome to all of you. We're excited to have you here. So, just to give some more context, why are we even doing this webinar?

So as Chris alluded to, something is amiss in data and analytics, right? So, the data proves this out. So according to Gartner, 85% of big data projects fail altogether. More recent data from New Vantage Partners show that the number of data-driven organizations has actually declined from 37% to 24% over the last several years, so that's a pretty startling number.

And then the CDL role, which is a fairly new role, the average tenure is only two and a half years. So, all of these facts together definitely support the story that something's broken in data and analytics. But among all this, one area that hasn't really been studied is the data engineering role. So we felt it would be interesting to look at how data engineers are doing under these circumstances, right? Are they thriving, or are they feeling the impact of failed data projects? So we surveyed 600 data engineers, including 100 managers, to

00:05:00

understand how they're faring and feeling about the work that they're doing. So, the top-line result was that data engineers are suffering, and in fact, 97% of data engineers are feeling burnout, so I think that's a pretty startling number. Probably higher than I expected. I expected it to be high, but not quite that high.

And then in addition to that, 78%, or a majority of data engineers, are so stressed out that they wish their job came with a therapist. So before we jump into the reasons why on this, what do you guys think about this? Is this surprising to you? Do these numbers make sense to you?

Yes, it does. I got to say, the 97% burnt out, that one's a little surprising. I guess my own team can't be a simple sip of the world, but I don't feel we're that high. Well, good, you're doing DataOps, so

Oh, yeah. I did not put two and two together. That's sort of the whole point we're here today. Right. Okay. Yeah, I think that the challenge is a lot of data engineers are kind of caught between a rock and a hard place, right? Their data providers don't care that they exist and give them crappy data, and their customers sort of want things fast and want things perfect.

And I

think the common way that data engineers interpret it, and in fact, this is kind of a theme of Eric and Gil and I, is they think it's their fault. Like, it's their fault that the data providers are giving them bad things. It's their fault that they can't go fast enough. And that actually is very stressful when you internalize a systematic problem into a personal problem.

And so, in some ways, the genesis of DataKitchen is to realize that the problem isn't a personal problem, that it's a problem in the system that you work in, and that you can actually affect change of that system. And Eric and Gil and I have spent years trying to think about what kind of system we want to work in, what kind of processes, how does that work?

How can you live when your data providers are going to give you crappy data? That's never going to go away. Your customers are always going to want more. That is never going to go away. And there's always going to be some kind of new tool or tech out there that you're going to want to try and play. And let's say that that's the world, and let's say that that world is right. So how do you live in that?

And how do you not be stressed and burnt out, and how do you thrive? And I think that's what we want to talk about, is to sort of change the assumptions here, and that's what I have to say. Okay, great. Thanks, Chris and Eric. So we also asked the data engineers why they're feeling so burned out, and so here are some of the top reasons we'll go through.

And love to get all of your thoughts on whether you agree with this and your thoughts on if you experienced it, and some solutions. So reason number one, 50% of respondents feel burned out because of the relentless flow of errors. Does that resonate with your experience, Eric? Yeah, it does. I think early on when I shifted my focus from software development to the data side of things, I think one of the first things I noticed is that some of the data engineers I'd inherited were just constantly struggling with errors, right?

And a lot of times because of SLAs, you get the error, you want to just get it fixed and get it out the door and right, and you make the SLA, you're only a little bit late, and so you feel like you've accomplished something. And then you barely have time for a breather, and then the next set of data in or delivery comes and again, there's more errors. So you're back in that kind of endless cycle of just trying to fix errors as opposed to getting to the root of the problem.

There's lots of sources for it. There's lots of causes, tools, people, everything like that. Oddly enough, one of the biggest things we've found is just track everything. If it's your error, if it's your software's error, if it's your data provider's error, just start tracking all of those relentlessly and be honest with yourself, as well as everyone who's kind of causing the errors. And once you start to do that, you start to find patterns. You may say, "Oh, this thing happens every two weeks, and it takes us eight hours to recover from it." Well, that might be a low-hanging fruit to fix. Just automate that, fix it. So it all starts with the tracking, and the tracking really is about getting you kind of focused on what to fix. And also know who you need to talk to.

00:10:00

So Chris mentioned could be the data providers, right? You could have one errant data provider that's causing most of the problems. We've also seen things where you've got this recency fallacy where you're like, "Oh, that cardinal is the worst specialty distributor, and their data's just bad." It's like, well, if you're tracking everything, you may see that really it's only been two weeks and they've had a clean record for the whole year, whereas three, four months back, you've got other people that are just nonstop errors. So you've really got to track so you have the data so that you can make good decisions on what to put your efforts to.

And almost always, those efforts end up being automation. So many errors are caused by people having a process, and there's a lot of manual steps in it. So you may find things where file names are slightly different, which breaks your automation. Maybe somebody's building a table for you, but they run it out of order, so all of a sudden the columns are in different orders.

It's all subtle things like that, and until you have some way to track everything that's going on, it's almost impossible to fix it and kind of automate and get yourself out of that hole. So start tracking. Do any of you have any particular war stories you can share and how you went about solving the problem?

Yeah, I think... Oh, go ahead, Gil. If I could interject, I think the worst is when your own customer finds an error and you don't, and that's the worst. So that was happening to us, and we realized, hey, if we can add data tests, we could find the errors first and work on them, and that would at least take a lot of stress out of the situation.

Yeah, and there's nothing worse than having a data error, but having it exist for months. Yeah. And then have to talk to your customer, say, "I'm sorry, the data's been wrong for the last three months," and then say, "Well, which exact data was wrong, and which should I trust?" And, "Don't you guys know what you're doing?" And the sort of shame, and they're like, "Oh." It's such an awful conversation to have to To go in.

And there were days at our previous company where I would dread going into work, and I wouldn't actually check email before work because I just would be bracing myself in the drive, like, "Oh, what went wrong that night?" And this sort of morning dread, and I'd get in and go, "Oh, nothing went wrong." I'd be happy.

And so I don't think you really have to live in that world, right? You're not an awesome person if you live in the world where you're stressed out and things are breaking and you're expecting to get yelled at. That's not an optimal way to do it. There's a better way to think about things.

Yeah. And sometimes, it's interesting. You can tell which suppliers are giving you data through a manual process and which are giving you ones through an automated process. Generally, the manual ones, like Eric says, the file names change, the columns order change, and if you can detect, if you say, "This is a troublesome supplier," talking to them. You could just deal with the errors, or you could call up and say, "Hey, here's a spec.

Could you please follow this?" And that's one way to make it a bit better.

Yeah, on the supplier thing, so some organizations that supply you data have well-controlled processes, and really that's all about consistency, right? Others just don't have it, and they're not organizationally set up. Like if you're dealing with digital marketing data, right? Digital marketing agencies, unless they're pulling right from Google Analytics or something like that, if they're doing their own kind of data aggregation, it's almost always messy, especially the smaller shops. In those cases, you've got to assume they're going to give you bad data more frequently than everyone else, and you just got to kind of put your testing of their data sooner in your process and just build it in. Because catching your errors sooner than later is a good way to head it off. You're not going to magically update their organization to have well-controlled data processes, but you can put a lot of tests in to protect yourselves from bad players, so...

Great. Thank you all, and since we're on the topic of errors, this question just came in, so I'm going to throw it in. So how do you distinguish between a data error and an outlier?

Gil, any thoughts on that one?

00:15:00

That's a good... A data error versus an outlier. Well, at face value, you'd have to dig into that, right? Because I could see an outlier. So I mentioned earlier tests. Tests are really important, and I always think tests are a gift that you give to your future self, right? Because they're going to find things for you.

And with an outlier, you may have a test that fails or has a warning, right? And there is a question, is it an outlier or not? And I think that's where you then need to do some investigation, I think, and make that determination. And Eric has an opinion on warnings that I've heard many times.

I kind of step back a bit to the outlier. So the one we use all the time is, we deal with a lot of companies with pharmaceutical data, and so it's all prescribing behavior. So there's seasonality to some drugs, right? You may have ADHD medication that's prescribed to schoolchildren, and so what happens during vacations or summer breaks, you may see a drop in the data. So I remember one customer, we had good testing on their data, but they were newer, so they'd only been with them for six or nine months. And so the summer lull came in, and a whole bunch of our tests went haywire.

So we spent a bunch of time internally seeing, did we drop something because it looks like there's half the data there. And once we had our ducks in a row on like, "Okay, here's the raw file, here's what we were given. We don't know what's going on," then we were able to escalate to the customer, and the customer then let us know, "Oh, here, that's seasonality.

Here's the behavior of the drug." So then that flowed back into our testing. We were like, "Okay, we expect a 5% increase a month, roughly, never a drop, but if we get to June and July, we can expect up to 15 or 20%." And so that's where the, again, the testing that we had in place stopped it from becoming something that went to the customer or even made it way through the process, but then we were able to improve our test as we went.

So yeah, I always bring that one up. All right. You guys ready to move on to the next one? Mm-hmm. Sure. Okay. So reason two, manual processes crowd out innovation. This was a problem for 50% of respondents. Eric, I think you have some good examples of this. Do you want to share them? Yeah.

Depending on how fast you need to get something up and running, depending on how much you want to invest in it, you may inevitably have some manual processes to start with. Now, I'm a big opponent of manual because, well, we'll get to that why. But there are reasons you have manual processes sometimes. You want to prove something out.

You want to get something into some customer's hands for review. So you may cut some steps, and you may have an end-to-end process, and there's five manual steps. But what you're doing is you're showing the customer, "Hey, here's what we're going to build for you." They give you the thumbs up. Now you've built up some debt there, and you need to go back and address those manual processes, because you get situations like, we had this release process that we never quite put the effort into.

So what it became was we had this one engineer He had a long checklist, and so every time there was a release, he'd go through the checklist, which sometimes was involved writing an email, moving a file to here or there, maybe running one little test. And so he had this long checklist, and it was going out to some Salesforce. So there was hundreds, I think it may have even been up in the thousands of people that were looking at this data.

A checklist day in, day out gets a little boring. Well, he got bored finally and skipped a step or thought to himself, "I know this process well enough. I'm just going to go through it." And he didn't adhere to his checklist. And so there was a time when it bit us. I forget what it was exactly, but the email didn't go out to the right people, or we missed a QA step, or it could have been something as blatant as we forgot to publish the reports to the people. They're in the staging area ready to go, the email went out, and this guy, just that last step of dragging and dropping them live, he just missed it.

So, that was high on our list to automate very quickly after that. And again, manual steps are going to be inevitable when you're doing out a new process and you've got that ethos of, I want to get value in my customer hands sooner than later, and I can get it to them today, or I can wait a week while I automate this other thing.

You probably want to get it to them today, but you can't wait too long for that manual step to get automated. Yeah. And he was a good guy. We all loved him. He was humorous.

00:20:00

Absolutely. Yeah. He brought good coffee to the office, and it was hard to be mad at him. But even the best people that you like, checklists are hard, and they just don't get done. And so, I think that's why Eric says automate, don't do checklists. And just to be clear, the fact that he skipped that checklist, that step, that was my fault.

It wasn't his fault. It was my fault for setting up a process that I know from experience is going to fail at some point. And it's going to be because of a person, and it's going to be because we didn't invest in the automation. It's like, there's no way around it. Should've done something. We did, but just not soon enough for that specific event.

So, speaking of getting bored, I got another good one. So, early on when Chris Gill and I were starting the company, we had a bunch of deliverables that went out to our customers. And, I was still in this mode of, it was early on in the process, we were adding new features all the time.

So we were building up automation debt. Well, I handed off my process to somebody, Chris, over there on the panel there. Me. So I handed it off to Chris. I'm like, "Here you go. Here's the checklist. Just follow this. If anything doesn't look like it is in the checklist, just reach out to me or there's a problem, so don't go through with it." So, I gave it to Chris to run it, and just like I did with that other person, Chris is awesome, but Chris didn't follow the checklist.

Yeah. I think what I remember of it is that Monday morning, where it was like we've always built this company profitably, and Gill and Eric and I are technical people, so we can do the work, but Eric very calmly said to me, "Chris, you sort of skipped this one step. And you know what?

This is a lot of risk for our customers. And why didn't you follow this step?" And I was like, "Oh, Eric, you're right. I'm sorry. It was Saturday, I was in a mood, I was frustrated, and I just sort of skipped it." And I think that's the other part, too, is people get in moods, and people want to skip things. And Eric had to go on vacation, and I think when you don't automate, what happens is you end up, and I know this personally, you end up screwing up.

And so I think that's one of the reasons why investing in automation, and the other part is learning from problems, I think is in there saying, okay. The nice thing about Eric is he was saying, "Okay, how are we going to fall forward on this? How are we going to learn?" And the other nice thing, too, I think, and I just want to reflect on what Eric said is when he was talking about the previous case is, whose responsibility is it when the checklist isn't followed?

Right? Is it the person who did it, or is it the leader who... And I think it's the leader's responsibility, right? Because leaders own the process. And as the leader of the team, that was the hardest thing in my learning sort of 15 years ago, is sort of reading Deming and the difference between a special cause and a process cause.

And Deming felt when things went wrong in a factory, 94% of the time it was the process and not the person. And that means it's a leadership problem. So leaders own the process people work in. And that's actually pretty hard as a leader to realize when this crap goes wrong, it's because you haven't set up a system for people to be successful in, and that's your job to fix it.

But I think that's the right perspective, to love your errors and use those errors to inform how you're going to improve and automate the system people work in.

And if we reflect back, I think what happened is the checklist was long enough, so by the time we got to the end, you were kind of tired, and the part that you skipped was investigating the warnings, which is outlier or not outlier, issue or not issue, which was the really important part, but unfortunately, the system was set up that it took a long time to get to the part where we needed a person to look at it.

Yeah. And while Eric got back from vacation, he had his weekend off, and he saved me.

We're a team. All right. So I'm going to move on. Just to close that last one out. Go ahead. No, that's okay. So we talked about, I'm looking at the reason, manual processes crowd out innovation, right? So Manual process is going to cause errors. If manual processes cause errors, it's those errors that start crowding out your innovation.

If you're always playing catch-up or spending too much time with manual things or investigating why something didn't go well, you don't have time to make improvements. So it's critical to address that. Okay, great. So the next reason

00:25:00

responsible for burnout is constantly playing catch-up with stakeholder requests. Gil, I think you have some good examples of this. Yeah. I'm actually a little surprised it's only 50%, because I think there's this phenomena where people can think of a lot of work in a short amount of time. You can say, "Well, what do you want to see?" "Oh, for reporting or for this, I'd like to see all these different views." And I think, if you think about it, in 10 minutes of time, you could think of a year's worth of work for a person or for a department. So I think there's always catch-up.

And

one thing that also compounds it is I don't think that your stakeholders always know what they want. They'll tell you what they want, but I think the key is to figure out what the priority is and if they really, really want it. So I had an experience where I was the BI engineer working with a customer, and they were very interested in process metrics coming out of an issue tracking system.

And I had a conversation, "What are the things you're looking for?" And I wrote it down, and I didn't feel that this person knew exactly what they wanted. So what I did is I just sketched out a bunch of... I really sketched out everything, 20 or 30 different views and set up a review.

And I would show the person, because I thought this is exactly what they wanted. I showed, "Is this useful?" And they go, "No." "This?" "No." "No." "That's a good one." So out of the 30, maybe only three were really hitting the mark. And if I had brought all 30 to production, that would've been really waste when three was the answer.

Great. Chris or Eric, do you have anything to add to that?

Let me think. Yeah. I always go through this exercise of when people give me a long list of things, I'm like, "You can do one thing. What's it going to be?" And everyone, it just causes them so much pain. "Well, I can't do one is not useful on its own versus that." So it's always an

exercise to reconfigure or regroup things such that there's these logical bits of delivery. And again, so you don't do things they don't need. You're always doing only what they need as soon as they need it, so. Yeah. Because I think the key question is what can you deliver that will give the most value to the organization?

Right. And my sister-in-law, who was at the time a CIO of a college, near Boston, was struggling with her shareholders. Everybody wanted everything all the time. And she was implementing an agile process and it's like, "How do we do the grooming?" And I said, "Well, what if you invite all your constituents to that process?

So you can say, 'We're going to, for our next sprint, this is our priorities. Do it in the open.'" And she said she tried it and she said there was something remarkable that happened. We put the first thing up there, then we brought the other up and someone said, "No, it's not as important as what you have on the board.

I could wait." And that- Yeah ... really floored her. So with the focus of delivering value, being transparent, that's a way to do it. And I think a corollary of that, this is borrowing from the agile community, is delivering often helps. Because if people know they're going to get something and it's going to be valuable, they could wait a little bit.

Where if there's releases that are like every six months or every year, what I've seen is people try to cram. I've seen this when I was a development manager, back using a waterfall process. People would say, "This is my shot. If I don't get it in now, I'm not going to get another crack at this for another six months." And I've seen it at an insurance company.

If the releases are too long, people tend to just cram as much in there because they're worried they're never going to see something come out and get value. Yeah. The long releases are a little susceptible to scope change as well,

where because it's so long, the priorities have changed, so people add things to the list and then, a lot of times they're not willing to take something off and so that's one of those other risky things. If you can... Yeah, because I've had many projects where there's 20 things to do. I'm just throwing that number out there.

And you deliver quickly, in small chunks,

00:30:00

all those 20 things, and you get halfway through and everyone's like, "You know what? We're good enough. And here's another 15 things that are completely new. We don't need to do the other 10." Yeah. So this is all about communicating with the customer and then getting a situation where everyone keeps each other informed, collaborates, and there has to be this level of trust, and the way you can prove to them you're going to be able to deliver is just delivering more often and showing value.

So then they start to believe you. And then you get into the thing where you say something's going to take 20 weeks, and they don't doubt you anymore because you've got this track record of small releases delivering value You gain this credibility, it builds on itself. And so you can take on bigger chunks for them, and they're not going to balk at the cost or time.

Yeah. And I think that trust is really important, but I also think the perspective of the data and analytic team is important, because a lot of times I think teams are sort of thinking of what they do as building a house. I'm going to build a house, it's going to take a while, and I'm going to have a punch list, and then I'm going to be done.

And I think it's much more about a relationship with your customer and building that relationship over time. And also, I think what is valuable to that customer is really almost the most important question that you have, and being humble about what you know and being humble about the ability to communicate. Because communication is incredibly hard to get a shared understanding of what's value, what's not, and we're all sort of touching different parts of the world.

And I think that really comes, when it's working, it works wonderful because your team is actually delivering value. The customer finds you valuable and that iterative process actually is so much more fun to work in. It's so much more enjoyable. It's so much less stressful because then you get to say to them, "Well, you know what?

We've been doing something quickly, but we want to be able to-- Can we put some documentation in this? Can we remove some technical debt in this sprint?" And oftentimes they're like, "Fine. Yeah, go ahead.

If it'll allow you to deliver more faster later, let's go and put it in right now." And so I think that's where it comes in. Search for value. That's the key thing that you're trying to do as a data and analytic team, and then use that as a way to build a good relationship with your customers.

Great. And I think we have a related question here from Jeff Hoover, who I believe is an old friend. So how do you advise clients to avoid being overloaded with too much data that they lose the key insights that they should action? With misguided insights, companies end up with novelty, not innovation. Thoughts?

A lot of times with the data sets, I want to know the business question that they're going to answer. Because sometimes you'll say, "Oh, just load it. We might need it." It always becomes, well, if that's the case, can I just put it on the list and we'll revisit it every week, right? As opposed to, oh, we have an immediate need.

We have clarity on what we're going to use this data for. I always push people for that. Another one is, like we said earlier, sometimes they don't know what they're going to use it for. So then it's like, okay, well, let's think of some scenarios where you could use it. Let's do a lightweight kind of proof of concept off the main production line so it's not in main production, and let's just see what it looks like. So we'll integrate these two data sets, we'll link them up on the key dimensions and then give you a couple new measures to work with, and then see if you can do something with it.

And they may say, "Okay, now that I got my hand on it, if you can just shape it a little bit differently or give me some other calculated measures or something." So that's kind of the key thing is lower the expense for them to do something. Yeah. Yeah. I'd sum it up and say, what's the least I could do for the organization to learn something?

Right. This is not useful or like, oh, this is a direction we want to pursue. Yeah. And then the other part too is it's almost like the psychology of the team. Because in those beginning years where we were constantly playing catch up, things were constantly going wrong, one of the biggest temptations is to get defensive, right?

And

what you want to do in your defense is build a wall, and, okay, I'm going to defend my wall. There's the opposition is on the other side, pummeling me with extra requests, pummeling me with bad data. I'm going to wall myself in, and then I'm going to be safe. And the reality is, what happens in that case is, number one, you avoid customer interaction, and then number two, you will create these moats of sort of almost vomiting up data sets and

00:35:00

reports. Here's all the data. You go figure it out. I don't know what it is. And you're not really searching for value. You're searching to build a huge wall around you and your team so that you don't get bothered with customer requests and problems. And that's not actually helpful to the organization. And whether it means you're giving them a large amount of data sets or you're giving them incredibly complicated dashboards with hundreds of pull downs, you're not actually helping the organization search for value.

You're just building a defensive wall around you and your team. And it's psychologically reasonable given the survey, right? You have too many customer requests and things are going wrong, but it's not actually the way to drive value to your customer. Great. One thing we do with that is, if you're doing Tableau or Qlik, you can monitor usage, right?

Chris mentioned the worst is when you have a data error and nobody notices it for a few weeks. Well, that happened to us. We implemented a business rule wrong. The QA group for the company validated it, said it all looked good. We basically had the inverse. So instead of 100, we were reporting 10, so it was off.

No one said anything about it. It was live for six weeks, not a single peep out of anybody. So it then starts to bring in, well, that thing you spent time and effort on, did you actually talk to the people that you're giving it to, and did they want it? Were they even going to need it?

And it turns out no one actually uses that metric because it was their old business model and it had just decayed. But we still spent time on it because our stakeholders hadn't talked to their end users and customers. So it's another thing to think of. So it's all great advice. So I'm going to move us along To reason four, disruptions to work-life balance.

Mm-hmm. Chris, I think you have a lot of good examples of this. Oh, yeah. Both personally and-- So at a conference, I was talking to a guy, we were talking about errors and DataOps, and he goes, "Oh, yeah. One of my worst cases is my daughter's fifth birthday, and it's a Saturday, and the house is full of kids, and my wife is barely hanging on, and I had to go into the bathroom and sit on the edge of the tub and fix a data error during the problem, because if I didn't, my boss's boss's boss was going to chew me out." And that's just sad, right? Your kids are young, they're having a birthday party, and your life is on hold because of it.

And I think that's really tough, and I don't think that's heroism, honestly. I think every once in a while, you want to go the extra mile and work nights and weekends because that's good, but not as a routine, not all the time. And so heroism is only good in small quantities. And so this idea that you're chest-beating, I'm working hard, I'm working nights and weekends, what happens is heroism leads to hairballs, leads to people quitting. And hairballs are people just don't do work well, and they only can understand it, and then your organization, then they inevitably will leave because they're tired of being a hero.

And then as a manager, you're stuck with this big, old, complicated thing that no one knows how to do, and your team's ability to actually do innovation gets further decayed. Yeah. And I think heroism is, again, a management issue. Why is there a hero? And you think of who's a hero, someone who runs into a burning building and saves somebody. But why is the building burning? Could that have been prevented? Is there a better system in place?

But this is about work-life balance, I guess heroism is part of it. It's interesting when work-life balance is designed in from the beginning, and what it means. So in one of my first projects out of school, so this is back in the mid-'80s. The project, it was for a government project, it was behind schedule, and there was a management edict, we're going to work on it, and we're going to work Saturdays as a workday until this is caught up, and I forget, it was four to six months of Saturdays.

And what I remember is the first few Saturdays, we did have an improvement in work, but then after just a few weeks, the behavior was, well, I was going to go do this errand. I'll do it now. Oh, you know what? I'll be in on Saturday. I could push this off and do this other thing.

Kind of the work expanded to fill the time,

00:40:00

and the plan of putting in extra work on Saturdays didn't really do the trick. So I think that's one case where designing nights and weekends or designing in heroism from the beginning doesn't really pay out in the long term. Yeah. The designing nights and weekends into your project plan. More often than not when that happens, like I said, it's the boss's fault.

It's like someone upstream screwed up. They totally underestimated the project, or they overcommitted, or they took in changes without modifying, right? Because you've always got that triangle, feature, time, quality, plus your staff, right? And so anytime you're getting into a project where there's nights and weekends, it's typically because someone upstream made a mistake and they kind of lacked the courage to go say, "Hey, we screwed up. Here's what it's really going to look like," or, "Here's the resources we need to make the date," right?

Now, I know that's oversimplifying. There's sometimes fixed cost projects if you're a consultant and a few other things. But nine times out of 10, I've found you've just got to be upfront and say, "We screwed up. We're not working nights and weekends. Here's what the new plan is," or, "Here's what I need to keep it on track." You've just got to have that. And that actually recently happened with me with a customer.

There's this new dashboard going live, there's a bunch of data handling to shape the data to help the dashboards be performant. And so my team, they're very familiar with the data. They knew what the transforms, so we had a really solid estimate on it. I had very high confidence level. And so it was coming out to some date six weeks after, and they're like, "Well, we told the field this is going to happen, or it's going to be released on this." And it's just like, I don't care what you told them.

You didn't ask me, you didn't ask the technical team what the date was going to be. I'm not going to cover for your poor management. You go tell them this is the new date. Because we're not going to work nights and weekends. And obviously I put it a little nicer than that. But the one thing my stakeholder who had to then tell the other person that they had to go fix their mistake, my stake for my stakeholder, though, is you do this once, they're going to think they can do it again. And what they're going to do is they're going to burn through your team, they're going to make this an environment where it's hard for you to retain people. And also those nights and weekend things, it's not going to go well, because now they're going to try to slip in scope because they feel they can manipulate the project on you.

Your team's going to make errors, right? Your quality's going to be low. It's not going to end well, and you're going to pay for it somewhere eventually. So why not the person who did poor estimation or poor negotiation on the project scope, why not make them suffer instead of burning out a larger group of people?

Yeah, and I think that's why it's all about having the bigger team and learning and communicating and trusting. Right? Right. If they

can learn that you can deliver, you can learn what they need, and we can all work better together and fast cycle time and sort of over commitments and heroism and wall building. What we're trying to do is get away from that, right? Because that all gets in the way of delivering value to your customer, and I think if I think about it, that's what we're searching to do and why Beth talked about these data and analytics projects failing and CDOs leaving and customers going back, the number of self-reported data-driven customers is going back.

That's why, I think, because we're not focusing on being able to build those relationships. Yeah. Great. Yeah, and the last one on that, where I was advising my stakeholder, "Hey, we got to go tell people this is the date, reset it." I was totally willing to stand behind my estimates, everything like that. I'm going to own, like, "Look, this is what the team can do." So we're not just throwing it out there.

I'm there to help them sell the case, too. All right. So reason number five, which is our last reason, blame and finger-pointing. So 63 of respondents are experiencing this. So I think, Chris, you had some examples of this, right, and Gil? Yeah. So, on my 42nd birthday, I had just joined this company where I think Gil hadn't joined yet, and maybe Eric as my first month or two, and I was going around having one-on-ones with everyone, and

00:45:00

there was one of the guys, a smart guy, went to an Ivy League school, 24 years old, and he came in my office, and he cried because he couldn't feel successful. He wasn't going fast enough, and he was having errors, and he just felt blamed, like, "I suck." And not only because we shared the same birthday, but it's like, he's a good guy and how can you build a situation where your team members who are well-intentioned don't have this life with them? And honestly, too, I was sort of part of the problem. Early in my career, I thought the right thing to do was to find people to blame. And as a leader, the easiest thing to do is when things go wrong, is to find a person at fault, not say it's the process at fault, because of course, you own that, saying, "It's a person.

They screwed up. I fire them. I get a new person that's better." And yeah, that's almost always not the case. Every once in a while, it is, right? But it's almost not the case. So, these are two examples that I think as a leader, scapegoating is often a bad case, and just noticing your team is feeling blamed and stressed is also part of an indication that you as a leader have some work to do.

Yeah. I

also think it's almost a natural thing. If something goes wrong,

someone must have caused it. So, there was an issue where actually things didn't go well, and what I do is there's still that in myself. I feel this like, "Uh-oh, it's this person's fault." But what I do is I keep quiet, and I just let the situation unfold, and I'm glad I didn't blame the person, because as the facts come in, it becomes more and more evident like, "Oh, it wasn't that person's fault.

It was this other thing, this other reason." So I guess as far as advice, if you feel like blaming someone, just don't. Bite your tongue, even though you really feel like doing it. And what does it feel like to be blamed? Well, I was on the receiving end of that. This is back at the company with Chris and Eric.

We had a new software initiative. I was the VP of engineering, hired a new team. We had the specs. We delivered the first release, and the first release was late. Right? There were some issues to work through, and the team hadn't quite gelled, and I was committed to improving it. Fast-forward a little bit to the Christmas party. Our CEO, at the Christmas party, kind of had a little thing to say about each department and that, "Oh, services did this. It was great.

Consulting did this. It was great. Engineering, well, kind of missed the mark." And it was like, oh, did I feel bad. And that was not a motivating event for me. I don't know if you remember it differently, Chris. I do, and I remember your wife being really p****d, too, like, "What the hell?" Yeah.

And I thought she was going to go up and tear him a new one, which she probably should have, honestly. And yeah, I think blame is a natural human tendency, and the thing of it is, it ends up not motivating people, and it also ends up hurting everyone. And so I think playing the blame card, playing the it's your fault card, it doesn't help a team move forward.

Yeah. And I think just that, and so many times, it's been really a systemic issue. It's not that one person, it's something else. It's part of the system that led up to it.

Great. Well, thank you all for highlighting your stories. I hope it helped people feel not so alone. So how do we make this all better? So we have some core DataOps principles which can help give you hope. Chris, do you want to talk about the first one? Don't suffer, build a system. Yeah. That's it. Don't suffer, build a system.

You can solve it. You are a technically good person, and you can use technology to actually build the system around it. And I've given versions of this talk now for years and years, and oftentimes I have people come up and say, "Wow, thank you for that. You've put words on my suffering, and I really appreciate it." And this perspective is not new. It happened in the software industry.

It happened in the manufacturing industry. And the systems and processes, the factory lines, the delivery lines that we all work in,

00:50:00

they can be improved, and you just don't have to live with the crap that goes on. And that sort of system-level thinking beyond what you do in your day-to-day work is, I think, really important. And that's why we've built this company, is to build that system, and why our software exists, to be that system that people can build and improve upon to make their life better.

Number two, don't agonize, automate that s**t. Chris, do you want to elaborate on that one, too? Yeah. Again, it's another alliteration. A lot of times, it is agony. It's agony to have your customer call you up and say the data's wrong. It's agony to be able to not make them happy. It's agony when your data providers give you

bad data. I

don't think that the way to get out of the agony is to automate things, and what we mean by automation is not checklists. We mean writing technology to do it, automated testing, automated deployment, infrastructure as code, ways to wrap all those nuggets of value that your team are creating in a system and processes and software to make that happen. And so if you think about it, that job of automating in a lot of data and analytic teams is seen as lesser work, and we have a different perspective on that.

We actually think that that's an empowering work that can, if you automate more and more, your team has lots and lots of leverage to do more work. So system level thinking on the processes that your team works in, the production process, the deployment process, and automation level thinking of, well, like Eric said, find the problem and how do you make sure it never happens again?

Well, don't put it in your checklist. Write a script on it. Automate it. Write a test that does it. Okay. Next is run toward errors. Yeah, I could- Eric, yeah. I think Eric may have dropped. Oh, okay. I could take that one. Okay. This is something I learned from Chris that I made my own, I really internalized, where if there's an error, it's an opportunity for improvement.

I think that's a really great way to frame it. And it's like in Eric's story, he had so many errors. What did he do? He tracked them. So, step one is love your errors, track your errors, and then you could do analysis on your errors. Is there one supplier that's causing more errors than not?

And that Pareto analysis actually keeps coming true, that 80% of the issues are caused from just 20% of the causes, so go after that first.

Done means live in your customer's hands. Gil, this is something you like to talk about. Can you elaborate on this? Yeah. I think it's really in the theme of done means... It gets back to our conversation on value. So you want to deliver value. That doesn't mean you've worked on it. If you work on something and don't give it to someone so they could use it, then it's really wasted effort. So I think that should become the mark is how can we actually push this over the finish line, put it in production, and like Eric said, sometimes people don't use what's in production.

Make sure there's adoption as well. So I think a focus on that is important, and if you can't see a path to delivering over the finish line, can you trim it down so you can? I think that's really the yardstick. Can I deliver value and get it in the hands of my customer? Great. So next is, don't be a hero. Make heroism a rare event.

Yeah. Whenever I think of heroes, I think of this old song.

Don't be a hero. Don't make a fool of your life. I don't know if anyone remembers that, but it was very popular, but then voted one of the worst songs of the decade.

Well, why is it the worst song, Gil? Why are you making a fool of your life when you're a hero? I think it gets back to what we said before. Heroism means that there's-- Now, we're always thankful when someone is a hero, but if someone's a hero too often, they're compensating for some systemic problem.

So again, you could say if someone has to work a night and a weekend, it's like, eh, that's a type of an error. So I would say, why is that? Work to eliminate the need for that.

00:55:00

It ties into burnout and sometimes if people work too long, they start inserting errors instead of taking them away. So if someone is a hero, there's something to be fixed up. And even as you lead your team, we're all tempted to sort of clap people on the back and say, "Hey, thanks for working that weekend.

Thanks for going through the extra mile to make a customer." And you should do that. But you know what? In the one-on-one the next time, sit down and say, "Well, why did you do that? Did you really need to do that? What's the problem?" Think of that in your one-on-one as a problem to be solved, not as something to be rewarded.

So sort of praise in public, and I don't know if the term punish in private, but get perspective on it in private, because I think the more you, as Gil said, all these issues with a company of heroes doesn't make for a successful team. Great. And the last bit of advice is, don't put your head in the sand.

Focus on value delivery. What do you mean by that, Chris? Yeah, I've struggled with this. As technical people, we all sort of want to build our sort of crystal castles of technology coolness, right? And having customers get involved, kind of gets in... It's like you're solving a puzzle. Let me just get it done, or building a model, and you just want to do it and not interact with the world. And I get that.

I'm a technical guy and have been so for a long time, but that involves you putting your head in the sand. And it means that you're submerging from interacting with a customer, you're spending a long time, and that's an anti-pattern of success. You want to focus on value delivery, meaning you talk to your customer, they've seen it, they've touched it, they've given you feedback, and then you keep trying and trying. And I think that is...

And then the other anti-pattern of that is people build these, they decide that their job is to fill a bunch of database tables, and that's it, "I've done the database tables, I'm done, and I can walk away." And does that mean that they're being used? Does that mean people trust them or understand them?

I think what we've got to own as sort of mature people in the data and analytic industry is that we're having an effect on the world. And that effect on the world is not just the data in the table, it's data that they trust in a form that they want, visualized or modeled, governed in a way that is successful. And all those things have got to be right.

And I think that's where we don't want to get too far back from building crystal castles and walls and sort of pull your head out of the sand. And you're going to be happier for it, and you're going to actually enjoy your life and not suffer as much if you do that, even though at first it may actually be a little harder.

Great. Well, thank you, all three of you, I don't think Eric's here yet, but for this great conversation. We had such a good conversation. We ran out of time for questions, but I'm going to sneak in one just to close it off. Love to get all your thoughts on this. If there was one thing you would recommend to do to make your life better, what would it be? Where do I get started?

Chris, do you want to take this? It's just start. I

often recommend the idea of make a checklist of your errors and find one to automate. Like that's the biggest, a data error, put in a test, very simple. And look at one. It's slow to deliver, look at one manual step and automate. And so start today, start small, do something. And it doesn't have to be a big project, but I think you can start seeing the compounding effects of working on these systemic things. And if you do that, you can actually, over time... And one of the first things that Gil and Eric and I did is we formed a quality circle, and we met every week, and we looked through our list of errors, and we did it. And that really became a very powerful meeting for me, not only because it made me feel better that we were doing something, but it became a way that we actually improved our whole work for our team.

So, it's do something, automate something , pick something. Gil, would you agree with that? Or what would you do? Yeah, that's a really good one. It's hard to pick one. Automation is great because it's a bank that yields dividends. Looking at your errors is important. But I'm just a big fan of how can I deliver value? Because maybe it's the third leg of that stool is why are you doing all this for value? And with that, you need to engage with your

01:00:00

customer, and I'm kind of adding, and do it in a respectful way. So I'm going to go with value, focus on value, and be respectful. Okay, great. Well, thank you both. Unfortunately, we're out of time. We could probably talk all day about these topics. Thanks for joining, Gil, Chris, and Eric, and sharing your experiences and your advice. Thanks to all the attendees.

We hope you found this useful and hopefully it gives you some ideas to alleviate some of the suffering you may be feeling. We do have other resources that can help you. First, you can sign the DataOps Manifesto. Just go to dataopsmanifesto.org. You can read the "DataOps Cookbook," which talks about all the main principles of DataOps. And then we recently published a second book called "Recipes for Data Op Success" that help you get a program off the ground at your organization. And then also on our website, you can also take the DataOps Maturity Model and see how your team is doing and benchmark against other teams.

So again, thanks for attending. We will be sending out the recording and the slides in the next 24 to 48 hours, so be on the lookout for that in your email. If you have any additional questions, don't hesitate to reach out to any of us. You'll have our contact information in the email. And so that's it. With that, we'll sign off, and I hope everyone has a great afternoon and evening.

All right. Bye everybody. Bye.