Since the first edition of the DataOps Cookbook in 2019, we have talked with thousands of companies about their struggles to deliver data-driven insight to their customers. In many ways, they all have the same problems. They have built data and analytic systems with great hope of success. Still, they are burdened with too many errors, are overwhelmed with custom requests, and know they fail to succeed in their goal of leading their organizations to be more data-driven.
And on top of that, they are all unhappy, stressed, and wondering what went wrong. As an industry, we have a conceptual hole in how we think about data analytic systems. We build them and put them into production, but then we hope all the steps data goes through from source to customer value work out correctly. We all know that our customers frequently find data and dashboard problems. Teams are shamed and blamed for problems they didn’t cause. They need help with existing complicated multi-step data systems that often fail and output insights no one trusts.
We had the same problem starting in 2005 when we left software development and started to lead data teams. Over the years, we have put together these principles of DataOps, built multiple generations of software to solve the problem, and helped many customers succeed. We talk with data teams every few days with the same “morning dread” I had lending data teams 15 years ago. That feeling that something is going to go wrong, you’ll have no idea how to find it, and fixing it will put off all the other tasks that need to get done.
The new idea showcased in the third edition of the DataOps Cookbook is to focus first on understanding and observing the journey that data takes through your production environment – from ingestion to processing to delivering actionable insights. This monitoring process identifies data errors, tool problems, and timing issues, enabling a quick win for your DataOps implementation by driving immediate improvements. Lowering production errors increases the reliability of your data and gives your team more time to focus on automation. And all this has to happen in days, not months. The DataOps Cookbook-‘Data Journey First DataOps’ Third Edition is the answer to that challenge.
We’ve included many new chapters about Data Journeys, Observability, and the benefits of focusing on production errors as the first step in DataOps. These include
- Data Journey Manifesto
- Why the Data Journey Manifesto?
- Five Pillars of Data Journeys
- Data Journey First DataOps
- The Terms and Conditions of a Data Contract are Data Tests
- “You Complete Me,” said Data Lineage to Data Journeys.
- Two Downs Make Two Ups: The Only Success Metrics That Matter For Your Data & Analytics Team
- DataOps Observability: Taming the Chaos:
- DataOps TestGen: ‘Mystery Box Full Of Data Errors’
- Tomorrow’s Forecast: Cloudy with a Chance of Data Errors
Data Teams often need more. They desire quick, valuable assistance without significantly changing their existing systems. The ‘Data Journey First DataOps’ approach aligns perfectly with these needs.
You can download the new edition today!
