Data Observability and Data Quality Testing Certification Series

Join Our Free Webinar Series: Unlocking the Power of Data Observability and Quality Testing

Written by Chris Bergh on May 14, 2024

DataOpsData ObservabilityDataOps ObservabilityDataOps TestGenOpen SourceOn-Demand Webinar
Data Observability and Data Quality Testing Certification Series

Key points

  • Session one of four. The series ran May 21, June 4, June 18, and July 2, 2024, with 243 people signed up from over 100 companies, 35 countries, and 6 continents; certification came from completing all four parts and the homework exercises.
  • The framing problem is waste, not tooling: 60% of projects fail (Gartner), 79% have too many errors (Eckerson), 73% of data practitioners do not trust their data (IDC), and 78% of data teams are stressed enough to want therapy (DataKitchen).
  • The cause named is a Day 1 focus — building with individual tools and chasing immediate tasks — against four standing pressures: bad raw data, a complex and fragile toolchain, customer-visible errors, and too much to do. DataOps is the Day 2 and Day 3 answer.
  • There are five data observability use cases, and they differ by when they happen: evaluating a new data set before it goes to production, monitoring ongoing ingestion, watching multi-tool production runs, testing in development, and checking a data migration against the legacy system.
  • Data checks and tool monitoring are not the same discipline. Data checks test the data itself — schema, row count, drift; tool monitoring watches the tools and servers acting on the data — logs, metrics, tasks, schedules. Most use cases need both, in different proportions.
  • Profiling is the tool for the Pass, Patch, or Pushback decision: is this new data good enough to pass along, are the errors ones I can patch in the pipeline, or do I need to push back to the provider? The profiling output is the evidence that makes that conversation with a stakeholder fact-based.

We are excited to invite you to a free four-part webinar series that will elevate your understanding and skills in Data Observation and Data Quality Testing. This series is crafted for professionals eager to deepen their knowledge and enhance their data management practices, whether you are a seasoned data engineer, a data quality manager, or just passionate about data.

Session 1: Setting the Stage for Data Excellence

In our opening session, we will explore the foundational concepts of data testing versus data quality and discuss the critical role of data testing within the data journey. Our expert, Eric, will guide you through a live demonstration of pre-production data testing strategies, including data profiling and initial data hygiene scans, focusing on identifying and addressing the dozens of anomalies impacting data quality.

Session 2: Enhancing Data Reliability in Production

Our second session will delve into the intricacies of data testing during production. Chris will overview data at rest and in use, with Eric returning to demonstrate the practical steps in data testing for both states. You will learn about our twenty-eight data profiling quality checks and the eleven best practices for custom data validation tests, ensuring you can maintain data integrity even in the most dynamic environments.

Session 3: Mastering Data Testing in Development and Migration

During our third session, the focus will shift towards regression and impact assessment in development cycles. Discover the best practices for functional and performance testing and essential insights into data testing as part of data migration projects. This session is crucial for those looking to ensure data consistency and accuracy across different stages of data handling.

Session 4: Towards Data Testing and Observability Maturity

Our final session will cover the roles, management, and metrics involved in data testing and introduce you to the Data Testing Maturity Model. We will discuss measuring testing efforts effectively and how robust testing practices can lead to fewer errors, increased data trust, and enhanced productivity.

Reserve Your Spot! Don’t miss this opportunity to transform your data practices. Each session is designed to build upon the last, providing a comprehensive learning journey. Register for free today and take the first step towards mastering data observability and quality testing!

About The Series

Participants will receive a free Data Observability and Data Quality Testing Certification upon completing the four-part series and successfully answering homework exercises using DataKitchen’s open-source Data Observability Software.

Get Certified Today Upskill in Data Observation and Data Quality Testing Install Open Source TestGen Free, no vendor lock-in
Chris Bergh

Chris Bergh

CEO and Head Chef at DataKitchen. He is a leader of the DataOps movement and is the co-author of the DataOps Cookbook and the DataOps Manifesto.

LinkedIn →

Don't want to give us your email address? Go directly to the webinar recording here.