On-Demand Webinar · 1 hr 1 min

DataOps & Analytics: A Recipe for Accelerating Business Value

Mark Marinelli, Head of Product at Tamr, and Chris Bergh of DataKitchen introduce DataOps: why large organizations need it, the benefits teams report, how to get past the obstacles that stall adoption, and real examples. Recorded 2020; updated August 2026.

Presented by Chris Bergh

What you'll learn 8 points
  • In a DataKitchen and Eckerson research survey, 79 percent of respondents said they have an unacceptable number of errors and incorrect values in their data analytics products every month.
  • Gartner's March 2020 survey, Data Management Struggles to Balance Innovation and Control, found data analytics teams spend only 22 percent of their time on new initiatives and 56 percent on operational execution.
  • Deming's finding that 94 percent of causes are common cause is used to make the case for fixing the process instead of looking for a person to blame, alongside Elon Musk's line that the real difficulty is building the machine that makes the machine.
  • DataOps runs three orchestrations, not one: the Value Pipeline in production, the Innovation Pipeline that moves changes from development to production, and an Environment Pipeline underneath both.
  • Tamr's DataOps framework has three components: process, meaning an agile incremental delivery model; technology, meaning the architecture of the data supply chain and the platform under it; and organization, meaning roles across mixed-skill teams and a working model between technical and business teams.
  • Two operating models are offered. Shared services centralizes technical knowledge and resourcing but creates contention over priorities. The advisory model bootstraps projects with experts rather than implementers but leaves development and maintenance on each department. Start with shared services for the first project.
  • The recommended way in is to pick one constraint, solve it, measure it and iterate: too many errors, slow deployment, or poor coordination between teams.
  • An American transportation company running Nifi, Kafka, an ESB and Informatica across on-premises and AWS, with separate teams in different locations owning creation and operation of the pipelines, started with errors. It used a DataKitchen Recipe to test streaming data for fitness for purpose across the toolchain and route alerts to JIRA, email and Slack.

Slides

59 slides

Questions from this session

How much time do data analytics teams spend on operations instead of new work?

Gartner's March 2020 survey, Data Management Struggles to Balance Innovation and Control, found only 22 percent of a data analytics team's time delivers innovation and new insight, against 56 percent spent on operational execution. The rest goes to improvements and technical debt. Errors and manual production tasks are what consume the difference.

What are the three DataOps pipeline orchestrations?

The Value Pipeline is production: data flowing to customers, which has to be tested and monitored so problems are not reported by the customer first. The Innovation Pipeline is deployment: changes moving from development to production without breaking what is running. Underneath both sits the Environment Pipeline, which creates and manages the environments the other two depend on.

What is the DataOps framework?

Tamr's framework has three parts. Process is an agile incremental delivery model instead of a labor-intensive, monolithic, IT-driven one. Technology is the architecture of tools in the data supply chain and the infrastructure under it: cloud first, loosely coupled, best of breed, continuous, with lineage treated as essential. Organization covers roles across mixed-skill teams and the structure connecting technical and business teams.

How do you get started with DataOps?

Inventory the projects available and score them on data availability against the value of solving the problem, then pick one high-value, data-rich project complex enough to force end-to-end coverage. Inventory the current tool set on cost and skills, decouple monolithic processes by wrapping components in APIs, and build a cross-functional team with data engineers, a curator, a steward and the consumers, with executive alignment behind it.

Which bottleneck should a DataOps program tackle first?

Whichever of the three constraints the team already complains about: too many errors, meaning customers find the data problems first; slow deployment, meaning changes risk breaking production; or poor coordination between data science and analytics teams. Select one, solve it, measure the result, then iterate. Trying to fix all three at once is how programs stall.

What are the common mistakes when starting a DataOps program?

Five. Running a waterfall project measured in years instead of delivering analytic value along the way. Overestimating what a single platform can do. Overestimating what a single vendor can do, rather than aligning vendors on APIs and the expectation that they work together. Underestimating the effort to make open source work. And underestimating the human and behavioral challenges, which are the most common reason projects stall.

Where to go next