Why choose DataKitchen? During my nearly seven-year tenure leading an analytics function at Celgene, our partnership with DataKitchen was a critical component of my team’s data and analytics strategy. DataKitchen preaches the message of DataOps, a philosophy they shaped years ago to express their fundamental belief that focusing on data quality, automation, and accessibility at every stage in the data continuum will have a transformative effect on the organization. Let me just say with respect to DataOps – I’m a BELIEVER!
The “Why Choose DataKitchen” question for me always had two parts:
- The belief that DataOps delivers on the transformation promise (it does!).
- DataKitchen’s ability to deliver DataOps benefits (they can!).
The impact of partnering with DataKitchen and establishing effective DataOps practices at Celgene was that the team did a whole lot more analytics, faster and with fewer errors than the other companies we benchmarked. This allowed the team to focus more energy on bringing value to our partners.
Organizations, such as Gartner, report that analytics and data science teams spend significant time and energy on transforming and correcting data, leaving much less time to do value-added analytics. That was my personal experience as an analyst, and I never met a colleague who didn’t encounter those same challenges. Using DataOps, our team spent dramatically less time with data prep, rework, and error tracing, freeing the team to innovate, iterate, and improve.
There are many tools in the DataOps kit. From my perspective, testing, automation, and Agile practices laid the foundation for the team’s success. Testing in the way DataKitchen tests completely changed our team’s relationship with the data. In the DataOps world, testing is easy to set up and deployed across the data continuum. Data was tested at the point of origin and then at each stage, as data propagated throughout the ecosystem to include the analytics and reports.
DataKitchen helped us completely transform our operations by broadening our definition of testing. Testing the DataKitchen way was not limited to checking for basic attributes such as columns and rows; we expanded testing to include data accuracy and continuity. Tests assess important questions, such as “is the data correct?” When business rules are applied, “does the data make sense?” “Is there something new or unusual,” and does the data align with its history and related data sets?” The DataKitchen platform not only makes this level of testing possible but also practical.
The team’s increased testing capability led to another best practice. When a new error or anomaly is detected in the data, the platform makes it easy to build a new test allowing the team to catch that same error in the future. The real-world impact of this type of testing cannot be understated. When the team can be confident that 99%+ of data errors will be caught before analytics or reports are published, it gives everyone confidence in the results and builds credibility across the company.
Many teams automate, but DataKitchen inspired a new level of automation in our analytics. We challenged the Celgene team to think in these terms. If I am going to do a task more than twice a year, why wouldn’t I automate it? The relatively moderate effort in automation paid massive dividends by allowing team members to always be working on the next important question or searching for the next big insight. Testing and automation alone helped the team take back hours and hours of non-value-added effort each week.
Agile practices also helped to transform the team. We never did Agile just for the sake of doing Agile. Instead, we adopted the mindset of trying many techniques and adopting what worked. For the Celgene team, Scrums had a positive impact, as did working from a backlog. In the case of Scrum, the simple act of being able to adjust efforts daily across the team helped to ensure people were working on the highest priority project at any given time. Working from a backlog helped the team prioritize work without the fear of “dropping the ball” on a request or project. Over time the team developed the practice of sharing the backlog with our partners, who helped us prioritize efforts. Because of this practice, the status of a request was always transparent, and if the work was further down the backlog or removed altogether, everyone understood why.
Over this multi-year journey, the DataKitchen team never lost their drive to help make our team successful. They always delivered exceptional results and taught us how to do the same in the DataOps arena. The team at DataKitchen does have the recipe for success in helping teams spend more time bringing value through a DataOps transformation.
About the Author
