Navigating a Recession with DataOps

Recession research shows agile, efficient companies pull ahead. See how DataOps helps data teams automate operations, cut errors, and do more with less.

Written by DataKitchen Marketing Team on March 31, 2020

DataOps Principles
Navigating a Recession with DataOps

Key points

  • Written on 31 March 2020, in the first weeks of the COVID-19 disruption, this post argued that data leaders should reset plans made only months earlier rather than wait for conditions to settle.
  • The post cited 2010 Harvard Business Review research on 4,700 public companies across the recessions of 1980, 1990 and 2000, in which 17% went bankrupt, were acquired or went private, while 9% outperformed rivals by at least 10% on top-line and bottom-line growth.
  • That same 2010 Harvard Business Review research found companies focused only on cost-cutting had a 21% chance of emerging stronger and companies focused only on growth had a 26% chance, while the “progressive” group that combined efficiency-driven cuts with confident investment reached 37%.
  • Citing Gartner in early 2020, the post said data teams spent an average of 56% of their time manually executing and maintaining the data operations pipeline, and that data professionals spent only 22% of their time creating analytics and models.
  • The March 2020 advice to data organizations was to abandon multi-month waterfall projects, automate orchestration and testing across both the data operations and analytics development pipelines, and cut idea-to-delivery cycle time from months to hours or days.

The Corona COVID-19 virus has completely changed the landscape for business in 2020. Strategic plans that were approved a few short months ago are being scrapped. With a recession at hand, it’s time to hit the reset button on your plans.

Business thought leaders were writing about a potential downturn last spring. No one can predict the timing or duration of a recession, but business cycles are inevitable. Let’s take a moment and review what researchers and management consultants have written about preparing for tough times and laying the groundwork to emerge even stronger during the inevitable recovery.

Recession Strategy for Superior Performance

In a March 2010 article in Harvard Business Review, researchers conducted a study of the performance of 4,700 public companies in the periods three years before, during, and after the recessions of 1980, 1990, and 2000. Here’s what they found:

Few business leaders have a master plan already figured out when entering a recession. Further, executives must walk a delicate balance between financial and brand management. Large companies operate in an environment that includes shareholder and employee activism, community stakeholders, and a wider societal conversation about social impact and income equality. These factors can create a political backlash or affect employee morale and force managers to consider the public impact of their decisions.

As data professionals, we know that data-driven decision making is now more critical than ever. Data analytics can serve as a company’s most valuable tool when evaluating proposals for cost-cutting or investment. Analytics could mean the difference between finding the right mix of strategic moves and falling behind. About a year ago, an article written by management consultant McKinsey & Company encouraged companies to plan for a future recession by turning to digital tools and advanced analytics to manage offensive and defensive strategies. Management consulting firm Bain & Company offered similar recession-planning advice. They recommended looking to digital tools and next-generation data analytics to find and pursue customer-focused opportunities that capitalize on the upcoming recession and subsequent growth cycles.

Preparing the Data Organization

As a leader of a data organization, you must prepare for this critical phase. It’s time to get your own house in order. You’ll need to respond to requests quicker than ever and accomplish a lot more with the same or fewer resources. Here’s how you can do that:

There are several other tips for streamlining analytics development – see our blogs on analytics in the on-demand economy and the 7 Steps of DataOps.

Figure 1: Data professionals spend only 22% of their time creating the analytics and models that drive innovation and monetize data. Most of their time is spent on tasks that could be automated.

DataKitchen DataOps Automation

We rolled these capabilities and many more into our DataOps Platform. Yes, you can build these functionalities in-house, but if you haven’t done so already, you are behind. The fastest way to get your data organization ready for the upcoming recession is to leverage a DataOps Platform that is already implemented and proven. Our DataOps customers have reduced error rates to virtually zero per year, slashed cycle time, and have greatly improved the ability of their data team to support users. One of our customers improved the productivity of their data analysts and data engineers by an order of magnitude. See the Celgene case study for more information.

Figure 2: Productivity of data engineers and data analysts before and after implementing DataKitchen DataOps Automation.

DataKitchen DataOps Automation enables your data organization to:

Differentiation Through Data

Research shows that a difficult economic environment can separate the leaders from the laggards. Data analytics and digital automation promise to be among the key ingredients that successful companies use to find opportunities to improve efficiency, productivity and growth. Data organizations need to prepare themselves for this remarkable moment in history by restructuring their own workflow processes to rapidly deliver high-quality analytics that address the enterprise’s strategic goals.

For more information about DataOps and DataKitchen DataOps Automation, see our blog, “What Is DataOps: Ten Most Common Questions.”


FAQ

What are the key points in this blog?

Written on 31 March 2020, this post argued that a downturn separates leaders from laggards and that data teams should prepare by getting their own operations in order. It cited 2010 Harvard Business Review research on 4,700 companies, a Gartner figure of 56% of data team time spent manually running pipelines, and recommended agile delivery, automated orchestration and automated testing over multi-month waterfall projects.

What does recession research say about which companies emerge stronger?

The 2010 Harvard Business Review study cited in this March 2020 post tracked 4,700 public companies before, during and after the recessions of 1980, 1990 and 2000. Seventeen percent went bankrupt, were acquired or went private. Nine percent thrived, beating competitors by at least 10% on both top-line and bottom-line growth. Most of the rest still struggled three years later.

What is a progressive company in recession research?

Progressive is the label the 2010 Harvard Business Review research gave to companies that combined cost-cutting aimed at operational efficiency with confident investment in marketing, research and development, and new assets. In that study they had a 37% chance of emerging strong, against 21% for cost-cutters only and 26% for growth-only firms. This March 2020 post held them up as the model.

How much of their time did data teams spend maintaining pipelines in 2020?

This March 2020 post cited a Gartner report saying data teams spent an average of 56% of their time manually executing and maintaining the data operations pipeline. The accompanying figure showed data professionals spending only 22% of their time creating the analytics and models that drive innovation. The post called that split a travesty and argued for automating the routine work.

What did this post recommend for data teams facing a downturn?

Writing in March 2020, the post recommended four moves: drop waterfall projects that defer value for months, adopt agile development so priorities can shift quickly, automate the data operations and analytics development pipelines, and add quality tests at every pipeline step so errors are caught before users see them. The goal was cycle time measured in hours or days.

What did DataKitchen say its DataOps Automation delivered for customers in 2020?

As of this March 2020 post, DataKitchen said customers using DataOps Automation had reduced error rates to virtually zero per year, slashed cycle time, and improved their data teams’ ability to support users. One customer was described as improving the productivity of its data analysts and data engineers by an order of magnitude. The post pointed to a Celgene case study.

Install Open Source TestGen Free, no vendor lock-in Request a Demo See TestGen Enterprise in action

DataKitchen Marketing Team

The DataKitchen marketing team curates industry news, resources, and thought leadership on DataOps, data quality, and data observability.