NOTE
First published on Medium in March 2017 and republished here in August 2026. The surveys and figures are as of the original publication.
A quick guide to DevOps for non-engineers.
In a recent post we discussed how software development teams accelerated average release frequency from 12 months to three weeks using Agile development. Today we will talk about taking the next gigantic step: accelerating from three weeks to three minutes.
Everyone was running a different machine
Before the advent of on-demand cloud services, the various groups in software development (design, development, test, quality, support) had to set up their own infrastructure. Whatever components were needed (physical servers, networks, storage, software) had to be ordered, installed, configured and managed by the IT department. Servers would be ordered at different times and from different vendors, each slightly different from the other. Depending on the task at hand, different machines could have a different array of software applications, with revisions of each app being continuously updated. With all this variability, it was quite common for individuals within the company to be running code in different environments. Outside the four walls of the company, customers could be running in yet another environment. This situation presented challenges.
If, for example, the customer reported a problem, it might not be replicable in the support, test or development groups due to differences in the hardware and software environments being run. This lack of alignment fostered misunderstandings and delays, and often led to a lack of trust and communication between the various stakeholders.
Infrastructure as Code
About a decade ago, Amazon Web Services and other cloud providers began offering computing, storage and other IT resources as an on-demand service. No more waiting weeks or months for the IT department to fulfill a request for servers. Cloud providers now allow you to order computing services, paying only for what you use, whether that is one processor for an hour or thousands of processors for months.
These on-demand cloud services have enabled developers to write code that provisions processing resources with strictly specified environments, on demand, in just a few minutes. This capability is called Infrastructure as Code. It has made it possible for everyone in the software development pipeline, all the different groups mentioned above, to use an identical environment tailored to application requirements. With Infrastructure as Code, design, test, QA and support can easily get on the same page. This leads to much better collaboration between the groups and breaks down barriers that prevented open communication. In other words, no more finger pointing.
With IT infrastructure being defined by code, the hard divisions between IT operations and software development are able to blur. The merger of development and operations is how the term DevOps originated.
What a fully automated release process bought
Average release frequency over time. Agile took it to three weeks; DevOps took it the rest of the way.
With the automated provisioning of resources, DevOps paved the way for a fully automated test and release process. The process of deploying code that used to take weeks could now be completed in minutes. Major organizations including Amazon, Facebook and Netflix now operate this way. At a recent conference, Amazon disclosed that their AWS team performs 50,000,000 code releases per year. That is more than one per second. This methodology of rapid releases is called continuous delivery, or alternatively continuous deployment, when new features and fixes are not only delivered internally but fully deployed to customers.
DevOps improves collaboration between employees from the planning through the deployment phases of software. It seeks to reduce time to deployment, decrease time to market, minimize defects, and shorten the time required to fix problems.
The impact of DevOps on development organizations was shown in the 2014 State of DevOps Report by Puppet Labs, IT Revolution Press and ThoughtWorks, based on 9,200 survey responses from technical professionals. The survey found that IT organizations implementing DevOps were deploying code 30 times more frequently and with 50 percent fewer failures. Further, companies with these higher performing IT organizations tended to have stronger business performance, greater productivity, higher profitability and larger market share. In other words, DevOps is not something that engineers are doing off in a dark corner. It is a core competency that helps good companies become great.
What this has to do with data
The data analytics team transforms raw data into actionable information that improves decision making and provides market insight. Imagine an organization with the best data analytics in the industry. That organization would have a tremendous advantage over competitors. That could be you.
The lessons learned in DevOps can be applied to data analytics. At DataKitchen, we call this DataOps. The third tradition it draws on is the statistical process control that comes out of lean manufacturing.
