As DataOps activity takes root within an enterprise, managers face the question of whether to build centralized or decentralized DataOps capabilities. Centralizing analytics brings it under control but granting analysts free reign is necessary to foster innovation and stay competitive. The beauty of DataOps is that you don’t have to choose between centralization and freedom. You can choose to do one or the other – or both. Below we’ll discuss some standard DataOps technical services that could be developed and supported by a centralized team. We’ll also discuss building DataOps expertise around the data organization, in a decentralized fashion, using DataOps centers of excellence (COE) or DataOps Dojos.
DataOps Technical Services
A centralized team can promote DataOps adoption by building a common technical infrastructure and tools to be leveraged by other groups. Centralizing analytics helps the organization standardize enterprise-wide measurements and metrics. For example, some teams may recognize services revenue in the quarter booked, and others may amortize the revenue over the contract period. With a standard metric supported by a centralized technical team, the organization maintains consistency in analytics.
A centralized team can publish a set of software services that support the rollout of Agile/DataOps. The DataOps Technical Services (DTS) group provides a set of central services leveraged by other groups. DTS services bring the benefits of DataOps to groups that aren’t ready to implement DataOps themselves. Examples of technologies that can be delivered ‘as a service’ include:
- Source code control repository
- Agile ticketing/Kanban tools
- Deploy to production
- Product monitoring
- Develop/execute regression testing
- Development sandboxes
- Collaboration and training portals/wikis
- Test data management and other functions provided ‘as a service’
The DTS group can also act as a services organization, offering services to other teams. Below are some examples of services that a DTS group can provide:
- Reusable deployment services that integrate, deliver, and deploy end-to-end analytic pipelines to production.
- Central code repository where all data engineering/science/analytics work can be tracked, reviewed and shared.
- Central DataOps process measurement function with reports
- ‘Mission Control’ for data production metrics and data team development metrics to demonstrate progress on the DataOps transformation
DTS creates robust DataOps services and capabilities, but if an organization wishes to seed DataOps practices throughout the organization, it should plan methods to transfer DataOps solutions and “know-how” to data scientists and engineers in the periphery of the organization.
DataOps Center of Excellence
The center of excellence (COE) model leverages the DataOps team to solve real-world challenges. The goal of a COE is to take a large, widespread, deep-rooted organizational problem and solve it in a smaller scope, proof-of-concept project, using an open-minded approach. The COE then attempts to leverage small wins across the larger organization at scale. A COE typically has a full-time staff that focuses on delivering value for customers in an experimentation-driven, iterative, result-oriented, customer-focused way. COE teams try to show what “good” looks like by establishing common technical standards and best practices. They also can provide education and training enterprise-wide. The COE approach is used in many enterprises, but the DevOps industry has more often standardized on Dojos as a best practice.
DataOps Dojo
Demand for skilled DataOps engineers is skyrocketing, and like DevOps engineers, they are hard to find and harder to hire. Enterprises moving towards DataOps transformation may find it worthwhile to build DataOps expertise organically in each team within the data organization.
A DataOps Dojo is a place where DataOps beginners go for a short period of intense, hands-on training. In Japan, a dojo is a safe environment where someone can practice new skills, such as martial arts. Companies like Target, Delta Airlines, and John Deere employ the Dojo concept effectively to build lean, Agile, and DevOps muscles. The Dojo offers a separate workspace where teams learn new skills while working on actual projects that deliver customer value.
Dojos provide an environment where teams gain practical experience without worrying about introducing errors into the production environment. The staff rotates in for weeks or months at a time to learn new skills by working on real-world projects. They then bring those skills ideas back to their original teams.
DataOps Transformation
Each of the approaches described above can deliver DataOps benefits to the enterprise. Nevertheless, it can be challenging to grow DataOps expertise in-house without the benefit of mentorship. DataKitchen offers DataOps Transformation Advisory Services that address DataOps methodologies, strategy, tools automation, and cultural change. Learn more about our DataOps Transformation Advisory Services here, or feel free to reach out if you just want to chat.
FAQ
What are the key points in this blog?
DataOps capabilities can be built centrally, locally, or both. A DataOps Technical Services group offers shared infrastructure such as source control, deployment, monitoring, and test data management as services. A center of excellence proves a solution at small scope and spreads the win. A DataOps Dojo gives teams a short period of intense, hands-on training on real projects, away from production.
What is a DataOps Dojo?
A DataOps Dojo is a separate workspace where people go for a short period of intense, hands-on training in DataOps practices. The name comes from the Japanese dojo, a safe place to practice a new skill. Staff rotate in for weeks or months, learn by working on real projects that deliver customer value, and take the skills back to their own teams.
Should DataOps be centralized or decentralized?
Both, in most organizations. Centralizing analytics brings consistency to enterprise metrics and lets one team maintain shared tooling, while decentralized practice is what keeps analysts free to innovate. The usual arrangement is a central group publishing services plus local expertise grown team by team, so a broader DataOps transformation does not depend on a single group staying available.
What is a DataOps Technical Services group?
A central team that builds common infrastructure and offers it to other groups as a service. Typical offerings are a source code repository, Agile ticketing, deploy to production, product monitoring, regression testing, development sandboxes, collaboration portals, and test data management. It also runs process measurement and reporting, which gives the organization one place to see progress on adoption.
What is the difference between a DataOps center of excellence and a dojo?
A center of excellence owns outcomes; a dojo owns skills. The center takes a widespread organizational problem, solves it at proof-of-concept scope with full-time staff, then tries to scale the win and set common standards. A dojo exists to teach: teams rotate through, build capability on real work, and carry it home. The DevOps industry standardized on dojos.
How do you build DataOps skills when DataOps engineers are hard to hire?
Grow them internally on real projects. Skilled DataOps engineers are scarce and expensive to recruit, so enterprises build the capability organically inside each team, using a dojo or center of excellence as the training ground. Growing expertise in house is hard without mentorship, which is why many teams pair it with outside coaching and a first project chosen for a quick, visible win.
