Insights from the State of DevOps Report 2023
Data-Driven Insights and Practical Advice
The new state of DevOps Report is out, and it is full of insights and best practices for software delivery and operations. You can get the detailed report here. I highly recommend reading it if you want to improve your software delivery and operational performance. It provides data-driven insights and practical advice on how to develop the capabilities and culture that enable high performance. .
After a motivation why it is valuable to read the current report and even older reports, I list some quotes I have found interesting to share from the recent edition of the DORA report.
Why the DORA report is valuable for leaders and development team?
The DevOps Research and Assessment (DORA) report is a valuable resource for both leaders and development teams. It provides a comprehensive analysis of the state of DevOps, offering insights into the practices and capabilities that drive high performance in software development and delivery.
Identifying Best Practices: The DORA report identifies the best practices that lead to high performance in DevOps. These include continuous delivery, a strong focus on security, and a culture of learning and experimentation. By adopting these practices, development teams can improve their performance.
Insights into Trends: The DORA report provides insights into the latest trends in DevOps. This can help leaders to stay ahead of the curve and make informed decisions about the direction of their DevOps initiatives.
Evidence-Based Decision-Making: The DORA report is based on rigorous research and data analysis. This makes it a reliable source of information for evidence-based decision making. Leaders can use the insights from the DORA report to justify investments in DevOps and to guide their strategies.
Improving Collaboration: The DORA report emphasizes the importance of collaboration between development and operations teams. By fostering a culture of collaboration, organizations can break down silos and improve the speed and quality of software delivery.
Benchmarking Performance: While the DORA report can be used as a benchmark for organizations to measure their DevOps performance against. It categorizes organizations into elite, high, medium, and low performers based on metrics like deployment frequency, lead time for changes, time to restore service, reliability and change failure rate. This allows leaders and teams to understand where they stand in comparison to industry standards. But be carful with benchmarking:
There are contextual differences. Your organization's context (industry, size, regulatory environment, etc.) may be significantly different from the average represented in the report.
You might be tempted to compare your organization's performance directly to the highest performers in the report, without considering the differences in context or the journey those organizations took to get there.
Much more important than benchmarking is to focus on continuous improvement and learning, rather than just trying to match or exceed a benchmark.
While the DORA report can provide valuable insights, it's important to use it thoughtfully and in the context of your organization's unique situation and goals.
Why it can be valuable to read old editions of the DORA report?
Each DORA report focuses on different aspects of DevOps. Reading older editions can provide a more comprehensive and in-depth understanding of the field. In conclusion, while the latest DORA report provides the most up-to-date insights, older editions can also offer valuable information and context. By studying these reports, organizations can gain a deeper understanding of DevOps and how to improve their practices.
Categorized Quotes from the DORA report 2023
The following quotes are categorized based on the topics of the DORA report 2023. The quotes are not in the same order as in the report. I have selected the quotes, which I found most interesting to share.
Continuous Improvement and Metrics
Context is Key: Improvement work is never done. Find a bottleneck in your system, address it, and repeat the process. The most important comparisons are from looking at the same application over time, not by looking at other applications, organizations, or industries.
Fixating on performance metrics can lead to ineffective behaviors. Investing in capabilities and learning is a better way to enable success. Teams that learn the most improve the most.
Setting metrics as a goal. Ignoring Goodhart’s law and making broad statements like “every application must demonstrate ‘elite’ performance by year’s end” increases the likelihood that teams will try to game the metrics.
User Focus
User Centricity: Teams that focus on the needs of users are better equipped to build the right thing. Combining user focus with software delivery performance and operations performance means those teams are also equipped to build the thing right.
Platform engineering teams might adopt a “build it and they will come” approach to building out a platform. A more successful approach might be in treating developers as users of their platform. This shift in focus requires platform engineering teams to understand how developers work today to successfully identify and eliminate areas of friction. Teams can use the software delivery and operational performance measures as signals to monitor whether platform efforts are helping teams achieve better results
Build your team’s performance by adopting user-focus capabilities like customer feedback, visibility of work in the value stream, working in small batches, and team experimentation.
Technical Practices
This year, we investigated how the following technical capabilities predict performance:
Artificial intelligence
Trunk-based development
Loosely coupled architecture
Continuous integration
Rapid code review
When teams have the autonomy to improve and maintain a reliable system that delivers value to their users, they experience improved job satisfaction, team performance, and software delivery performance.
Architecture plays a significant role in a team’s ability to focus on the user and improve their software delivery. By starting small and focusing on the user, teams saw significant improvements across trunk-based development, loosely coupled architecture, continuous integration, continuous delivery, and SRE. To improve your technical capabilities, provide opportunities for team experimentation and continuous improvement.
To keep our software in that golden, releasable state, we need to work to establish fast feedback and recover from failures very quickly
The metrics of Stability (change failure rate and failed deployment recovery time) are all about quality, and the metrics of Throughput (change lead time and deployment frequency) are all about feedback and ease of detection of any problem.
Culture
We know that culture drives success. But what drives culture? This is an interesting question with everyone’s favorite answer: It depends! From a practitioner perspective, improving the way you work day-to-day has a positive impact on cultural elements such as sharing risk, increasing cooperation, and establishing psychological safety. For example, regularly integrating changes into the main branch of the version control system increases knowledge sharing and collaboration. Having security teams work alongside developers, collaborating on policy-as-code, increases trust between the teams and confidence in the changes being deployed.
From a leadership perspective, culture starts with awareness and education on the importance of culture. Transformational leadership can help foster a blameless environment that encourages experimentation and learning, and gives trust and voice to the practitioners. Engineers are there to solve complex problems, not just respond to task requests. In order to do this, they need visibility into the business and the autonomy to take action. Ultimately, culture is downstream from leadership. Ideally, the best results come from looking at culture from both a top-down and a bottom-up perspective
AI
In the current state of DevOps report 2023, AI does not show yet a positive contribution to team success. But there are lot of changes with generative AI, which are probably not incorporated yet, since many companies are in exploration phase. Insights:
The contribution of AI tools to team success is pretty low compared to other practices (CI, Code Review practices, Loosely Coupled Architecture, Maintainable Code...)
Documentation
Quality documentation is foundational. It drives the successful implementation of technical capabilities and amplifies the impact those capabilities have on organizational performance. Documentation also has a positive impact on outcomes, such as team performance, productivity, and job satisfaction.
Creating high-quality documentation requires teams to decide on processes in the first place. Documentation can force teams across an organization to explicitly discuss and get aligned on what to do and how to do it.
Quality documentation also acts as a repository for team knowledge, even as people come and go. It helps knowledge scale, both throughout the organization and through time
Reliability unlocks performance
Reliability is a widely used term in the IT operations space. We define reliability as the extent to which a service meets its stated goals for measures like availability, performance, and correctness. A common approach to achieve reliability outcomes is SRE.
Reliability practices
We asked respondents to think about reliability by having them think through three essential aspects of their operations. First, do they have mitigation plans for their dependencies? Second, do they regularly test their disaster recovery plans through either simulated disruptions, practical failovers, or table-top exercises? Finally, when they miss their reliability targets, do they perform improvement work or otherwise reprioritize and adjust their work?
Reliability and Well-being
On-call alerts for things that don’t impact users’ experiences, repetitive manual tasks, fear of making mistakes, and similar experiences lead to burnout and poor well-being for individuals on the team. We see the opposite in teams that leverage reliability practices. Teams report higher productivity and job satisfaction and lower levels of burnout than their counterparts who are not using these practices. We suspect that these improvements in well-being are driven by some published SRE practices:
Reducing toil
Blameless postmortems
Team autonomy
Sublinear scaling of teams
Moreover, high-performing software delivery teams won’t achieve very high team performance and organizational performance without also achieving high operational performance. Both are needed. In fact, teams that improve their software delivery performance without corresponding levels of operational performance end up having worse organizational outcomes. So, if you can quickly write stunning software, but it fails to run in production in a way that meets its audience’s expectations, there won’t be any reward from the marketplace.
But above all, SRE teams stick to their principles: embracing risk, measuring service levels, eliminating toil, embracing automation, and striving for simplicity.
Flexible infrastructure is key to success
Throughout much of DORA’s research, we’ve asked practitioners about their infrastructure by focusing on the essential characteristics of cloud computing— as defined by the National Institute of Standardsand Technology (NIST):
On-demand self-service
Broad network access
Resource pooling
Rapid elasticity
Measured service
We have consistently seen that these five characteristics predict improved organizational performance and improved software delivery performance
In fact, we see strong indicators that public cloud leads to decreased software and operational performance unless teams make use of flexible infrastructure. This finding further promotes the idea that simply “lifting and shifting” (the act of shifting workloads from a data center to the cloud) is not beneficial and can be detrimental.
None of this works without investing in culture
Culture is a key driver of employees’ well-being and organizational performance. A healthy culture can help reduce burnout, increase productivity, and increase job satisfaction. It also leads to meaningful increases in organizational performance, in software delivery and operational performance, and in team performance. A healthy organizational culture can help teams be more successful at implementing technical capabilities associated with improved outcomes.
The lines between cultural aspects, process capabilities, and technical capabilities are not always clear. We believe that culture emerges from practices, and practices emerge from culture.
Organizations can experience a cascade of benefits when they put the user first. User feedback helps teams prioritize projects and helps them create products and services that meet user needs. This leads to a better user experience, increased user satisfaction, and increased revenue
When information flows easily, things get done. We found that higher levels of information sharing were associated with increased software delivery performance and operational performance. When information is readily accessible and when there are few knowledge silos, people can spend time on tasks that matter instead of chasing information needed to perform those tasks.
Finally, flexible work arrangements, where employees can determine when, where, and how they work, have a beneficial impact across all performance metrics. This is particularly true for software delivery performance. Even as organizations tighten their remote-work policies, allowing employees to maintain some flexibility is likely to have a benefit.
Culture emerges from practices, and practices emerge from culture
For example, leaders can create incentive structures that promote a generative culture. Both leaders and individual contributors can emphasize a user-centered approach to software development. Individual contributors can help drive the implementation of technical capabilities that improve performance—trunk-based development, continuous integration, reliability practices, and loosely coupled architecture. Implementing these technical capabilities is not easy, and successfully doing so requires people to work together, to have an open mind, and to lean on and learn from each other. These are all components of a healthy culture. These teams can become examples for others within the organization, who might feel more empowered to drive change using whatever levers they have within their grasp. Long-lasting and meaningful changes to an organization’s culture come about through concurrent top-down and bottom-up efforts to enact change.
Healthy culture improves employee well-being
A healthy culture leads to high levels of employee well-being by reducing burnout, increasing job satisfaction, and increasing productivity. Employee well-being is not a nice-to-have: it’s foundational to an organization’s overall health and success. What happens when organizations don’t invest in a better culture? The likelihood of burnout increases, and job satisfaction decreases. Employees become cynical and their productivity declines. Their physical and psychological health are also negativelyimpacted.Burnout is persistent; it’s not something people get over after taking some time off. Burnout also increases turnover—employees leave to look for healthier work environments.Therefore, alleviating burnout requires organizational changes thataddress its causes.
Continuous Improvement
The most important takeaway from our years-long research program is that teams who adopt a mindset and practice of continuous improvement are able to achieve the best outcomes.
Improvement work is never done but can create long-term success for individuals, teams, and organizations. Leaders and practitioners share responsibility for driving this improvement work
Subscription: If you want to get updates, you can subscribe to the free newsletter:
Mark as not spam: : When you subscribe to the newsletter please do not forget to check your spam / junk folder. Make sure to "mark as not spam" in your email client and move it to your Inbox. Add the publication's Substack email address to your contact list. All posts will be sent from this address: ecosystem4engineering@substack.com.
❤️ Share it — The engineering ecosystem newsletter lives thanks to word of mouth. Share the article with someone to whom it might be useful! By forwarding the email or sharing it on social media.