Top Factors Plaguing DevOps Evolution and Tips to Overcome the Challenges
Enterprise DevOps practices always need to evolve to align with organizational objectives and the needs of customers. Let’s look at some challenges that hold them back from evolving.
It is widely believed that automation and the use of public cloud enable organizations to evolve their Devops practices. Yet, a large number of organizations that have automated up to 90% of their tasks and use cloud services are still unable to fully evolve their DevOps approaches. Is organizational culture holding them back? Are they not using cloud platforms effectively? Or is there a communication gap between stream-aligned teams and platform teams? Let’s look at the major hindrances to DevOps evolution and how organizations can overcome these challenges.
To first understand where your organization is in the evolutionary process, you should first map out how much your DevOps practices need to evolve based on the sector in which it operates and the scale at which it functions. A middle-level organization that requires only a handful of internal applications to function may not need a highly-evolved Devops program to function optimally. An evolved Devops practice primarily enables efficiency- letting developers roll out applications, fixes and software updates at scale to keep the workflows running. The larger and diversified an organization, the greater is the need for a fluid and flexible DevOps practice that drives organizational efficiency.
What is DevOps Evolution?
Software solutions giant Synopsys recently explained four primary approaches to software development- the waterfall approach, the DevOps deployment methodology, rapid application development, and finally, the Agile methodology. Each new approach reflects a demand for shortening lead times, minimizing disruption, and optimizing product development processes. The need for offering more feature-rich applications and for those that function optimally across platforms and devices will drive further developments in the years ahead.
Once an organization adopts a DevOps practice based on its needs, the need to evolve it takes center stage, considering that consumer needs and demands change quickly, and so does the scale at which services have to be rolled out to meet those requirements. One of the most cited DevOps evolutionary approaches was laid out by software solutions company Puppet, and this foundational approach consists of five practices– monitoring and alerting configured by a team running the service, laying down tried-and-tested deployment patterns across teams, laying down tried-and-tested testing patterns across teams, enabling various teams to contribute towards improvements in tooling, process, and measurement, and managing configurations through a configuration management tool.
Learn More: Why Agile DevOps Is Now the Default Standard for Software Development
Top Factors Plaguing DevOps Evolution
Puppet’s State of DevOps Report 2021 highlights that many organizations have not been able to evolve their DevOps practices due to a combination of a number of factors. These are a lack of skilled practitioners, resistance to change, lack of automation, unclear goals or objectives, and the use of legacy architecture, among other factors. Let’s look at the top factors plaguing DevOps evolution that many organizations fail to take note of.
Automation and cloud support, but do not drive Devops efficiency
While it is true that automation and the use of cloud services facilitate the evolution of DevOps, they certainly do not drive it. Puppet found that as many as 62% of organizations that are in the mid-evolution stage have high levels of automation in place. The company advises that rather than relying solely on automation and the cloud to make their DevOps practices more efficient, organizations should also focus on enabling smooth communication between teams, enable teams to clarify their objectives, and address existing organizational aspects.
While running applications and their infrastructure in the cloud will give you programmable infrastructure and enable a more complete infrastructure-as-code approach, many organizations fail to exploit this advantage to reinvent existing processes, and to optimize for fast flow and low cognitive load, says Puppet. Organizations should automate systems configurations and automate provisioning to advance their DevOps capabilities. For instance, 79% of highly-evolved DevOps teams can provision computing capabilities, such as server time and network storage, as needed automatically compared to just 16% of poorly-evolved teams who are capable of doing the same.
A lack of effective tools to maximize efficiency
A survey carried out by Puppet revealed that at least 29% of mid-evolution teams are held back from transforming their DevOps practices by legacy architecture and 19% are held back by a lack of automation. The presence of a complicated technology stack and the use of inadequate technical tools also serve as barriers to scaling DevOps. Puppet says that highly-evolved teams have found a solution to these niggling issues by using internal platforms for their engineers of a large scale.
The use of internal platforms enables developers to seamlessly access authentication, container orchestration, service-to-service authentication, tracing and observability, and logging request services via self-service. These internal platforms are curated to reduce unnecessary interactions between teams and to enable optimal relationships with the consumer.
Organizational culture acting as an impediment and not an enabler
Organizational culture, based on how flexible and accommodating it is, can make or break a project no matter how much funds or resources are involved. Most DevOps teams that have so far failed to evolve have also failed to incorporate a culture of knowledge. These teams are found in organizations that have insufficient feedback loops, unclear responsibilities of teams and individuals, and whose employees fail to share best practices.
A DevOps team that has introduced automated testing and version control, hired and/or retrained teams, and is working to improve CI/CD pipelines may still struggle to evolve to the next level unless cultural issues are sorted out quickly. A successful DevOps team is one, according to Puppet, that does both the top-down and bottom-up work to build momentum behind its DevOps practices, creates knowledge and pattern sharing practices that enable fast flow optimization, and establishes productive change approval processes.
Under-representation of platform teams in the DevOps cycle
Possibly the most significant contribution towards the evolution of DevOps practices is made by the platform team. An effective internal platform helps reduce complexity in the technology stack, automate repetitive processes, and reduce unnecessary human interactions. They allow DevOps teams to access CI/CD workflows, public cloud infrastructure, monitoring, alerting, and observability, development environments, and internal infrastructure via self-service. It is, therefore, absolutely necessary to have a platform team in place that enables stream-aligned teams to accelerate delivery. However, a challenging organizational culture that is resistant to change, runs legacy architecture, or does not promote inter-team communications prevent platform teams from operating at their full potential.
Learn More: Face DevOps Challenges Head-on With a Redefined Approach to Application Monitoring
Top Tips to Make Your DevOps Program a Success
To gauge how an organization can effectively help its DevOps teams to evolve and scale their processes to meet emerging challenges, Toolbox spoke to Nigel Kersten, the Field CTO at Puppet. Kersten said that unless managed effectively, DevOps can be a prohibitively expensive challenge for organizations and DevOps projects can quickly go into a tailspin if inherent contradictions within organizations are not solved.
Devops evolved in the first place because having the right level of collaboration between developer teams and operations teams is actually the path to success and to align incentives. Normally, there exists a classic contradiction in the DevOps world where developers are incentivised for pushing changes and operations folks are incentivised for ensuring stability or in other words, saying no to changes. Therefore, Kersten said that the old school centralized IT paradigm needs to change.
Instead, to ensure the success of their DevOps projects, organizations should adopt the platform table approach. This involves the setting up of a platform engineering team that can build sub-services on top of an IT platform to enable developer teams to access these services via APIs and seamlessly make changes to the platform.
In a well-organized enterprise environment, you should have way more developers than people in IT and operations. Your platform teams should be able to serve hundreds and hundreds of value-stream teams if you’re providing a standardized and rationalized platform that serves most people’s use cases. You should give your developers the freedom to compose the solutions in different ways, and also provide suggestions. To let developers succeed, platform teams should closely collaborate with them, make sub-sub-solutions in their builds, and let developers offer suggestions in the form of code as code collaboration is a more efficient process than trying to describe in human language what a developer wants.
“Giving the freedom to the applications team to design and compose their own interfaces to the sub service is really key because different value-stream teams have different contexts they work in. It sounds really simple to implement, but there are a few pitfalls. You have to utilize this platform, it can’t be a top-down mandate,” says Kersten. He advocates that organizations must appoint a platform product manager who can reconstitute an IT platform for the benefit of developers working on top of the platform.
The platform product manager should also curate each service with a product mindset. This involves assuming that developers are their customers and are not bound to use all services offered to them. Hence, the manager should determine how to build compelling solutions, discover problems, prioritize them, and solve them. They should also evangelize the capabilities of their services, collaborate with users on the design, conduct early testing, and invest the right amount of collaboration in the design and discovery phase to enable the service to scale efficiently.
Learn More: Is It Time To Re-Examine the Role of Devops Engineers?
Is a Hybrid Work Environment Suitable for DevOps Projects?
According to Kersten, ever since the pandemic occurred, there was a lot of hesitation among enterprises around whether employees should have access to enterprise infrastructure when they are at home. However, they must keep in mind that the business risks of not allowing that access, and not being able to create change in your environment is much greater than the risk arising out of workers accessing things from home.
“I think the hybrid model is going to become the standard in the future. It is important to allow a distributed workforce, whether they are remote or not, whether they are in different countries, working in different time zones, that sort of a synchronous collaboration is necessary. Many organizations wouldn’t survive if their workers from different regions couldn’t collaborate,” he said. However, he added that there is an immense place for in-person high bandwidth collaboration. It is very difficult to replace the experience of ten people sitting in a room trying to find solutions to problems. Technology will enable collaboration to an extent, but may nor replace the value of high-bandwidth interaction which is critical for many initiatives to succeed. It’s much more difficult to practice when you’re online.
“Based on the inner source model, organizations should create interaction paradigms between teams. You should also create communities for practice to let users get together and discuss how using a platform is like and come up with ideas. There should be a strong communication channel connecting builders of a product with end users,” he said.
In conclusion: Are modern DevOps practices meeting the needs of organizations?
Kersten says that while it is relatively easy to create fully automated pipelines where developers can create code and deploy in the infrastructure, DevOps is in reality a lot more than just Dev and Ops. One of the biggest misreadings of DevOps is treating it as developer operations, as if it’s just the team that supports the CI/CD system. However, DevOps is much more than that. It is about creating efficient systems.
Organizations should break down silos, ensure smooth communication between teams, and implement the right level of collaboration by putting in place teams with well defined objectives. They should also promote lots of collaboration in the design phase, the definition phase, and should create ways for users to provide accurate feedback. There is also a huge scope in terms of policy-as-a-code and compliance-as-a-code, particularly around continuous compliance, Kersten adds.
Do you think DevOps teams are successful in evolving their practices quickly in response to the changing needs of end users? Comment below or let us know on LinkedIn, Twitter, or Facebook. We’d love to hear from you!
