Automating Architectures for Rapid Delivery and Governance at Scale
Robert Sang, 20 Digital Architect, Vector Limited
Robert Sang, 20 Digital Architect, Vector Limited
Vector is New Zealand’s largest energy portfolio business. It operates businesses performing electricity and gas distribution, smart metering, LPG wholesaling, fiber connectivity and commercial new energy solutions.
When I joined Vector as Lead Digital Architect, I was excited to be a part of a talented team creating a new energy future. The business had ambitious plans, but we knew that with the diversity of Vector’s portfolio, we would have to find a way to manage a lot of complexity to safely deliver those plans.
The solution we landed on was to create software-defined reference architectures.
Scale and Complexity
The adoption of Agile methodologies and cloud computing over the last decade has seen organizations drastically increase the speed at which they can deliver change. Automated delivery pipelines can now deploy changes to production multiple times daily while the scope of those changes has also grown through infrastructure as code, pipelines as code, software-defined networking, and automated testing.
For many large enterprises, the new pace of change has outstripped their ability to effectively govern that change, leading to an explosion in complexity. That complexity brings with it unexpected costs, project delays due to incompatible technologies, and increased cybersecurity risks.
By combining new technologies with techniques borrowed from the world of manufacturing – such as Kanban and Lean – companies have been able to optimize for both speed and cost of delivery. However, it is the responsibility of Enterprise Architects to ensure that the changes being delivered are correct. They must be fit for purpose, aligned to business strategy, cost-effective, and safe.
Architecture as Code
Traditionally, Enterprise Architects have used reference architectures to manage complexity by defining known solutions to known problems. However, maintaining a reference architecture catalog and managing compliance with it can be time-consuming. This either causes a bottleneck in the delivery process or is bypassed by it, negating its value.
To overcome this problem, we examined how we could create a reference architecture catalog that could somehow be automatically enforced. Our solution was to automate the reference architectures themselves.
To that end, we determined to create CI/CD pipelines that were not merely a part of the manufacturing process, but which were a convergence of the engineering and architecture processes. In collaboration with our software development team, we designed a “pipeline of pipelines” that would automatically create software and infrastructure deployment pipelines that always conformed to our architectures. We called the templates for these pipelines Reference Architecture Templates (RATs),and the pipeline of pipelines irreverently became known as the RAT Pipe.
Reference Architecture Templates
A RAT is a set of scripts implemented using a tool like Ansible. The RAT Framework defines a lifecycle for the products built from RATs, from creation to destruction, and the scripts define the behavior of the pipelines for each of the lifecycle events.
There are two things that set this framework apart from other automation mechanisms; abstraction and immutability.
The level of abstraction for defining a RAT is the same as that for a reference architecture; it is a known solution to a known problem. Rather than being merely focused on the output of the automation(as a simple infrastructure-as-code script or CI/CD pipeline would be)a RAT is also concerned with how components interact, why we use them and constraining their behavior to what’s appropriate for the use case.
The pipelines and infrastructure built by RATs are immutable from the perspective of the developer or engineer, meaning that no individual product can diverge from the standard. If a change to the reference architecture is required, it can automatically be applied to all the instances.
Creating an instance of a RAT is a self-service action from the reference architecture catalog.
Seeing the Benefits
To make the most of our ability to deliver safely at speed, we conducted a business capability modeling exercise and constructed cross-functional teams to deliver and operate within the bounds of their specified set of capabilities.
Using the framework, these teams are free to operate without cumbersome delivery gates and architectural reviews. Empowered by automation, software developers within the teams have reported tasks that previously consumed two weeks can now be performed in fifteen minutes.
By harnessing this convergence of Agile, DevOps, and Enterprise Architecture, Vector has been able to grow its transformation program. We have scaled our digital capital plan from $17m in FY18 to $44m in FY20.
In November, Vector won global awards for Digital Architecture and Digital Governance at the ICMG Enterprise, Digital and IT Architecture Excellence awards, recognizing that the solution is not unique to our business or our industry.
Software-defined architectures extend the power of automation from the DevOps perspective into the architecture. They create feedback loops between development and architecture and maintain alignment from design through to outcomes.
In a world where changes come at ever-increasing speed, controlling and guiding that change must be a priority for every enterprise.
Weekly Brief
Read Also
