Harnessing the Wind of Change:
OVO’s Journey to Continuous Delivery with Test Automation
Published: January 2024
Client Name: OVO Energy
Industry: Energy supplier. Gas and electricity provider.
Executive Summary
OVO, like all energy suppliers in the UK, needs to comply with a complex set of regulatory requirements (Ofgem) for paying energy generating consumers. Therefore, Quality Assurance plays an important role within all development teams at OVO. Irysan worked with OVO to improve the standards of Quality Assurance and Test Automation within the Exports project - a platform which enables OVO’s consumers to upload their energy generation and export reads and be paid for that energy.
After the initial assessment, Irysan created a map of challenges in the delivery team. Then, working with the client, we collaboratively developed a high level plan of addressing the identified challenges, with a focus on redesigning the QA approach towards Shift-Left Methodology and placing test automation right at the heart of the development lifecycle. Moreover, we developed a tailored QA framework which focused on all levels of testing at various stages of development.
Client Background
Founded in 2009, OVO Energy has grown to become the second-largest energy supply company in the United Kingdom, with around 5 million customers and over 8,000 employees and an annual revenue of over £5 billion (2022). OVO Energy is regarded for its innovative approach and commitment to sustainability: specifically, for the dedication to promoting green and renewable energy sources and the emphasis on a reduced carbon footprint. The company advocates and implements smart metre technology to empower consumers with real-time energy usage insights and improve energy efficiency. OVO Energy's focus on sustainability, innovative solutions, and customer satisfaction has positioned it as a notable player in the global energy market.
Challenges
Some of the challenges which our team faced during the implementation of the solution, included:
Almost non-existent automated testing.
The automation tests which did exist, depended on environment-specific test data, making them inconsistent and unreliable.
No clear segregation of QA responsibilities between developers and testers.
Inconsistent test coverage, implemented by developers on an ad-hoc basis in the absence of a reliable QA function.
Code releases to production were performed without Quality Assurance.
Additionally, as often happens in teams, QA was not an integral part of the development culture, and the success of this project hinged on Irysan’s ability to work effectively with client’s teams, build consensus and achieve collective buy in into the new solution, without compromising on quality.
The solution
Irysan QA team worked with OVO’s delivery team to design a QA function, focused on efficiency and reliability. In this process, we have implemented the following:
Introduced Shift Left QA methodology and embedded it within the client’s team;
Created a new QA framework;
Built automated tests for ongoing development;
Added automated regression tests for existing features;
Implemented test stages as a part of Continuous Integration and Delivery;
We introduced a Shift Left methodology that focuses on earliest QA intervention. We started with QA involvement at the earliest point of requirement analysis, which was followed by writing tests alongside developers during feature development, as opposed to adding tests after feature completion.
A QA framework was developed that started with domain mapping of the application within the framework to assist with test data generation. This allowed tests to be environment independent and more robust when it came to writing tests for new features. The framework focused on ease of test delivery and maintenance right from the start.
Putting a new QA framework in place meant managing demand for QA coverage across the application as efficiently as possible. Since retrofitting test coverage for existing features is always a challenge, we adopted the approach of prioritising ongoing and new development. This allowed us to ensure that no new code was being pushed out without thorough QA checks - which was both a process change and a significant cultural shift within the team. Alongside this, we created a backlog of tickets to fill in testing gaps for the existing features.
Irysan QA team worked with the client’s in-house Platform team to embed automated testing into the CI/CD pipelines, so that any defects could be detected before the code was allowed to be released into production.
The outcomes achieved
Upon completion of the project, the client received a robust automation-based QA framework, which is now at the centre of feature development. This enables early detection of defects, improves team efficiency and reduces the risk of production failures.
Comprehensive end-to-end test coverage drastically reduces the need for manual QA intervention during the rollout of new releases.
These improvements also enable a more focused developer effort with confidence in the QA team, which in turn, improves overall team performance.
Results and Benefits
Initially the result of Irysan’s intervention was that OVO teams started noticing an increased volume of defects, which previously went undiscovered. In the process, we also made “unexpected” discoveries, such as gaps in developer documentation and previously unknown API errors - this helped engineering teams prioritise technical debt reduction alongside feature development.
After a couple of sprints of work, the code quality visibly improved and as priority defects were being fixed collaboratively by testers and developers, the QA function became deeply embedded into the client’s team structure and culture, which enabled OVO to more effectively serve their customers and remain compliant with Ofgem requirements.
With a strong QA foundation now in place, the Irysan team is focused on taking it to the next stage, which focuses on:
writing automated tests for any inflight work
adding tests for existing features that don’t have coverage
building large e2e tests that test longer journeys in a single test to be run for releases in a shared environment
Running the whole regression suite before a release in a dedicated environment for automated tests.
Building Slack webhooks to alert teams when a release candidate has failing tests.
We will discuss the challenges and outcomes of the above in the next case study.
Conclusion
Irysan QA team started with the analysis of the delivery challenges at OVO and ensured that our solution catered to all these challenges, as opposed to applying standard ‘best practice’ blindly, or throwing the latest technology at the problem. Once the key challenges were understood by the whole team, we designed a robust QA framework that is easy to maintain and is scalable and importantly, one that addresses the particular challenges of our client, producing the best client outcome.
The testing is more efficient and most importantly, it plays a critical role within the team. The challenges which we encountered along the way, helped us refine the final solution.
While we continue our collaborative journey of QA transformation, this study demonstrates the benefit of investing in dedicated QA experts.
While some companies adopt a “no QA and engineers writing all tests” approach, this would only work for small companies and simple applications.
In any nontrivial environment, a dedicated QA function and automation-focused team culture reduces risks, improves confidence in releases and increases velocity and performance in engineering teams.
Contact us today to discuss how Irysan’s QA expertise can benefit your organisation.
