Being a mid-level software engineer in a global enterprise, I've often pondered what initiatives would genuinely bring value to our software development lifecycle. This article is the beginning of a series where I'll share one such initiative
A Developer's Dilemma
Speed is one of my assets. I can write code efficiently, but the real hold-up? Deployment to the dev environment and subsequent manual testing. Multiple tasks often wait in the pipeline, all poised for deployment and verification.
The root of the problem? Our limited test coverage. With sparse unit tests, no integration tests, and only a handful of end-to-end tests, confidence in merging PRs is invariably low. This lack of confidence translates to a substantial amount of manual testing, time drain, and an overall suboptimal developer experience.
Taking the First Step: Analyzing the Problem
Although we never generated a coverage report, I had a feeling that the test coverage would be fairly low, so I decided to verify that feeling.
I generate a test coverage report for our services. As suspected, we stood at around 30% coverage, with some services completely uncovered.
Now, while raw test coverage numbers might not always be the final word, they do emphasize a point here. A lack of valuable tests correlates to a dip in confidence, leading to extended manual testing hours.
Increasing test coverage is the obvious solution. However, it's more than just about the numbers. The real objective? Bolstering the confidence of developers during PR merges, and ensuring efficient workflows.
Why It Matters to the Business
The link between low test coverage and extended manual testing times directly impacts our delivery speed, or in technical terms, our team's velocity. And, velocity is something businesses definitely value.
To gauge the potential impact of improved test coverage, I looked hard for a KPI I could easily measure.
I've chosen to monitor 'commit frequency' as a key metric. My hypothesis: as test coverage grows and developers gain confidence, our commit frequency, particularly on the main service, will rise. Increased commit frequency can be equated to enhanced team velocity, translating to faster quality deliveries.
Conclusion
Identifying a challenge is only the beginning. Pairing it with a meaningful metric that aligns with business objectives is where the real value lies. For instance:
"We lacked tests, so I added a few." - Only scratches the surface.
"Our test coverage was minimal; I've augmented it by 20%." - Provides data but lacks business context.
"We were hesitant with PR merges due to insufficient tests, resulting in delays. With improved testing, our team's delivery rate improved by 5%, as evidenced by these metrics." - Aligns the challenge, solution, and business value seamlessly.
What's Next
I intend to expand this topic into a series, sharing insights and updates about our journey to enhance test coverage. Follow along as I document this challenge and our strides toward its resolution.
A Note to the Reader
While I share my experiences and findings, remember, every organization and team is unique. I encourage open discussions and welcome insights from others who've walked a similar path.