There is a reporting pattern I have seen repeated across teams, across companies, across industries.
After every sprint, every release cycle, every test run, someone pulls together a report. It usually has a table of numbers. Bugs filed. Bugs fixed. Test cases executed. Pass rate. And somewhere in that report, there is an implicit scorecard showing how many bugs did this QA find, and how many bugs did this developer produce.
On the surface, it looks like accountability. It looks like data-driven quality.
It is neither.
The Real Problem with How We Report
When we measure QA by the number of bugs filed, we are rewarding the wrong behavior. A QA who files a hundred low-severity cosmetic bugs looks more productive than one who catches the single critical defect that would have taken the system down in production.
When we measure developers by the number of bugs attributed to them, we are punishing the wrong thing. Bug count is not a measure of a developer’s skill or care. It is a measure of complexity, of requirements clarity, of how well the team collaborated upstream. Punishing someone for a high bug count does not make the product better. It makes people defensive, cautious, and less likely to take the risks that lead to innovation.
And then we take those same metrics: bug counts, test execution numbers, pass percentages and we present them to business stakeholders as the story of our quality. This kind of reporting practice can also easily backfire against the work quality of the team, as these numbers could easily work against the team’s measure of effectivity in ensuring the quality of the product under test.
Another additional note, the business stakeholder sitting in that meeting does not care how many bugs the QA filed this week instead, they care about one thing: can we release on time, and is it safe to do so?
What we give them is a rearview mirror. What they need is a decision enabler dashboard.
The bug measures mentioned above are what we call lagging indicators. They tell you what happened but do not tell you what it means, what the risk is, or what to do next.
Business stakeholders walk away from those reports without the one piece of information they actually needed, and they are left making release decisions based on gut feel instead of intelligence.
That is the problem I set out to solve.
What Is a Quality Intelligence Report
A Quality Intelligence Report is not a test results export. It is not a dashboard showing pass and fail counts. It is an interpreted, decision-ready summary of what your pipeline data actually means for the release.
The difference is interpretation.
Any test reporting tool can show you that 47 tests failed. A Quality Intelligence Report tells you that 38 of those failures were caused by environment instability and not the product. 6 were automation scripts that need updating, also not the product and 3 are potential product defects that a developer needs to investigate before you release.
That is a completely different conversation. And it is the conversation that business stakeholders actually need to have.
I built Releasure to make that conversation happen automatically, after every single pipeline run.
How Releasure Works
Releasure connects to your Azure DevOps CI/CD pipeline via the ADO REST API. After a recent build completes, it pulls the test run data, automatically removes duplicates, analyzes failures across parallel stages, and classifies each into one of three categories.
Infrastructure Noise — failures caused by the test environment itself. Network timeouts, element visibility issues, locator failures. These are not product problems. They are pipeline stability problems, and they should not block a release.
Automation Errors — failures caused by test scripts that need updating. Null pointer exceptions, script logic issues. Again, not product problems. They are technical debt in the automation layer that needs to be addressed in a refactoring sprint.
Potential Product Defects — failures where the product behaved differently than expected. These are the ones that matter. These are the ones that could affect your users. These are the only category that should trigger a block on release.
Once every failure is classified, Releasure calculates a Release Health Score out of 100, applies a weighted penalty formula based on the severity of what it found, and assigns a verdict:
- PASS — RELEASE READY
- AT RISK — REVIEW REQUIRED
- BLOCK — DO NOT RELEASE
Then it sends a sanitised summary to the Claude AI API, which generates a Release Intelligence narrative written in plain business language: no jargon, no technical terms, just a clear explanation of what the data means and what the team should do next.
The whole process runs quietly in the background for a matter of seconds, and then outputs single HTML report, ready to share and access to any one from the delivery team, and most importantly the business stakeholders.
Watch how Releasure works in action 👇
What the Report Tells a Decision Maker
The person opening this report is not an automation engineer. They are a Head of Product, a Head of Application, or a CTO. They have two minutes before their next meeting and they need to know if it is safe to ship.
The report gives them that answer immediately by using a lean format and meaningful leading indicators.
The verdict pill at the top with green, orange, or red colour codes tells them the status in half a second.
The Release Health Score gives them a confidence number.
The KPI scorecard shows them the shape of the build at a glance.
But the most important section is the Release Intelligence narrative. Because instead of reading “47 failed out of 200,” they read something like this:
“This build is safe to release. The 38 infrastructure failures reflect environment instability in the test pipeline, not product behavior. The 6 automation errors are known script issues that the team has queued for refactoring. The 3 potential defects detected have been investigated and confirmed as pre-existing known issues with no impact on critical user journeys. The product is behaving correctly.”
That is the difference between a test report and a Quality Intelligence Report. One shows data. The other supports a decision.
What Building This Taught Me
When I got Releasure running, it was a validation of something I have believed for a long time, and now finally had proof of.
For decades, QA has been framed as execution. Run the tests. File the bugs. Report the numbers. We have been treated as the last checkpoint before release and almost close to never, as strategic partners in the product journey.
When I built this tool, I was not executing. I was designing an intelligence system. I was thinking about what a Head of Product actually needs to make a confident release decision, and then building the thing that gives them exactly that. A report engine that speaks their language and ultimately solves their problem.
And that is where I believe our profession is going.
In the next five years, QA professionals will not just be test executors or automation builders. We will be the architects of our own test intelligence labs. We will build the tools and reports and systems that give business stakeholders exactly what they need, in the language they actually speak. We will move from lagging indicators to leading ones. From data dumps to decision support. From “here are the numbers” to “here is what you need to know.”
Our role is more critical today than it has ever been. And it is only going to become more central.
Because we are the ones who understand both sides. We understand the pipeline and the product. The technical and the business. The failure and what it means.
Now we can build tools that translate between those worlds automatically.
That is what Releasure is to me. Not just a script that generates a report. A proof of concept for what QA leadership looks like in the next chapter of our profession.
If you are a QA leader, Head of Product, or engineering manager who sees value in bringing this kind of intelligence to your release process, I would love to connect. Reach out to me on LinkedIn and let’s talk about what this could look like for your team.
Michelle Advincula is a QA Practice Lead and Strategist with 15+ years of experience building quality systems for distributed engineering teams. She holds ISTQB and Lean Six Sigma certifications and runs QA Foundation at qa-foundation.org.
Leave a Reply