AQA Processes

AQA Processes

As noted, the literature on AQA is fairly sparse, and when it comes to processes the few tomes that do exists tend to disagree with each other - and sometimes with themselves. None of the Scrum or DevOps handbooks I have read had much to say on Test Automation (at best they say to ‘do it’ leaving both the ‘do’ and the ‘it’ vague). That said, I'm going to share my experiences and what I’ve pieced together for how to fit AQA as part of your processes.

Don't Automate the Mess!

 US Department of Defense Lifecycle Flowchart

US Department of Defense Lifecycle Flowchart

For those unaware, this is the actual US Department of Defense application lifecycle flowchart. Makes you feel safe just looking at it right?

If this flowchart closely matches your current QA or general development process, automation will not help you. As stated before, AQA is not magic and it is not a replacement for good manual QA. Injecting AQA into an existing messy system will only make it worse. You will simply add another node on an already convoluted org chart. If you are frustrated with your current QA process, don’t turn to automation as a solution - don’t automate the mess.

I would take this one step further actually: don't automate your existing process. It’s well known to developers and project managers that you don't write an application that exactly mimics what the client is doing. Yes, what the client is doing would be faster if a computer did it by sheer nature of the speed of electrons - but that’s not the real power of software. With a computer you can streamline things, do things that would be impossible with a manual process. It gives you the opportunity to root out inefficiencies in processes that would be impossible to avoid in a system that involved humans - think threaded processes or performance testing.

Use AQA to build smarter processes, not faster ones.

Team Structure

There’s a lot of debate about how to structure AQA into a team. Some advocate for an independent team that handles all AQA (and/or QA) across the company, while some argue embedded resource as part of a scrum team working independently, with a whole rainbow inbetween.

In my experience, AQA should be embedded as part of a scrum team. Each team should have an embedded AQA resource that focuses on that teams projects, just the same as you would a Developer or QA.

But, AQA requires a lot of tooling and maintenance. You don’t want every team using a different framework for all their tests, it’s a lot of unnecessary overhead. So to combat that, AQA should have a regular, once a week or so, standing meeting to talk and discuss various problems and come up with solutions as a team. These are often called a Community of Practice and can be applied to all of your specialists. This should keep all of your automation congruent to a certain set of standards, and will also be vital when it comes to reporting metrics.

AQA Handoff

 And around and around we go...

And around and around we go...

I love this little piece of art because nowhere in this process are the developers or other stakeholders considered. It’s just like this infinite QA vacuum.

There’s debate about when to involve AQA in the development process. The answer is to involve them right when you would manual QA.

As discussed previously, manual and automated QA do different things - they are separate and equal. AQA needs to sit in next to manual QA and provide their own input for the testability of a feature. Test cases should be divided up between manual and automation. They should be treated as two different things, each getting its own set of test cases to work on.

Some things you’ll want to run manual tests on first and automate later - that’s perfectly fine. Other things, AQA simply won’t be able to test, or the cost would be too high for the payoff, and those will stay in the realm of manual QA - this is also fine. Involving both QA and AQA early in the process allows them to sort out what goes where. They will decide as a team where AQA fits into the project based on their resources and skills and the architecture of the product.

Sometimes there is an urge to run test cases through manual QA before handing them off to AQA. The general idea is that manual QA creates and vets all test cases, maintains the test plan, and runs through all test cases manually at least once before approving them for AQA. Generally this process happens if you don’t involve your AQA earlier in the development process or if you don’t feel your AQA resource has adequate QA knowledge to create their own test plans. I’m not going to say this is a bad process; sometimes it’s required. I would recommend avoiding this structure if possible. It creates an unnecessary bottleneck where AQA can be sitting waiting for manual QA to approve test cases, effectively restricting you from working in parallel. One great way to avoid this pitfall is by using behavior-driven development, especially when paired with a test management system or a paradigm such as Gherkin and Cucumber.

Retro-Automation

Retro-Automation

AQA Resources and References

AQA Resources and References