Retro-Automation

Retro-Automation

In a basics physics class, it’s often that you’ll use equations that only really work in spherical vacuums. This is because including variables for things like air resistance and imperfect shapes makes things harder and much more complicated. The models work well enough for most theoretical questions, however, and their relative simplicity makes them easier to understand - and therefore to teach. But the real world is not a spherical vacuum, and thus things will need to be more complicated - and messy.

In my last post, I talked about how to integrate AQA into development process. That’s all well and good when you’re starting with a V1 project where AQA can be part of the design and included from the beginning. I find that many programming books also take this approach - they talk about best practices but rarely offer a roadmap of how to implement those as part of existing processes. Just like frictionless vacuums, this is not realistic.

So the elephant in the room: how do you add AQA to an existing, legacy project?

The thought here is simple: you have an old project or an old version that you find yourself spending a lot of time maintaining manual test cases. Someone on the team suggests, hey, maybe we can automate some of these tests! You want to go back and add test automation to the project to cut down on that tedious overhead.

A warning. Understand that this will be a substantial effort. It will not be over quickly and you will not enjoy it. Any claims to the contrary are incorrect. I’m sorry if that hampers your enthusiasm, but it’s important to set expectations early.

Rule 1: Don't Automate the Mess!

This rule is absolute no matter the case. Clean up your existing processes before trying to automate anything, or at least make streamlining current processes the first task of the AQA - and give them broad authority to do so.

Best case scenario, your team has great harmony and good QA and Developer resources with a well-groomed backlog of features, a functioning test plan, and well maintained test cases. Worst case, you are amazed your source control doesn't literally catch on fire on a daily basis. Both of these cases will still require some work before AQA can be implemented.

Of course you can't expect everything to be shiny clean either. If you’re working on 20 person ‘scrum team’ - with 1 dedicated QA for a monolith project that ideally should be broken into 4 projects and 4 seperate teams - or if your project is heavily coupled with another teams project to the point the codebases are intertwined, that mess is just going to stay messy. It’s going to create problems for AQA, but it just is.

(Digression: At the opposite end, don't wait for - or expect - a reorganization from higher up to come down and magically solve all your teams problems before cleaning it up as best you can yourself. Some things may just stay a mess, but there’s always ways you can make it more efficient without needing to involve upper management. The key here is team ownership of the project, which is a large topic outside the scope of this post.)

Find an AQA Resource

First, you’ll need to procure some AQA resources and allot some of their time to these specific tests. Notice I said tests, not projects. We’re going to need to violate that team structure I talked about earlier: you probably don’t want to assign a dedicated AQA to a regression project.

This is for two reasons: First, AQA on a regression project is going to be slow. Most of the resources on the project are probably already straining with bug fixes and client feature patches, so they won't have a lot of time to assist the AQA, meaning roadblocks will happen frequently. Second, it’s out of respect for the AQA’s career. It’s unlikely there’s much career opportunity for them on such a project, and assigning them full time to it for a year is basically a year without much career development for them. Give them some room to grow, please (really, do this for all your resources - host an internal code camp, encourage on-the-job training, set aside a budget for classes or books, etc.).

Create a Test Plan

If you already have a test plan, it’s likely specific to manual QA. If you don't have a test plan, well, better late than never.

The AQA is going to need to sit down and pour over the backlog of manual test cases and/or work closely with existing manual QA. The goal here is identifying which ones make the most sense for automation. Remember, in an ideal scenario manual and automation would have broken down QA tasks between them at the start of the project, but since in this scenario automation didn't exist, it’s likely that manual QA has a backlog of test cases that could have been automated.

You’re going to spend a lot of time in analysis before AQA actually starts writing tests. Remember, you don’t just automate manual test cases. AQA works differently and finds different bugs in different places. The trick here is working with manual QA to find what can be automated while still ensuring the same or higher levels of coverage. You’re not just going to pull test cases off the manual QA backlog and throw code at them, you need to analyze the test cases and think, ‘gee, if we apply automation to the API layer, then the front-end test cases can be paired down and simplified’ or ‘if we create a script to do this, then these cases can be combined in manual testing’.

Don't think in terms of eliminating your manual QA. Think instead of how you can optimize it. Where does manual QA really need to spend their time on this project?

Start Small and Build Up... Or Not

Typically, in any AQA, I recommend starting small with integration tests and building up to front-end testing. If that’s possible for your regression project (e.g., if your project has APIs to test), than absolutely follow that same approach. If it does not, then you’ll need to work at the top - and probably only the top.

Start with small, happy-path testing. Creating a new user, logging in, etc. Build a framework. If you have existing frameworks for more current projects, don't expect them to easily port over. If you’re using Javascript frameworks on certain pages, avoid those pages as much as possible early on - web drivers do not play well with Javascript frameworks, especially older ones.

Take Your Time

If all this sounds like it’s going to take a while, that’s because it will. Adding AQA to a regression project is a commitment to increasing the stability and quality of that project. It is NOT a way to cut resources on that project. You may be able to cut resources eventually, but that is a side-effect, not a feature.

The Watchmen Problem

The Watchmen Problem

AQA Processes

AQA Processes