Testing Taxonomy

Testing Taxonomy

Over the next couple of weeks I plan on posting about the various types of testing and how to best apply AQA to them (or how to avoid providing AQA for them). Before I dove into that though, I wanted to give a brief primer on testing terminology. Testing, or QA, as a field has evolved over time and has adapted to many changes in technology and processes. QA is a humanistic field, there’s not a lot of hard science behind it’s methodology so it’s difficult to define at times.

But, we humans love labeling things, putting things in little boxes, and saying ‘this is a thing that is specific and definable’. A quick google search can find you some lovely articles, such as ‘List of 100 Different Testing Types’ or you can go down the rabbit hole that is Wikipedia’s Software Testing Portal. Don't envy Biologists.

I do find that these more complex taxonomies are waning in favor of a more generic approach to the problem. Most lists you’ll find today contain about 8 or so different types of testing generally broken down by level:

  • Unit
    • The smallest level of testing, focuses on a single unit (often single function or method) within the code.
  • Integration
    • The gold-standard of modern testing, this is usually driven by an API layer.
  • End-to-End
    • Tests multiple systems in concert. Often is combined with UI testing.
  • Front-End
    • Tests the front end such as form validation and user flows.
  • UI
    • Similar to front-end, but with more focus on the UI - look and feel, browser compatibility, and accessibility.
  • Acceptance
    • Testing vs the contract or with the end-client or end-user.
  • Environment
    • Tests the software on multiple different operating systems or environments (if applicable). Often includes installations, drivers, etc. Sometimes called ‘System Testing’, I prefer Environment as to not get it confused with the old concept of System-Level Testing.
  • Performance
    • Tests the speed of the system, server response times, load times, etc.
  • Security
    • Gaining in popularity these days, this pokes the software for any security flaws or back doors.
  • Exploratory
    • Handled exclusively by Quality Advocates (manual QA), this entails using the software as a human in a uniquely human way with a human eye.

Ultimately, it’s less important what you call it and more important that you’re testing the right thing. Each of the types given above are extremely broad. Once I was asked to go write ‘front-end tests’. A while later after the tests had been running, I found out they were also using those tests to test the queueing integration piece between our various systems. While the tests were capable of finding bugs in the queue system (submitting forms does, under certain circumstances, trigger a message to be sent to the queue), they were not designed for that purpose - I would be highly suspect of any bug they found there as the tests were literally wrong for the given system.

When someone asks you to go write ‘integration tests’ don't assume they know what it means or that you understand what they’re asking. Always ask what they mean by ‘integration test’ - what bugs are they trying to find and in what places do they expect to find them.

“Clear communication requires people to share definitions of the terms they use. If you’re not certain that you know what someone else means, ask them.”
— BBST
Unit Testing: Minimum Viable Test

Unit Testing: Minimum Viable Test

Vermont Code Camp 2018

Vermont Code Camp 2018