Well, I didn’t expect the reaction I got when I was talking to a load of QA managers recently about how checking isn’t testing… it reminded me of how I reacted to it years ago when I was first fronted with the concept. I remember struggling and thinking “No, testing and checking are the same thing”. but I soon realised after doing some further research that I was wrong in my assumptions, given the explanations I found throughout the wider testing community online. So, Why is this so contentious, even now, that many years later?
It’s interesting to note that, after mentioning this to a fellow tester that I used to work with, their response was “Oh God. I hate this! It’s such a painful thing!”.
A common understanding, we do not have…
Please, allow me a slight tangent, if you would.
Whilst I do not agree with testers being required to have some sort of certification to undertake their roles, specifically, I do hasten to add that the one thing that these can provide us is a common language/understanding that we can all use and get behind to talk about testing as a profession. A lot of the testing methodologies provided in something like the ISTQB/ASQF/CTFL ISO-29119 are far too prescriptive and don’t allow for the tester to really be creative or really think about what testing actually is. It all comes down to “checking”!
The ISO is currently available for £188 British Pounds – making it really accessible to everyone… </sarcasm>
Can anyone say “Happy Path”?
Testing Vs Checking
Checking:
To put it into the same context as the creators of the concept: checking is confirmation of existing beliefs. This would be something like confirming that the requirements are met according to the assumptions of the thing being developed. As Callum says in his recent LinkedIn Post about the same thing:
Checking (testing the requirements) gives us our baseline of “does this work”, then additional exploration and testing can tell us “is this good?”
Callums post on LinkedIn can be found here
Testing:
Testing is Exploration and Learning. You don’t just want to “check” the requirements, but you want to explore outside of those and see what else you might notice about the system under test, whilst having the new changes applied. This is a really creative task, unlike checking, in that you have to think about all of the potential things that could be affected by the features, designs, behaviours, etc. you’re making changes to. Also, you might do testing outside of, or before, any changes are made.
With this type of testing, you want to find out more about the way that the system interacts with the users as it is, and not just say “It does what it was intended to do”, but to learn more about the experience that the system is providing at the moment. Testing can be subjective, whereas checking/automation is objective; giving a clear “yes” or “no” based on pre-defined assertions.
Whilst carrying out further research around this argument, I’ve also been reminded of a definition of testing by John Stevenson; someone I had the pleasure of working with only briefly, but they changed my outlook on testing forever. John defines testing as this:
A test is a series of experiments performed against a theory that evolves based upon observational and behavioral information uncovered by the test.
This is from Johns blog here from back in 2017.
With that in mind, this isn’t to say that one is better than the other, as both have their place, as long as they’re being utilised for what they’re best at.
I will also say this isn’t merely semantics either.
N.B. I’m not talking in terms of AI with this either. I’m also leaving out the further discussion around Human checking vs Machine Checking, as these are different, but would require a lot further explanation.
For further reading please see Michael Bolton’s post from Aug 2009 here and James Bach’s updated post is here
/B