If you are an operations or infrastructure type engineer, testing may mean traditional “QA”, UAT or the newer test automation. An alternate way to think about tests is that you are asking a question about the software or infrastructure. The question is open ended — there is no definitive answer. What will happen when the instance runs out of disk space? It will trigger an alert. Will it? (Why did the bedroom alarm clock not ring on the day of your important meeting?) If the alert is triggered, will the people who are paged know how to respond? If we have a separate operations team, will they understand the application, to trouble shoot the alert? What are the different situations when the instance can run out of space? Let’s call these type of questions tests.
There is another type of question for which we know the outcome. I verify that the new instance of Wordpress is working by logging into the application using username, ‘admin’ and password, ‘password’. I could use different types of usernames and passwords. Maybe try different lengths of strings and different characters. When you know the outcome of the question, let’s call these checks.
Whether it’s infrastructure or applications, I find it useful to think about tests and checks as distinct. Note that this is a thought experiment. This is a way of thinking. What, if anything, you do with this is up to you. When I test, I hold myself accountable for the questions (tests and checks) that I didn’t ask.
For your further thinking
- What is the value of thinking of checks and tests?
- How would you use this for infrastructure/DevOps?
- What tests do you use when commissioning a new server?
- Are Serverspec scripts, tests?
- How about Python unit tests (not related to infrastructure)?
- Are UAT scripts, tests?
- Are code reviews, tests?
- Is an AWS Well Architected Review a series of checks or are they tests?
- Is it easier to spot a check compared to a test?
- Are tests more common when working with infrastructure?
- Can you repeat tests? How about checks? Should you repeat tests?
- Can you write tests?
- Can you run tests?
Engineers who have a development background (as opposed to operations) will be familiar with unit tests and test automation. They may also be familiar with the idea of ‘checks’. Engineers from an infrastructure/hardware background (Ops-DevOps) may not be as familiar with tests and might find the idea of ‘checks’ useful. If you are a developer who is very familiar with unit tests and test automation, I recommend thinking of checks when you write your tests (checks?).
Tests and checks may not be binary, as described here. However, if you haven’t heard of checks, you can start with thinking of tests and checks as distinct. I’ll leave it as homework for you to figure out if they are distinct. I hope you don’t dismiss the concept just because they aren’t binary.
The concept of checks was proposed in 2009.
There is a strong school of thought which considers testing as critical thinking. A good starting point to know more is the Association of Software testing.
Originally published at https://www.linkedin.com.
