Agile software development radically changed the nature of software creation. One of the hallmarks of agile is the creation of small increments of end user functionality as opposed to creating software in one or few large portions. Agile also included many other innovations such as starting with confirmatory tests as opposed to creating tests after construction and creating self managed teams as opposed to traditional command and control. There was an earlier, parallel movement in understanding the design and creation of mechanical objects, including building architecture, product design and graphic communication. One of the major contributors to that movement was Bryan Lawson. In Lawson’s book, ‘ How Designers Think ‘[1], he explains the characteristics of design problems, solutions and the process. In this article I have explained Lawson’s ideas on design problems, solutions and the process. …
Good ideas in software testing endure, despite changes in technology and development approaches. In 1988, Cem Kaner first described a ‘defect catalog’ for software in his landmark book, Testing Computer Software. In this blog post I describe how to create a defect catalog for Infrastructure as Code as part of DevOps.
Kaner’s defect catalog was a list of common errors in software. He listed the different ways you can use such a list:
- Evaluate test materials developed for you by someone else.
- Developing your own tests.
- Generate hypotheses for bugs which are difficult to reproduce.
- Generate related ideas for unexpected bugs. …
There are different interpretations of the term, ‘DevOps’. However, in the DevOps values — CAMS (Culture, Automation, Measurement, Sharing), culture is considered the most important value.
The methodology begins by looking for the right people or person to fulfill a needed role. Secondly, ensure that role’s process is correct and optimal. Lastly, improve the process’ efficiency by reducing steps through some tool based automation. Notice, tools and automation are last!
People Over Process over Tools, Alex Honor.
Despite the focus on culture, collaboration and organization change, a critical part of DevOps is the enabling technology. There are different aspects of the technology. Continuous delivery automates the entire process of delivery, including building the software, validating the build and progressing through other steps in the form of a pipeline. A much more elaborate part of the technology is Infrastructure as code (IaC). Infrastructure as code was pioneered by Amazon Web Services (AWS). …
If you are an operations or infrastructure type engineer, testing may mean traditional “QA”, UAT or the newer test automation. An alternate way to think about tests is that you are asking a question about the software or infrastructure. The question is open ended — there is no definitive answer. What will happen when the instance runs out of disk space? It will trigger an alert. Will it? (Why did the bedroom alarm clock not ring on the day of your important meeting?) If the alert is triggered, will the people who are paged know how to respond? If we have a separate operations team, will they understand the application, to trouble shoot the alert? What are the different situations when the instance can run out of space? …
When testing API you use tools like Postman or write tests in Java, Python or another language. Reviewing defects reported by users, reveals critical techniques that you can use to test API. In this blog post, I review defects in a Python package to access Twitter, to get insights on how to test the API. There are useful lessons for testing API in general.
Twitter has it’s own Python package for accessing the API. The package I review here is not the official Twitter Python package.
When reviewing defects, I want to explicitly avoid ‘fixing’ the problem. When someone or a team looks at customer defects, all their energy goes into ‘fixing’ the problem. Instead, I will focus on what might have been difficult in anticipating that problem. I am also very aware of hindsight bias. When I encounter an issue, I explicitly think about how the learning could be applied broadly to other areas. …
What does it mean to test AI? If you look at the spam detection feature in your email, it sometimes works, less often doesn’t. Even if you flag items as not spam, some again show up in spam after some time. You could categorize that as a machine learning FAIL?
Amazon recommends books related to your purchase. Some of them make sense. Others don’t. Hopefully the algorithm learns and gets better over time. Are the incorrect recommendations a failure of the algorithm? Isn’t it expected that recommendations will learn and improve over time? So, failures are in fact not failures?
What does it really mean to test AI? …
Initial state of the system
Software Test Automation
When software testers first encounter agile, they can be intimidated by all the terminology and technology. A better way to understand testing is to answer a few fundamental questions. It’s a good idea to not accept any answers, even when they are widely adopted, at face value. You should try to find the answers for yourself and keep checking if there are alternative explanations.
Any discussion on testing in agile teams is dominated by test automation. The rationale for test automation is that it allows you to release software often, often many times a day. If you parse that sentence, what automation does is to make sure that what was working before continues to work. I don’t want to disparage automation. …
This is what goes on in the background of booking a cab. There is so much that can impact the outcome of using a software that makes software testing and development complex. Note that Uber is very different from other software in that it reaches out into people’s lives. Technical software may not have such an impact.
Mom calls at 8 am on Sunday.
She wants me to book a cab for her.
She sent a whatsapp message, since my bedroom doesn’t get good mobile reception (SMS/calls sometimes don’t come through)
We don’t live together (her idea).
I had a bad night yesterday and didn’t sleep well. …
Leaving aside all jargon, a key skill in software testing is the ability to ask questions. Before qualifying the type of questions, it’s a good idea to try asking questions without any restrictions. For most people, it’s much easier to ask questions on a given subject or on a specific object. The software requirements or preferably a user story provides such a context. However, it is a good idea to not get too caught up in user stories when coming up with questions. Rather than being restricted by the user story, it’s better to be able to generate questions and then check the relevance to the story. It’s also important to ask questions which might not have occurred to anyone, including the person/s who wrote the user story (You don’t have to treat the product owner as all-knowing). …
About
