Sometime back I worked with a tester who moved from a traditional team to an agile team. These are some of the thoughts I communicated with him. This was a small team and developers wrote unit tests and API based automation.
I often find testers are confused in agile teams. They focus on stories and acceptance tests . Instead, I would recommend the approach here to a tester on an agile team. This doesn’t change if the tester uses automation. However, in my experience very few testers can balance test automation and exploration based testing .
Don’t focus on validating what is written down as a story. Don’t focus on validating acceptance tests. Of course, you will cover those while testing (Michael Bolton). Think about what can go wrong. Think about questions that you can ask. It isn’t just questions that might occur to you. You need to figure out how to come up with good questions. Think about what you can explore.
What if the OTP for verification is not received? What does the user do? What if there is a delay in receiving the OTP and the user requests another OTP? Are they both valid? Does the user know that the account number has a 0 (zero) instead of o(oh) ?
These are the main deliverables from your testing —
- what questions[3,4] do I have about the software, and
- what can I explore.  (Here is a good resource to understand how to come up with ideas to explore.)
For every story you should have a list of what you think can go wrong, open ended questions as well as ideas to explore.
When you provide your scrum updates, you don’t need to talk about everything you have done. As a tester, you should evaluate what you want to do continuously. Keep an ear to the ground. Think about new features which you aren’t familiar with. What are the risky features? Every day, create a prioritized list. This can include the stories assigned to you, but should also include exploration or learning. e.g., ‘..there is a complex UI for the search page. I need to make sure I understand it…’, or, ‘…I know the code submission has potential vulnerabilities. I need to spend time on it…’
Talk to stakeholders or customers (In this project we had limited access to customers). If you don’t have access to a customer, you are the proxy for the customer. If you do have access to a customer, you are the proxy for the customer. The point is that you don’t wait for a customer, you must always think like a customer (while using other testing techniques).
You should make sure you create scenarios for testing. This also helps developers test. Scenarios are different from user stories. Scenarios are broader in scope. When writing scenarios use personas. e.g.,
Mike Smith is an administrator in Pacific Heights Bank. He needs to fax documents to Acme SaaS. Mike prepares a set of documents for four accounts. There are 25 pages in the set. After sending the fax Mike wants to make sure the documents have been received. He logs into the system…..
Using scenarios will force testers/developers/BAs/designers to manually use the system. This is but one testing technique among many.
You should make sure you can easily create realistic test data in the system. You should work with realistic data sets. You should have scripts to create test data. You should make sure you create large amount of data. You don’t need to create thousands of users. However, 10 users is better than 2 and 50 users is even better.
You should record configuration for tests. This may include operating system, browser. Use your judgment on which configuration is more risky and hence more important.
You should be familiar with tests written by developers on the team, i.e., API tests and any automation.
This is not meant to be an exhaustive list. This is just to realign how you think about testing. The ideas described here are a starting point for testing.
1. Acceptance tests are a very common idea in agile. I think it is a serious mistake.
2. All testing involves exploration. Exploratory testing is not a category or type of testing. The idea that some testing is not exploratory is a mistake. Automation as commonly practiced has nothing to do with testing.
3. It’s very important to ask questions and not make statements. It’s difficult to ask interesting questions. It is much easier to make general statements, reflecting the user stories or requirements (I don’t think the word ‘requirements’ is appropriate in agile teams. However, it is used very often.)
4. At this time I did not cover, ‘so what’? The biggest hurdle at this time is for the tester to generate interesting questions.
5. At this time I don’t think there is value in getting into the idea that there are different objectives of testing. That can come later.
6. Resource for understanding how to generate ideas for exploration: http://www.thetesteye.com/papers/redgren_moreandbettertestideas.pdf
Related Guidelines for a test manager
If you manage a test team or coach a scrum team and want to follow testing as described, rather than what is normally labeled ‘agile testing’, here are some guidelines:
- Traditional testers (and most others including developers, BA, project managers) will always gravitate towards the functionality of a story. This can occupy a lot of time and it seems like you are busy. This also gets support from developers. You will need to actively work to make sure testers don’t spend all their time with validating the steps in the user story. This is the biggest battle I face with traditional testers as well as with testers who work in agile teams.
- When testers stray away from acceptance tests, developers or developers turned coaches will question why they are doing that. This is another battle for a test manager.
- If testers test features which are not complete, they need to be careful that they talk to developers and don’t log defects on unfinished features. This may again be met with, ‘…why are you testing that…’
- I have seen traditional testers sometimes face, ‘…why do they need access to…source code….’
- You, as manager, should be part of the daily scrum as a pig.
- If there is no lead or senior tester, you should review what testers are doing. You should have a regular debrief with testers. This is what first line managers do. It’s wonderful if developers actively take part in these debriefs.
- As a manager you must have the ability to help a tester if he can’t come up with questions or ideas for exploration.
- It maybe a challenge to get testability for features. This might get a lower priority compared to actual features. This will require some influencing.
These guidelines can also be followed by a scrum master or a development manager.