A tester doesn’t just compare what happens against the results expected. Testing is not a mindless job, though some parts should be automated. Testing is a creative job where you ponder on the best way on testing, what to test, and what not to test. This is a continuation from my previous post, focusing on how to generate test ideas, managing your thinking bias, and how to work more effectively.
A heuristic is a model that allows people to create solutions more effectively in a shorter amount of time. Using a single set of heuristics for a problem can not be guaranteed to be the most optimal, logical, or rational approach.
We know that there is an unlimited number of test cases in any given software release, this would be fine if we had all the time in the world, however we only have a finite amount of resources and time. In order to cope with the amount of testing required, a good tester gathers and uses a set of heuristics to generate tests very quickly. Here are a few examples of testing heuristics
- Input fields - Long name (255 characters), Special Character ( ! “ £ $ % ^ & * ( ), Blank field, Space at the start, Space in the end, Numbers, Decimals, Accented Characters
- Smoke Testing - verify that all critical functionalities are working fine before any detailed testing is done to find any glaring issues.
- Boundary Testing - specification based testing in which tests are performed using the boundary values. For example, to get a first class degree, you would need to get 70%. If you input to the system 69%, will you get a first class degree? If you get 71% would you? How about 69.99, or 70.01?
- Cross Browser Testing - With numerous numbers of web browsers available and users using different browsers, it has become more critical to test how each system performs in different browsers
- Documentation - Are the documents required for testing up-to-date? If not,they need to be updated
- Communication - How will we communicate the progress of the testing to our stakeholders?
Heuristics are just suggestions, blindly following heuristics that you don’t understand or does not apply to the current situation is not good practice. It is important to understand what the situation is, understand the reason as to why you’d use a heuristic for testing, and agree on the team with how testing should be approached.
Consider your bias
A bias is the fact of preferring a particular subject or thing in any given situation. For example, you go to a restaurant and your friend has asked you what the best food in the place is, you’d give your opinion and then they would probably go for what you’d recommend. However, it is very likely that you’ve not tried every single item in the menu and that you have certain preferences when it comes food (spiciness, sweetness, sourness).
Testing is not any different to how you make decisions on every single thing you do in life. You select tests with greater likelihood than other tests. If there is an input field, you’d probably just test if it can handle numbers and letters because you’d never think a user to use special characters. Here are a few examples:
- Selection Bias - This occurs when you are selecting a sample or the data you’ve used for testing is less complete. This may mean that you have not considered the use cases that your actual users would do, working within the specific mindset of the development team, a fatal flaw of the development process.
- Self-Selection Bias - A subset of selection bias. Are the tests that are going to be used decided by yourself? What if there is a better way of testing or you’ve selected tests that are useless? Talk to your team about the best way of testing and you’ll find yourself removing a few tests that are not required and adding tests that you’ve missed out
- Reporting Bias - This occurs when the direction of the results influences whether how the findings are reported. If there is a bug in the product, will you tell your manager that you’re happy with the release?
Bias can not be avoided but they can be managed. The tester needs to be aware that his thinking may not be the best way forward to a solution, the team needs to be consulted to minimise the impact of one person’s bias.
Work in bursts
Testing can be overwhelming at times. You are given a deadline to test when sometimes you don’t feel like it’s enough time. What’s worse is when priorities shift and you are required to test something else as a matter of urgency. When you are testing, do it in bursts. The human mind has the ability to adapt to complexity, but you can’t do everything at once.
What I personally do is get a timer, set it to 40 minutes. In this allotted time, I am focused on working. Once the 40 minutes is up, I stand up, take a walk, drink water and then continue with the testing.
Reinventing the wheel is an idiom which is invalidated by the very metaphor it uses. This is because the wheel is one of the things that have been reinvented so many times that we probably don’t know the answer to “How many times has the wheel been reinvented?” The materials have changed from stone, metal, wood. You might also think that wheel is circular? Not necessarily, have a look at pedrail wheels.
Many advances in technology are not just done by adding things on top of them, but by reiterating on it, coming up with something better for a given scenario. That’s why most software companies do not have a unified way of testing.
Similarly to how you start off learning a skill. You get ideas from multiple sources, you play with the ideas, then you use which ones are best suited for you and stop using the ones that are not helping you out. The good tester’s mind is always turned on, they continuously evaluate their ideas.