Sometimes in software testing it’s hard to see the wood for the trees. As the whole discipline has grown so much in scope and detail, different ways of segmenting testing have been created. In a top-down approach, some divide testing into methodologies associated with the development methods such as agile testing and waterfall testing. Others take a bottom-up software testing approach, starting with unit testing and building to integration testing, system testing, and acceptance testing. Different approaches bring different insights; here’s another way of looking at it.
A Software Quality Slice
Neither top-down nor bottom-up, the model we discuss here is based on six facets of software that are important for making quality software. They are functionality, compatibility, robustness, performance, scalability and usability. It’s a model that should supplement others, not supplant them. It also allows the possibilities of automated testing to be positioned in five out the six areas discussed.
Use cases (user scenarios) are an important tool for testing software functionality. They can be defined as scripts that can be automated, run on their own and also reproduced for post-test analysis or regression testing. Testing Anywhere allows testers to record whole sequences of user actions with an inbuilt recorder. The advantages for software testing include better approximation of real-life human user behavior and faster test script generation. For programmatic interface testing for application to application working however, building the scripts from individual action building blocks also works.
Compatibility across Platforms
Specific versions of a software application may need to be produced (ported) to different operating systems. Even with commonly used programming languages, different bugs will need correction according to the specific operating system. These cases may be viewed as complete test projects in their own right. Applications written in Java should run in the same way no matter what the underlying platform is, because of the underlying Java virtual machine. However, as the good software tester knows, ‘should’ is not equivalent to ‘will’. A similar situation exists with web enabled applications, where cross-browser testing is essential, even if everybody claims to conform to standards. Test Automation can be used in these situations as well.
This may extend to physically switching off systems or subsystems in mid-operation to check on overall effects on the application and its continuation or recovery afterwards. The increasing use of virtualization also allows random, yet automated, robustness tests in which a program stops the operation of virtual instances of an application to allow testers to observe the effects. Chaos Monkey is one example of this approach.
Testing for Performance
These tests can be automated as well; the question is when to apply them. Earlier in the development cycle, performance test results can give developers hints about the quality of the code being created. However, realistic performance testing may only be possible near the end of development when all the components can be evaluated together and any remaining bottlenecks identified. The answer is often to do both, using automated testing to save time in each case.
Will It Scale?
Increasing numbers of users will show how software performance changes with load. The interaction of input-output modules, queuing systems to hold requests, and mechanisms to solicit more system resources may lead to surprises and non-linear performance curves. Automated simulation of loads at different levels using distributed test scripts (Testing Anywhere allows distribution of EXE files for this) will give results that can be visualized to identify any discontinuities or abrupt degradation with scaling.
The only place where automation does not have a role to play: usability assessment is, naturally, all done in the user’s head. However, best practice rules and recommendations exist for improving usabilityas well. By following these, software presented for usability testing already has a better chance of passing instead of failing.
If you’d like to know how many different ways Testing Anywhere, the automated software testing tool, can help you save time over a whole range of software testing activities, try a free Testing Anywhere trial to see what it can do for you.