Executing the (re)tests

Aim

To obtain test results, on the basis of which evaluation of the test object can take place.

Method of operation

The method of operation includes the following subactivities:

1. Executing dynamic explicit tests
2. Executing dynamic implicit tests
3. Executing static tests

1. Executing dynamic explicit tests

In dynamic explicit testing, explicit test cases are executed to obtain information on the property (quality characteristic) or system part under test. Results are obtained by running software and executing operations on the test object. These results are compared in the subsequent activity against the expected results, thus delivering any defects. Dynamic explicit testing is the most usual way of testing. There are two possible types of dynamic explicit testing: 

• Testing on the basis of specified tests created in the Specification phase.
  The specified tests that are created in the Specification phase form the starting point for the tests to be executed here. These may be test scripts containing the test actions and checks or the physical test cases. The test scripts are described in an optimal sequence and form the stepped plan for the test execution. If it has been decided to use tools for automated test execution, then the specified tests are executed with the aid of a test tool. In addition, tests can also take place on the basis of checklists or in another form, as described in building block “Test varieties“. An important condition for a worthwhile dynamic explicit test is that the testers do not deviate from the test cases and execute at least the described test cases. Otherwise, there is no way of guaranteeing that the strategy laid down in the test plan is actually being carried out. 
• Testing on the basis of an exploratory technique.
  With this type, the tester carries out exploratory work during the dynamic explicit test. This means that the tester is examining the application under test piece by piece, thinking about what should be or could be tested (test design) and then does it (test execution). In doing so, the tester is gaining knowledge of the application, considering what should be tested next, testing it, et cetera. The design and subsequent execution of the tests take place in close succession. Possible techniques are “Exploratory Testing” and “Error Guessing“.

These two types of dynamic explicit testing do not exclude each other. In fact, when applied in combination, they can reinforce each other. Reasons for combining the two types may be: 

• During the execution of the specified tests, it is felt that insufficient insight into the quality has been obtained. By now testing exploratory in a number of areas within the test object, this impression can be either substantiated or dispelled. 
• The strategy for a retest may be that only those parts of the test object are tested that have been amended by the programmers. In order to be sure that the unchanged parts still work, they can be subjected to some extra, exploratory, testing. 
• The addition of exploratory testing over and above the specified testing can be useful as a stimulus for creative testing. This could be scheduled, for example, for a Friday afternoon. Many testers are more creative during this part of the week. Just before the weekend, the mood is good and everyone is open to experimentation. These experiments may cover very exceptional situations, but perhaps also those that are so ordinary that they are overlooked. It is then that crucial faults may be found in the test object. 

During the execution of a test script, a fault may occur. This has to be investigated further, before it is reported as a defect. It can be observed whether the defect always occurs or only in the specific situation. Alternatively, perhaps the defect occurs in other (similar) areas in the test object. There is also the possibility that several defects are located together. This investigation can take place on the basis of exploratory testing.

2. Executing dynamic implicit tests

During dynamic explicit testing, information can also be gathered on other properties (quality characteristics). No explicit test cases are designed for these. This is referred to as dynamic implicit testing, and the tests can be executed planned or unplanned. If planned, it is agreed in advance that this is to form an actual part of the test strategy. Testers can then be asked in advance of the test execution to observe a number of characteristics (such as performance or usability) of the test object. This is therefore not based on any targeted test cases. Another way is to question the testers after the execution of the dynamic explicit test. However, there is the risk that, since no specific attention has been paid to these things, wrong information will be given. 

Unplanned implicit testing arises because, during execution of the test, certain things start to catch the attention. It is agreed to observe them more closely. If, for example, regular system breakdown takes place, a decision can be taken as regards reliability. Alternatively, if certain screens do not have an appealing look and feel, something can be said about the usability. 

3. Executing static tests

It is laid down in the test strategy whether static tests should be carried out. In static testing, end products are assessed without any software being run. This test usually consists of the inspection of documentation, such as security procedures, training, manuals, et cetera and is often aided by checklists. On the basis of these, it is attempted to obtain insight into the relevant quality aspect. Here too, any defects are registered and processed by means of the defects procedure. 

Products

Test results.

Techniques

Exploratory Testing
Error Guessing. 

Tools

Testware management tool
Defect management tool
Model-based testing tool
Automated test execution tool
Performance, load and stress test tool
Monitoring tool
Code coverage tool
Comparator
Database manipulation tool
Simulator
Stubs and drivers.