Once QA runs the tests and discovers a test that did not pass, then the next step is to send it back to development for analysis. A developer should be assigned the task of analyzing failed acceptance tests. This developer should debug the failed acceptance test if necessary to determine the component that failed. The failed acceptance test should then be forwarded to the manager / group lead or developer responsible for that component.
The first job of the developer is not to try to fix it yet. The first priority is to determine the legitimacy of the test. Here are some things that might not be legitimate:
- The test was not set up or initialized properly.
- The test is using invalid parameters.
- The test calls components in the wrong order.
- The test actually completed successfully, but wrongly reported as failed.
- The test is invalid because it is testing features that are not requirements and not documented anywhere within the system.
So, the first task is to verify that the test is legitimate. If the developer believes that the test is not legitimate for some reason, then he should send it back to QA and specify the reason why the test is not valid. The QA tester should then analyze this reason and make his own decision as to whether or not the test is legitimate. If the QA tester still believes the test to be legitimate, he should meet with the developer to discuss it and determine if a resolution can be found. If after the meeting the two are still at an impasse, then the issue should be escalated to the QA tester’s manager and the developer’s manager for resolution.
If the developer agrees that the test is valid and reflects a bug somewhere within the software, then the next steps are:
- Debug the acceptance test and figure out why the error is occurring.
- Write a new unit test that duplicates the problem.
- Fix the code.
- Run the unit tests and the specific acceptance test that reported the problem.
- Check in the code and report as fixed if the tests all passed.