Are Bugs in Production a Testers Fault?

Monday, November 03, 2014

Back in the day there was plenty of opportunity for a blame culture to exist in software development projects. Traditional methodologies encouraged rigid process, something along the lines of: develop, build and then finally, test. Inevitably, when following this process, milestones (in the lengthy project plan) were rarely met, meaning ‘slippage’ occurred at various stages, and activities, planned for the later phases of the project, were squeezed for time. This usually meant ‘testing’, so one of two outcomes would occur. Either the testers would have insufficient time to do a comprehensive job, or the go-live date would be delayed.

Typically this created an “us and them” culture between developers and testers, with each blaming problems on the other. In most cases, if bugs were found in production, the first question asked: ‘Was the testing effort sufficient?’. Crazy, we know, and crazier still is that these ‘traditional’ methods are still favoured by a large number of organisations!

An agile revolution

The agile development ecosystem sees developers, testers and the wider business work alongside one another as a single team, with common goals. Furthermore, the democratic approach of agile working sees the team come together to determine the acceptance criteria between them. This means everyone settles on achievable goals and timescales before the work begins. In short, developers know what to develop and testers are aware of what’s awaiting them. The benefit here is typically a vast reduction in defects.

To enshrine this acceptance criteria, the whole thing is written using common domain language and driven by user behaviours, so easily understood by all. Each development ‘task’ is only signed off as ‘done’ when the tester issues a pass.

And when test automation enters the agile process, a great many more tests can be executed. A fully automated regression test pack checks that nothing is broken in the new build, resulting in a much lower risk of new releases hitting the market with bugs present.

How to create an agile ‘Team Heaven’

All of the above may sound like something of a utopia to those still using traditional development methods; as realistically achievable as walking to the moon. By following the below guide, however, the concept of ‘Team Heaven’ could indeed become a reality.

  1. Let the code document itself

    Storing requirements in multiple different systems promotes disparity. Why use a myriad of spreadsheets, logs, databases, feature files and notes, when the process can be made much smoother if everybody had access to a single repository. Move all relevant documentation into the code and serve it at a universal point wherever possible.

  2. Limit long meetings

    Education professionals have long maintained that we switch off much earlier than we sometimes expect. As such, any meetings that need to be undertaken should be done so around a desk, not in a closed meeting room, with strict agendas so only pressing matters regarding agile ceremonies and major design or development decisions are addressed. Communication is essential but anything more than this is unnecessary.

  3. Use the retrospective to improve your process

    Despite the above limitations, it’s worth ensuring that meetings leave all parties confident about progress and certain of where they stand. Make sure your team attend ‘retrospective’ sessions at the end of each phase of work. Everyone should feel comfortable and confident enough to have their say on matters. Understanding and collaboration should bring with it a clear view of what should be done to keep the process improving.

  4. Promote visibility and reviews

    In order to achieve the agile utopia, code visibility, code reviews, testing reviews and process reviews are essential and should be encouraged. This means people, from all across the project, need to take an interest in the disciplines and actions of their peers in different functions. A fresh pair of eyes may identify any issues before they arise.

  5. Encourage collaboration

    Agile working is a much more collaborative, sociable affair. The lines of communication between teams should always be open. Whether this is via wikis or Skype chat (as just two examples), all parties need to get visibility on the whole project. Create an environment where pair programming is possible. Allow one developer to code while the other observes, thinking up new ideas and pre-empting any potential future issues.

  6. Detect bugs earlier with Continuous Integration

    Continuous Integration (CI) is where team members regularly integrate their work, usually on a daily basis. With each integration automated tests are executed to verify the build. If an issue is detected, it’s obvious where to find it, as we know which part of the system was just changed. And the better the test automation, the easier and quicker it is to find the issue.

  7. Its really good to talk

    Converse the old fashioned way – in person! In today’s offices it’s easy to hear little more than the click-clack of keyboards as co-workers speak to one another online. In an agile environment, though, there’s much to be said for simply heading over to a colleague and talking about an issue or suggestion. This also enables those nearby to hear what’s going on and add their own thoughts or opinions. The last thing you want is for your office to sound more like a library!

  8. Make decisions as a team

    Not all decisions need to go through every single person and fewer still will affect everyone. Despite this, it’s still worth ensuring visibility across all parties so, once again, nobody feels out of the loop. For example, your developers may wish to change one of their tools, or a process, for another that will enable them to work more effectively. There’s the argument that the wider team may not need to know this, but to leave them out of the loop may only cause problems later on. Instead, get everybody involved so that, if nothing else, there’s complete visibility.

  9. Don’t be afraid to start again

    Having invested time, effort, money and a great deal of care into a project, ripping it up and starting again can feel like a wrench. Leaving a project too long before scrapping, however, can result in more wasted time, effort and money. Instead, use the opportunity for starting again to learn and improve. If things go wrong, it’s perfectly acceptable to head back to the drawing board to come up with something that will eventually be decidedly better than its first iteration.

  10. The team decides when a feature is complete

    Either the whole team determines when a feature can officially be deemed ‘complete’, or a coalition of representatives from the team (as long as the business, a developer and a tester is present). Each person on the project is given a right to reply before any decisions are made, providing an opportunity for everyone to review each others code, ensuring that the feature is bug free and we have all built the right thing.

So are bugs in production a tester’s fault?

As noted at the beginning, it’s safe to say that finding bugs in production was never a tester’s fault, even though traditional working setups allowed such beliefs to seep through. This was even the case when efforts were restricted, so any issues or concerns hadn’t been given the optimal time or consideration.

Now, in an agile environment, labeling bugs in production as being the fault of testers simply isn’t possible. First and foremost, agile working should result in much fewer bugs being found on production. In the case of such concerns or complaints, though, collaborative working means the whole team shares responsibility. This does mean, of course, that in the case of projects being a resounding success, the entire team can also share the plaudits.

Team Heaven indeed. If you’re transitioning to an agile methodology or have achieved team heaven, we’d love to hear your story!

* Comments subject to moderation