Measuring performance based on bugs found

 

Bug reporting and performance measurement


Appraisal season

So it's time for reviews and we're looking back into the last 6 months of what's happened and reviewing them. First and foremost it's a measurement of the performance of the employees to make sure of the following things:

  • Working to the required standard
  • Address key areas to improve
  • Eligibility for bonuses and pay rises
  • Managing changing roles

These appraisals will typically ask you to fill out some sort of form that will answer the following questions

  • To what extent did you meet your goals for the year?
  • What accomplishments are you most proud of?
  • What has been the most challenging part of your role?
  • What do you think you should differently in the next 6 months?

Of course these questions will vary from company to company but these are general questions you would expect. A good appraisal is a two-way process, where you are encouraged to speak honestly and openly about your job. These are not intended to impose disciplinary actions. However, performance issues need to be dealt to keep the business running.For a great resource on appraisals have a look at this

Bugs found and performance measurement

#1 Developer Performance

It will be easy to report that one developer has a massive list of bugs on the ticket that they have been working on. However, the use of the number of reported bugs to make the developer more uncomfortable will mean that that developer and the rest of the developers on your  team will become very unhappy and defensive. They will start doing the saying the following things:

  • Non-reproducible bugs should not be reported 
  • The bug report is incomplete (lacking browser, operating system, configuration)
  • Testing is unfair
  • Tester is incompetent
  • Specifications are incomplete

The 4 points above are all valid, however, as a team member it is your duty to give information on what you have found. The bug report is incomplete, this may be a tricky one since a core principle of agile is being lightweight.

The list described above is small and every minor thing will be mentioned by the whole team. As soon as the bug-tracking system is used for human resources management purposes instead of trying to fulfil the mission and technical purposes, it will be treated that way. 

#2 Tester Performance

When you reward testers based on the number of bugs they report, there will be a lot more bugs reported, Great news right?! Maybe. For example, in a quick day review you've brought of your tester's bug counts, testers will easier to find and more superficial bugs and will be more willing to report multiple instances of the same bug.  Here is an example:

  • When you have an input that should not allow the English Alphabet on an input box, they will now report 26 bug instances from A -> Z
  • A bug that's already in the bug backlog

Another very important item to consider is that they will spend less time coaching inexperienced testers or other team members on testing so they'll have all the glory for themselves. Now this very competent tester is leaving your company and you're left with a less experienced tester, don't let this happen and keep information flowing!

TL:DR: Do not measure performance based on bugs found

Have a look at what I'm reading now  

Happy Testing!

Leave a Reply

Your email address will not be published. Required fields are marked *