Solid ‘Defect Reporting’ skill is mandatory for any Quality Assurance Engineer, since many of teams are working in a common environment with BA, DEV and Client teams – well and fully described bug – becomes crucial for an agile process and for the project flow.
Here is what I’ve been driven by while creating this “Thumb Rule” document, about 3 years ago I’ve found this ‘Can pig ride’ post and since I’ve never seen any better explanation on approach of creating a good bug report.
Please read it carefully and give it a thought, before we move to some examples.
Taken from:
http://canpigride.blogspot.com/
Defect Remarks
Here are some key points to make sure the next defect report you write is an effective one:
1. Condense – Say it clearly but briefly
2. Accurate – Is it a defect or could it be user error, misunderstanding, etc.?
3. Neutralize – Just the facts. No zingers. No humor. No emotion.
4. Precise – Explicitly, what is the problem?
5. Isolate – What has been done to isolate the problem?
6. Generalize – What has been done to understand how general the problem is?
7. Re-create – What are the essentials in triggering/re-creating this problem? (environment, steps, conditions)
8. Impact – What is the impact to the customer (i.e.: Priority)? What is the impact to test (i.e.: Severity)?
9. Debug – What does development need to make it easier to debug? (traces, dumps, logs, immediate access, etc.)
10. Evidence – What documentation will prove the existence of the error?
It is not just good technical writing skills that leads to effective defect reports. It is more important to make sure that you have asked and answered the right questions. It is key to make sure that you have covered the essential items that will be of most benefit to the intended audience of the defect report.
Once you’ve read it, lets take a more precise look at every point above:
1. Condense – Say it clearly but briefly:
It may sound like a strange advice, though it actually makes sense.
"The exported csv file report from the service while logged in as site admin user - doesn't contain last 10 rows of data if initial data was longer than 100 rows"
Here is an example where all information is represented in one sentence, on 1 hand it clearly shows what is wrong, though on the other hand – reading and understanding something like this – sounds like a challenge, isn’t it?
Here is an example where the same issue is described in a more appropriate way:
"Exporting tables with more than 100 record to CSV - results in deletion of last 10 rows"
It doesn’t contain any information about the page where export is performed and user access rights – while gives a proper understanding of what is wrong, while any additional information can be described in “Steps to reproduce” or “Pre-requisites” sections.
2. Accurate – Is it a defect or could it be a user error, misunderstanding, etc.?
Keeping in mind, that we are in Agile and this is a very fast growing and changing environment, we should be quite precise on what we see and get as a result of test, or if you use a Waterfall model – changes in specification go through many instances before they get into QA team, so it’s always better to double-check if there are any doubts.
So the advice here would sound like: “If you hesitate – discuss found discrepancy with DEV, BA, Tech Lead to confirm found issue – as relevant bug or requirement (logic) change”
3. Neutralize – Just the facts. No zingers. No humor. No emotion.
Some of your comments or notes may hurt someone’s feelings, if there is an issue – it’s definitely not anyone’s fault. It’s nothing more than an unexpected outcome and there is no doubt that everyone in your team is doing their best as part of the job
So, if you’d like to motivate Developer team and gain trust relationship between your teams, it would not be a good idea to write something like:
“And here is the 10th bug carefully delivered by our cool Dev team, BANG !! 🙂 “
4. Precise – Explicitly, what is the problem?
Well, you can believe it or not, but on 1 of the projects I used to work, I’ve run into an issue raised by client during the UAT phase. The problem was that (how we’ve discovered later) once user session expires – web app has been throwing exceptional error and killing browser, though initial description was like:
Steps: 1. I've woke up in the morning, and logged into the system. 2. I've been away from my computer making coffee 3. I haven't found my website and browser was closed Could you take a look at the issue as soon as possible?
What can I say more? What I’ve learned from that is that making coffee and leaving your PC for a while in the morning may cause serious project impact.
Try NOT to describe your defect reports from a 1st person perspective, and be more accurate to describe the problem itself.
Here is a good example on our issue with XML Export:
Steps to reproduce: 1. Navigate to and login as Admin user (test credentials provided) 2. Run a following query (query) within [Search] field 3. Ensure that query returned at least 100 rows 4. From a right-hand side corner click [Export] button 5. Open an exported csv file and review it Actual: Last ten rows of data are deleted from exported file Expected: All data as per query should be displayed in export file
5. Isolate – What has been done to isolate the problem?
Well this one is quite specific and not so common for all bug reports. If during the investigation on a given bug you see a way of a possible workaround for this type of failure – please provide the details to make sure that while bug is being fixed there is still (possibly) a way to use the feature/functionality.
Example (also from a CSV export):
Given bug: [Export] button is not functional (click doesn't call expected action) Though you've spotted that if you'll TAB and get focus on the button - Hit [Return] would work as expected.
6. Generalize – What has been done to understand how general the problem is?
Give a new-born defect a good investigation on what areas it effects. Maybe this one is a failure to your general business flow/process? Try to see an issue from a bigger picture perspective (provide a note on why you state so).
7. Re-create – What are the essentials in triggering/re-creating this problem (environment, steps, conditions)?
An idea of putting your research path / area to a defect report – should be a golden rule.
Ok, you’ve found an issue – is there a web log – to check on error message once it reproduces?
If you can not log in into the app with existing account – is there a DB to check if credentials are correctly stored?
Is your issue specific to any platform / browser version?
With no doubts you all do a proper check around and research – please do not forget to mention all these details – as any information is very useful to Development to reproduce and fix the bug.
Please do not forget a VERY detailed ‘Steps to reproduce‘ section, all dependencies, conditions and environmental information are must have attributes.
8. Impact – What is the impact to the customer (i.e.: Priority)? What is the impact to test (i.e.: Severity)?
This one has pretty much in common with ‘Generalize’ rule. As a QA Engineer, you should clearly understand the nature and impact of newly found bug.
Is it a blocker for further testing? How critical this one is for business?
This rule is about negotiation around ‘Severity’ and ‘Priority’
9. Debug – What does development need to make it easier to debug? (traces, dumps, logs, immediate access, etc.)
As has been said before (rule # 7): any information will be highly useful for faster reaction from Development team
10. Evidence – What documentation will prove the existence of the error?
Adding, even very obvious looking screenshot/log string/trace dump (if possible) with annotated area, is a rule of good manner. Remember that frankly saying – when raising a bug you are actually criticizing someones work (*sarcasm*;)).
Since, all the above looks quite familiar, I still think that it’s worth reading it one more time and keep everything in mind.
Following easy steps listed above will improve / keep your defect reports at a great level.
Here are some materials that might be of a help to evaluate the topic:
Eng:
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
http://itscommonsensestupid.blogspot.com/2008/07/tips-to-write-good-bug-report.html
http://timheuer.com/blog/archive/2011/10/12/anatomy-of-a-good-bug-report.aspx
http://www.softwaretestinghelp.com/how-to-write-good-bug-report/
http://andydowson.blog.com/2012/10/12/how-to-be-a-bug-reporting-superstar/
Rus:
http://habrahabr.ru/post/156069/
http://habrahabr.ru/post/156099/
Xnovo 